SWE-bench Verified
Not yet importedA human-validated set of real GitHub issues requiring repository patches that pass tests.
2 task previewsagent · sandboxedMITversion verified / 500
Start free
Complex import. Requires repository checkout, containerized execution, patch capture, and test grading.
What it measures
SWE-bench Verified evaluates whether an agent can understand an issue, navigate a real repository, produce a patch, and pass the relevant tests.
Trunchbull indexes the upstream project but does not yet maintain a runnable port. Each case needs an immutable repository snapshot, task-specific environment, patch capture, and test verifier.
not yet imported
Trunchbull has not ported this project yet. A port must package repositories, environments, and test-based verification into immutable releases.
Tasks
2 previewed| id | task | runtime |
|---|---|---|
| swe-bench/django-11001 | Django issue #11001Inspect a pinned repository and produce a testable patch.tools: sandbox.files, sandbox.commands | sandbox |
| swe-bench/sympy-13031 | SymPy issue #13031Repair symbolic behavior in a pinned source checkout.tools: sandbox.files, sandbox.commands | sandbox |
Tools
2 requested| tool | description | kind | required |
|---|---|---|---|
| sandbox.files | Reads and modifies the pinned repository checkout. | sandbox | yes |
| sandbox.commands | Runs repository tests and captures verifier evidence. | sandbox | yes |