HumanEval
Not yet importedHand-written Python programming problems evaluated by executing generated code against tests.
2 task previewsagent · sandboxedMITversion master / 2021
Start free
Moderate import. Requires isolated execution of untrusted generated Python code.
What it measures
HumanEval asks a model to complete Python functions from a signature and docstring. Generated code is executed against tests, making isolation part of the evaluation contract.
Trunchbull indexes the public project and its provenance, but has not published a runnable port. A port must define a safe sandbox environment and verifier without exposing protected tests.
not yet imported
Trunchbull has not ported this project yet. Porting requires a sandbox-safe execution contract and public release manifest.
Tasks
2 previewed| id | task | runtime |
|---|---|---|
| humaneval/0 | has_close_elementsComplete a function that detects nearby values in a list.tools: python sandbox | sandbox |
| humaneval/1 | separate_paren_groupsParse and separate balanced parenthesis groups.tools: python sandbox | sandbox |
Tools
1 requested| tool | description | kind | required |
|---|---|---|---|
| python sandbox | Executes generated Python against a bounded verifier. | sandbox | yes |