MMLU
Not yet importedMultiple-choice questions across academic and professional subjects, from elementary mathematics to law and medicine.
2 case previewschatSource termsversion test / head
Straightforward import. Prompt-and-label import with exact answer-choice grading. No sandbox or model tools required.
What it measures
MMLU is a broad knowledge and reasoning benchmark organized into subject-level datasets. A useful release must preserve subject labels, answer-choice ordering, split provenance, and the exact scoring convention.
This is a comparatively simple registry import: static prompts, fixed labels, no model tools, and no execution environment. Trunchbull has not yet compiled an immutable release.
not yet imported
Not yet imported. The first release needs pinned subject files, deterministic prompt rendering, and exact answer-choice grading.
Cases
2 previewed| id | case | runtime |
|---|---|---|
| mmlu/high-school-biology | High school biologySubject-level multiple-choice knowledge and reasoning questions. | chat |
| mmlu/professional-law | Professional lawProfessional-domain questions with a fixed four-choice answer set. | chat |
Tools
none requiredNo model tools. Cases are answered from the prompt and checked by the grader.