IFEval
Not yet importedVerifiable instruction-following tasks that test whether a model can satisfy precise format, length, language, and content constraints.
2 case previewschatApache-2.0version master
Moderate import. No sandbox is required, but a faithful import needs the upstream constraint verifiers rather than a generic text grader.
What it measures
IFEval turns instruction following into something that can be checked rather than judged by impression. Each prompt carries one or more constraints with deterministic verification logic.
Trunchbull has indexed the upstream project but has not produced a faithful release. Importing the prompts is straightforward; parity depends on packaging the specialized constraint graders and recording their per-constraint results.
not yet imported
Not yet imported. Trunchbull needs to package the upstream constraint registry and expose per-instruction verification evidence.
Cases
2 previewed| id | case | runtime |
|---|---|---|
| ifeval/keywords | Required keywordsProduce a response that includes a fixed set of required terms. | chat |
| ifeval/format | Exact response formatFollow structural constraints such as sections, bullets, or delimiters. | chat |
Tools
none requiredNo model tools. Cases are answered from the prompt and checked by the grader.