TruthfulQA
Not yet importedQuestions designed to reveal whether models repeat common misconceptions instead of giving accurate, well-supported answers.
2 case previewschatApache-2.0version mc / head
Straightforward import. The multiple-choice track requires no sandbox. Generative parity needs additional judge logic.
What it measures
TruthfulQA targets false beliefs and imitative misinformation. Its multiple-choice track is the clearest first import because it can be scored deterministically.
A later generative track would require the benchmark's judge and informativeness methodology. The initial Trunchbull release should not collapse those distinct evaluation modes.
not yet imported
Not yet imported. Start with the deterministic multiple-choice track; treat generative scoring as a separate release.
Cases
2 previewed| id | case | runtime |
|---|---|---|
| truthfulqa/mc1 | Single true answerChoose the truthful answer from one correct and several incorrect options. | chat |
| truthfulqa/mc2 | Truth probabilityScore a set containing multiple true and false answer choices. | chat |
Tools
none requiredNo model tools. Cases are answered from the prompt and checked by the grader.