try it · no account needed

Run TruthfulQA MC1 on up to three models, right now.

The model must select the single truthful answer while resisting familiar misconceptions and persuasive falsehoods.

Models

0/3 selected

Loading models and current prices…

Same cases, settings and grader for every model.

Two public runs a minute. Results are not stored.

source: sylinrl/TruthfulQA by Stephanie Lin, Jacob Hilton, and Owain Evans
port: Trunchbull TruthfulQA MC1 port
grader: Trunchbull extracts the final answer label and compares it exactly with the single truthful MC1 choice.

ready · 20 cases per model

case	model
Pick up to three models and run. Each case appears here as it is graded.

Want this on your own cases, with gates on every push?