Not yet importedNo sandbox required

TruthfulQA

Upstream publisher TruthfulAIsylinrl/TruthfulQA

Questions designed to reveal whether models repeat common misconceptions instead of giving accurate, well-supported answers.

truthfulnesssafetymultiple-choice

PROVENANCEUPSTREAM RECORD

Upstream source: sylinrl/TruthfulQA
Trunchbull port: Not packaged
Version: MC / HEAD
Release digest: Not compiled
License: Apache-2.0

Source commit: Upstream head not pinnedPublished Indexed July 30, 2026

WHAT THIS PROJECT MEASURES

README

TruthfulQA targets false beliefs and imitative misinformation. Its multiple-choice track is the clearest first import because it can be scored deterministically.

A later generative track would require the benchmark's judge and informativeness methodology. The initial Trunchbull release should not collapse those distinct evaluation modes.

EXECUTION CONTRACT

The multiple-choice track requires no sandbox. Generative parity needs additional judge logic.

TRUNCHBULL AVAILABILITY

Import status

Not yet imported

Not yet imported. Start with the deterministic multiple-choice track; treat generative scoring as a separate release.

Port this benchmark

2 CASES

Evaluation cases

truthfulqa/mc1

Single true answer

Chat lane

Choose the truthful answer from one correct and several incorrect options.

truthfulqa/mc2

Truth probability

Chat lane

Score a set containing multiple true and false answer choices.

0 REQUESTED TOOLS

Tools

No model tools required.

Evaluation happens through the prompt and grader contract for this release.

MACHINE-READABLE PROVENANCE

Registry record

{
  "name": "TruthfulQA",
  "catalogOwner": null,
  "source": {
    "url": "https://github.com/sylinrl/TruthfulQA",
    "version": "Multiple-choice track",
    "commit": "Upstream head not pinned",
    "license": "Apache-2.0"
  },
  "port": null,
  "release": {
    "digest": "Not compiled",
    "lane": "chat",
    "sandboxRequired": false
  },
  "cases": [
    "truthfulqa/mc1",
    "truthfulqa/mc2"
  ],
  "tools": []
}

BUILD ON THIS PROJECT