Registry
Official ports and the upstream projects behind them. Check the source, cases and runtime of a benchmark before you run it on your own models.
| benchmark |
|---|
GSM8KA friendly, practical test of whether an AI model can solve multi-step grade-school math word problems without losing track of the numbers. |
SkateBenchA playful, focused test of whether AI understands skateboard tricks and can follow a precise product recommendation task. |
ARC-ChallengeA clear, no-frills test of science reasoning on challenging grade-school questions with multiple-choice answers. |
Medical AI Failure AtlasA safety-first benchmark for finding dangerous medical AI behavior: missed escalation, unsafe medication advice, false reassurance, and overconfident claims. |
MIHENKA bilingual Turkish-English reasoning benchmark spanning 20 disciplines, four difficulty tiers, and both multiple-choice and short-answer tasks. |
TruthfulQA MC1A multiple-choice benchmark that checks whether AI models choose truthful answers instead of repeating popular misconceptions. |
Berkeley Function Calling LeaderboardExecutable evaluation of function selection, arguments, parallel calls, and multi-turn tool use. |
TerminalBench 2.0Evaluates whether AI agents can autonomously complete 89 hard, realistic, end-to-end tasks in containerized terminal environments, using task-specific verifiers. |
HumanEvalHand-written Python programming problems evaluated by executing generated code against tests. |
SWE-bench VerifiedA human-validated set of real GitHub issues requiring repository patches that pass tests. |
ParseBenchDocument parsing evaluated across tables, charts, content faithfulness, semantic formatting, and visual grounding. |
LlamaIndex ExtractBenchSchema-guided extraction from enterprise documents. Distinct from the Contextual AI benchmark of the same name. |
ExtractBench ProHard document extraction across conventions, vocabulary, counting, drawn relations, charts, and long chart sheets. |
IFEvalVerifiable instruction-following tasks that test whether a model can satisfy precise format, length, language, and content constraints. |
MMLUMultiple-choice questions across academic and professional subjects, from elementary mathematics to law and medicine. |
HellaSwagCommonsense completion tasks that ask a model to choose the most plausible continuation of an everyday situation. |
BoolQNaturally occurring yes-or-no questions answered from short passages, designed to test inference beyond keyword overlap. |
PIQATwo-choice physical commonsense problems about how everyday goals can be completed in the real world. |
WinoGrandeLarge-scale pronoun-resolution problems that require commonsense reasoning to resolve an ambiguous reference. |
TruthfulQAQuestions designed to reveal whether models repeat common misconceptions instead of giving accurate, well-supported answers. |
RuneBenchCoding-agent tasks inside an emulated RuneScape world, scored by skill progression and resource collection over fixed time windows. |
CrafterA lightweight visual survival environment that measures exploration, resource management, crafting, and long-horizon planning. |
BALROGA shared evaluation framework for language and visual agents across long-horizon reinforcement-learning game environments. |
TextWorldGenerated text-adventure environments for planning, memory, exploration, and language grounding. |
VideoGameBenchVisual computer-use evaluation across Game Boy, DOS, and browser games, with a paused Lite mode for latency-fair comparison. |
MiniHackShort NetHack-derived tasks for navigation, exploration, combat, and object interaction under controlled episode budgets. |
NetHack Learning EnvironmentA standardized interface to a procedurally generated, partially observed, system-rich, and extremely long-horizon game. |
MineDojoThousands of Minecraft tasks spanning survival, crafting, exploration, construction, and open-ended embodied play. |
VizDoomProgrammatic Doom scenarios for visual navigation, perception, planning, and combat. |
Pokémon ShowdownCompetitive turn-based battles with structured state, legal-action constraints, and objective match outcomes. |
Want to see a benchmark run first? Public demos need no account.
Try a public demo