Skip to Main Content
Trunchbull
Sign inSign up
PUBLIC REGISTRY
Trunchbull portNo sandbox required

GSM8K

Upstream publisher OpenAIopenai/grade-school-math

A friendly, practical test of whether an AI model can solve multi-step grade-school math word problems without losing track of the numbers.

reasoningmathchat
PROVENANCEIMMUTABLE RELEASE
Upstream source
openai/grade-school-math
Trunchbull port
Not packaged
Version
GSM8K / TEST
Release digest
sha256:5e1c…9c02
License
MIT
Source commit: main-2026-04-08Published July 28, 2026

WHAT THIS PROJECT MEASURES

README

GSM8K contains linguistically diverse grade-school math problems that usually require two to eight steps to solve. It is useful for measuring whether a model can preserve quantities and operations across a short chain of reasoning.

This Trunchbull release exposes a fixed public sampler with an exact numeric-answer grader. The upstream source remains authoritative; the release records the source version and evaluation contract used by the demo.

IN PLAIN ENGLISH

GSM8K is a simple benchmark to understand and a useful one to run: give a model a word problem, let it work through the arithmetic, and check whether the final number is right. Trunchbull CI turns that familiar test into a reproducible release with visible grading.

EXECUTION CONTRACT

Runs in the standard chat lane with no tools or sandbox environment.

QUICK ANSWERS

GSM8K, explained

What does GSM8K measure?

GSM8K measures multi-step mathematical reasoning on grade-school word problems. It is especially useful for spotting arithmetic mistakes, missed quantities, and broken chains of reasoning.

How is GSM8K graded on Trunchbull CI?

The Trunchbull release uses exact numeric-answer grading, so a result passes when its normalized final answer matches the published answer.

TRUNCHBULL AVAILABILITY

Import status

Trunchbull port

Trunchbull maintains a runnable twenty-case demo port with a pinned numeric-answer grader.

3 CASES

Evaluation cases

3

gsm8k/natalia-clips

Natalia's clips

Chat lane

Multi-step addition after calculating a fractional value.

gsm8k/robe-fabric

Fabric for robes

Chat lane

Unit multiplication and subtraction across two quantities.

gsm8k/duck-eggs

Duck eggs

Chat lane

Daily production, household use, and revenue calculation.

0 REQUESTED TOOLS

Tools

0

No model tools required.

Evaluation happens through the prompt and grader contract for this release.

MACHINE-READABLE PROVENANCE

Registry record

{
  "name": "GSM8K",
  "catalogOwner": null,
  "source": {
    "url": "https://github.com/openai/grade-school-math",
    "version": "test split · fixed sampler",
    "commit": "main-2026-04-08",
    "license": "MIT"
  },
  "port": null,
  "release": {
    "digest": "sha256:5e1c…9c02",
    "lane": "chat",
    "sandboxRequired": false
  },
  "cases": [
    "gsm8k/natalia-clips",
    "gsm8k/robe-fabric",
    "gsm8k/duck-eggs"
  ],
  "tools": []
}

BUILD ON THIS PROJECT

Bring this benchmark into your workspace.

Sign up