Not yet importedNo sandbox required

HellaSwag

Upstream publisher Allen Institute for AIrowanz/hellaswag

Commonsense completion tasks that ask a model to choose the most plausible continuation of an everyday situation.

commonsensemultiple-choicechat

PROVENANCEUPSTREAM RECORD

Upstream source: rowanz/hellaswag
Trunchbull port: Not packaged
Version: DATA / HEAD
Release digest: Not compiled
License: Source terms

Source commit: Upstream head not pinnedPublished Indexed July 30, 2026

WHAT THIS PROJECT MEASURES

README

HellaSwag evaluates grounded commonsense by pairing a short context with several candidate endings. Adversarial filtering makes incorrect choices plausible enough to defeat shallow matching.

The benchmark fits Trunchbull's basic multiple-choice contract and does not need tools or a sandbox. The import work is primarily provenance, prompt normalization, and exact label scoring.

EXECUTION CONTRACT

Static prompt-and-label evaluation with no runtime dependencies.

TRUNCHBULL AVAILABILITY

Import status

Not yet imported

Not yet imported. A first release needs the dataset pinned, contexts normalized, and answer ordering preserved.

Port this benchmark

2 CASES

Evaluation cases

hellaswag/activitynet

Activity continuation

Chat lane

Choose the most plausible next step in a described human activity.

hellaswag/wikihow

Procedure continuation

Chat lane

Select the ending that coherently completes a procedural context.

0 REQUESTED TOOLS

Tools

No model tools required.

Evaluation happens through the prompt and grader contract for this release.

MACHINE-READABLE PROVENANCE

Registry record

{
  "name": "HellaSwag",
  "catalogOwner": null,
  "source": {
    "url": "https://github.com/rowanz/hellaswag",
    "version": "Upstream dataset",
    "commit": "Upstream head not pinned",
    "license": "Source terms"
  },
  "port": null,
  "release": {
    "digest": "Not compiled",
    "lane": "chat",
    "sandboxRequired": false
  },
  "cases": [
    "hellaswag/activitynet",
    "hellaswag/wikihow"
  ],
  "tools": []
}

BUILD ON THIS PROJECT