ExtractBench Pro

Not yet imported

Hard document extraction across conventions, vocabulary, counting, drawn relations, charts, and long chart sheets.

100 documentsApache-2.0version 2026-10-15

Listed, not imported — scores come from LlamaIndex's official leaderboard.

View official leaderboard ↗

From the authors

Leaderboard

Sorted by the selected metric, then value F1, then lower cost. Failed or timed-out documents score zero.

upstream ↗ @ 741c1b6 · captured 2026-10-20

Showing perfect documents; higher is better. 12 systems.

The CSV publishes one result per system, with no separate no-limit results. Grounding metrics use the grounding-document subset; counts are shown per system.

Perfect documents leaderboard, source: upstream
#	system	perfect documents ↓	conventions22	vocabulary14	counting12	relations20	charts25	chart sheets7	¢ / page
1	O Codex GPT-6 AstraOpenAI · coding agent100 documents · 81 with grounding	58.0%	27.3%	64.3%	58.3%	80.0%	64.0%	57.1%	95.20
2	A Claude Code Opus 5.5Anthropic · coding agent100 documents · 81 with grounding	51.0%	22.7%	35.7%	33.3%	80.0%	80.0%	14.3%	41.20
3	L LlamaExtract Agentic PlusLlamaIndex · extraction API100 documents · 81 with grounding	38.0%	22.7%	64.3%	41.7%	10.0%	68.0%	0.0%	7.50
4	E Extend OperatorExtend · extraction API100 documents · 81 with grounding	34.0%	27.3%	64.3%	16.7%	0.0%	68.0%	0.0%	9.88
5	A Claude Code Sonnet 5.5Anthropic · coding agent100 documents · 81 with grounding	32.0%	22.7%	7.1%	33.3%	55.0%	44.0%	0.0%	19.98
6	L LlamaExtract AgenticLlamaIndex · extraction API100 documents · 81 with grounding	20.0%	13.6%	35.7%	16.7%	10.0%	32.0%	0.0%	3.12
7	P PulsePulse · extraction API100 documents · 81 with grounding	18.0%	9.1%	50.0%	25.0%	15.0%	12.0%	0.0%	10.50

challenge score0100Bold marks the best score in each challenge.

Quality against costPerfect documents by cost per page. Systems without positive reported cost are omitted.

coding agent
extraction API

Head to head

First system

Second system

OCodex GPT-6 Astra
AClaude Code Opus 5.5

What it measures

Released October 15, 2026, ExtractBench Pro contains 100 documents across six challenges. It uses LlamaIndex's ExtractBench evaluator and tests answers that require interpreting document structure and conventions.