Evaluation & benchmark infrastructure
MMLU/GPQA/SWE-bench/ARC-AGI and eval harnesses
Benchmarks (MMLU, GPQA, SWE-bench, ARC-AGI-2, FrontierMath, HLE) and harnesses (lm-eval-harness, HELM, Chatbot Arena/LMArena) are how model capability is measured, purchased and regulated. Benchmark saturation and contamination make eval design and private held-out sets a strategic, contested layer of the ecosystem.
Benchmarks
SWE-bench, ARC-AGI-2, FrontierMath
Arenas
LMArena/Chatbot Arena
How it fits the stack
Evaluation & benchmark infrastructure with what it depends on (above) and what it feeds (below). The figure renders as a crawlable diagram and upgrades to an interactive 3D graph as it scrolls into view.
Evaluation & benchmark infrastructure in the AI stack. Evaluation & benchmark infrastructure with its immediate upstream dependencies (top) and downstream dependents (bottom) in the AI value chain. Hover a node in 3D, or read the full relationships below.
Graph data (text) — 4 entities, 3 relationships
- AI Safety Institutes (US CAISI / UK AISI) —supplies→ Evaluation & benchmark infrastructure
- Claude (Opus / Sonnet) —used by→ Evaluation & benchmark infrastructure
- GPT-5 / o-series —used by→ Evaluation & benchmark infrastructure