[ 04 — DC Model Index ]
Choose with evidence. Build with confidence.
Compare model capability, cost and current releases. Inspect original benchmark results, source coverage, domain scores and workload costs. Compare many models with one reference before making a choice.
Update delayed · last successful import · 437 models · Sources & methodology ↗
Power combines four source families: 35% Coding, 35% Reasoning and 30% Agents. Only complete fixed panels qualify for Power, Eco and Overall. Partial domain scores are labelled with coverage; they do not establish a win against a complete panel. Compare the underlying tests or inspect every formula.
Inspect the evidence
Many models. One reference.
Compare original test results, not just a league position. Green + and red − show a numerical difference for the same named benchmark, effort and harness. Equal effort names can still use different compute budgets. Unknown setups have no coloured delta.
114 benchmark/setup rows. Up to 12 models, including the reference. “—” means missing evidence; “conflict” means no unambiguous observation. Click a score for source details. Scroll horizontally for more models.
Eco & Overall: change your workload assumptions
Only the scenario rows below change. Standard ranking: 3,000 uncached input + 1,000 output tokens, 75% quality. Reasoning tokens, tool calls, retries, cache writes and long-context surcharges are excluded. This estimates token spend, not cost per successful task. Unknown cache prices leave a cached scenario empty.
| Benchmark / setup | Mistral Small 3.1 24BReference | Claude Fable 5.1anthropic | GPT-6 Astraopenai | GPT-6.1 Terraopenai | Kimi K3.1 Promoonshotai | Gemini 4 Argongoogle |
|---|---|---|---|---|---|---|
| DC PowerFixed multi-source panel · score points | —0% fixed panel · Power withheldInspect Mistral Small 3.1 24B: weights, inputs & missing evidenceFixed-panel coverage: 0%. Power, Eco and Overall are withheld until every panel is available. Partial domain values use only available panels and are not directly comparable with complete domain scores. ECI contributes 0%. Coding: — · 35% of Power · 0% coverage · PARTIALLiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · — Missing: LiveBench 2 Coding · 20261025: no eligible original observation Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · — Missing: MirrorCode: no eligible original observation Reasoning: — · 35% of Power · 0% coverage · PARTIALLiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · — Missing: LiveBench 2 Reasoning · 20261025: no eligible original observation Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · — Missing: Chess Puzzles: no eligible original observation Missing: FrontierMath-Tiers-1-3-v2-Private: no eligible original observation Missing: OTIS Mock AIME 2024-2025: no eligible original observation ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · — Missing: ARC-AGI-2 · semi-private: no eligible original observation Agents: — · 30% of Power · 0% coverage · PARTIALLiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · — Missing: LiveBench 2 Long Agentic · 20261025: no eligible original observation Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · — Missing: Terminal-Bench 4.0: no eligible original observation Power — · incomplete fixed panel. Missing scores are not zero. | 80.9100% fixed panel · 4 source familiesΔ unavailableInspect Claude Fable 5.1: weights, inputs & missing evidenceFixed-panel coverage: 100%. All ranked models use this complete panel. ECI contributes 0%. Coding: 85.5 · 35% of Power · 100% coverageLiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · 97.7 LiveBench 2 Coding · 20261025: 97.70% ↗claude-fable-5.1 · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · 73.3 MirrorCode: 73.30% ↗claude-fable-5-1_high · Epoch Inspect · Epoch AI evaluation Reasoning: 87.5 · 35% of Power · 100% coverageLiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · 91.7 LiveBench 2 Reasoning · 20261025: 91.70% ↗claude-fable-5.1 · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · 79.4 Chess Puzzles: 48.00% ↗claude-fable-5-1_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel FrontierMath-Tiers-1-3-v2-Private: 90.18% ↗claude-fable-5-1_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel OTIS Mock AIME 2024-2025: 100.00% ↗claude-fable-5-1_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · 90.0 ARC-AGI-2 · semi-private: 90.00% ↗Claude Fable 5.1 (Max) · ARC Prize leaderboard baseline · ARC Prize official leaderboard Agents: 67.9 · 30% of Power · 100% coverageLiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · 77.9 LiveBench 2 Long Agentic · 20261025: 77.90% ↗claude-fable-5.1 · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · 57.9 Terminal-Bench 4.0: 57.88% ↗Claude Fable 5.1 · Claude Code · max · Terminal-Bench 4.0 · Claude Code · Terminal-Bench public leaderboard Power = 0.35 × 85.5 + 0.35 × 87.5 + 0.30 × 67.9 = 80.9. Full precision is used before final rounding. | 77.2100% fixed panel · 4 source familiesΔ unavailableInspect GPT-6 Astra: weights, inputs & missing evidenceFixed-panel coverage: 100%. All ranked models use this complete panel. ECI contributes 0%. Coding: 70.2 · 35% of Power · 100% coverageLiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · 93.7 LiveBench 2 Coding · 20261025: 93.70% ↗gpt-6-astra · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · 46.7 MirrorCode: 46.70% ↗gpt-6-astra_high · Epoch Inspect · Epoch AI evaluation Reasoning: 92.3 · 35% of Power · 100% coverageLiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · 92.7 LiveBench 2 Reasoning · 20261025: 92.70% ↗gpt-6-astra · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · 89.2 Chess Puzzles: 74.00% ↗gpt-6-astra_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel FrontierMath-Tiers-1-3-v2-Private: 93.68% ↗gpt-6-astra_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel OTIS Mock AIME 2024-2025: 100.00% ↗gpt-6-astra_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · 95.0 ARC-AGI-2 · semi-private: 95.00% ↗GPT-6 Astra (Max) · ARC Prize leaderboard baseline · ARC Prize official leaderboard Agents: 67.8 · 30% of Power · 100% coverageLiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · 77.5 LiveBench 2 Long Agentic · 20261025: 77.50% ↗gpt-6-astra · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · 58.2 Terminal-Bench 4.0: 58.18% ↗GPT-6 Astra · Codex · max · Terminal-Bench 4.0 · Codex · Terminal-Bench public leaderboard Power = 0.35 × 70.2 + 0.35 × 92.3 + 0.30 × 67.8 = 77.2. Full precision is used before final rounding. | —46.5% fixed panel · Power withheldΔ unavailableInspect GPT-6.1 Terra: weights, inputs & missing evidenceFixed-panel coverage: 46.5%. Power, Eco and Overall are withheld until every panel is available. Partial domain values use only available panels and are not directly comparable with complete domain scores. ECI contributes 0%. Coding: 94.3 · 35% of Power · 50% coverage · PARTIALLiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · 94.3 LiveBench 2 Coding · 20261025: 94.30% ↗gpt-6.1-terra · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · — Missing: MirrorCode: no eligible original observation Reasoning: 88.7 · 35% of Power · 40% coverage · PARTIALLiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · 88.7 LiveBench 2 Reasoning · 20261025: 88.70% ↗gpt-6.1-terra · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · — FrontierMath-Tiers-1-3-v2-Private: 90.88% ↗gpt-6.1-terra_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel OTIS Mock AIME 2024-2025: 100.00% ↗gpt-6.1-terra_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel Missing: Chess Puzzles: no eligible original observation ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · — Missing: ARC-AGI-2 · semi-private: no eligible original observation Agents: 48.6 · 30% of Power · 50% coverage · PARTIALLiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · 48.6 LiveBench 2 Long Agentic · 20261025: 48.60% ↗gpt-6.1-terra · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · — Missing: Terminal-Bench 4.0: no eligible original observation Power — · incomplete fixed panel. Missing scores are not zero. | —57% fixed panel · Power withheldΔ unavailableInspect Kimi K3.1 Pro: weights, inputs & missing evidenceFixed-panel coverage: 57%. Power, Eco and Overall are withheld until every panel is available. Partial domain values use only available panels and are not directly comparable with complete domain scores. ECI contributes 0%. Coding: 81.8 · 35% of Power · 50% coverage · PARTIALLiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · 81.8 LiveBench 2 Coding · 20261025: 81.80% ↗kimi-k3.1-pro · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · — Missing: MirrorCode: no eligible original observation Reasoning: 84.0 · 35% of Power · 70% coverage · PARTIALLiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · 91.1 LiveBench 2 Reasoning · 20261025: 91.10% ↗kimi-k3.1-pro · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · 74.4 Chess Puzzles: 45.00% ↗kimi-k3.1-pro-together_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel FrontierMath-Tiers-1-3-v2-Private: 78.25% ↗kimi-k3.1-pro-together_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel OTIS Mock AIME 2024-2025: 100.00% ↗kimi-k3.1-pro-together_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · — Missing: ARC-AGI-2 · semi-private: no eligible original observation Agents: 39.1 · 30% of Power · 50% coverage · PARTIALLiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · 39.1 LiveBench 2 Long Agentic · 20261025: 39.10% ↗kimi-k3.1-pro · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · — Missing: Terminal-Bench 4.0: no eligible original observation Power — · incomplete fixed panel. Missing scores are not zero. | —85% fixed panel · Power withheldΔ unavailableInspect Gemini 4 Argon: weights, inputs & missing evidenceFixed-panel coverage: 85%. Power, Eco and Overall are withheld until every panel is available. Partial domain values use only available panels and are not directly comparable with complete domain scores. ECI contributes 0%. Coding: 74.3 · 35% of Power · 100% coverageLiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · 89.7 LiveBench 2 Coding · 20261025: 89.70% ↗gemini-4-argon · high · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · 58.9 MirrorCode: 58.89% ↗gemini-4-argon_high · Epoch Inspect · Epoch AI evaluation Reasoning: 89.9 · 35% of Power · 100% coverageLiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · 95.1 LiveBench 2 Reasoning · 20261025: 95.10% ↗gemini-4-argon · high · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · 81.7 Chess Puzzles: 59.00% ↗gemini-4-argon_high · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel FrontierMath-Tiers-1-3-v2-Private: 85.96% ↗gemini-4-argon_high · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel OTIS Mock AIME 2024-2025: 100.00% ↗gemini-4-argon_high · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · 91.3 ARC-AGI-2 · semi-private: 91.25% ↗Gemini 4 Argon (High) · ARC Prize leaderboard baseline · ARC Prize official leaderboard Agents: 61.2 · 30% of Power · 50% coverage · PARTIALLiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · 61.2 LiveBench 2 Long Agentic · 20261025: 61.20% ↗gemini-4-argon · high · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · — Missing: Terminal-Bench 4.0: no eligible original observation Power — · incomplete fixed panel. Missing scores are not zero. |
| DC CodingPartial panels are not directly comparable | —0% panel · PARTIAL | 85.5100% panelIncomplete panel · no aggregate delta | 70.2100% panelIncomplete panel · no aggregate delta | 94.350% panel · PARTIALIncomplete panel · no aggregate delta | 81.850% panel · PARTIALIncomplete panel · no aggregate delta | 74.3100% panelIncomplete panel · no aggregate delta |
| DC ReasoningPartial panels are not directly comparable | —0% panel · PARTIAL | 87.5100% panelIncomplete panel · no aggregate delta | 92.3100% panelIncomplete panel · no aggregate delta | 88.740% panel · PARTIALIncomplete panel · no aggregate delta | 84.070% panel · PARTIALIncomplete panel · no aggregate delta | 89.9100% panelIncomplete panel · no aggregate delta |
| DC AgentsPartial panels are not directly comparable | —0% panel · PARTIAL | 67.9100% panelIncomplete panel · no aggregate delta | 67.8100% panelIncomplete panel · no aggregate delta | 48.650% panel · PARTIALIncomplete panel · no aggregate delta | 39.150% panel · PARTIALIncomplete panel · no aggregate delta | 61.250% panel · PARTIALIncomplete panel · no aggregate delta |
| Scenario token cost (USD)Lower is cheaper | $0.001608 | $0.08 | $0.08 | $0.008 | $0.016 | $0.016 |
| Scenario DC ecoConditional estimate; not a measured outcome | — | 9.0 | 8.6 | — | — | — |
| Scenario DC overallConditional estimate; not a measured outcome | — | 62.9 | 60.1 | — | — | — |
| Evidence coverage | 0 ECI tests · 0 raw benchmarks 0 Epoch runs · 0 publisher rows · 0 external rows | 22 ECI tests · 34 raw benchmarks 13 Epoch runs · 10 publisher rows · 43 external rows | 24 ECI tests · 36 raw benchmarks 20 Epoch runs · 11 publisher rows · 46 external rows | 10 ECI tests · 17 raw benchmarks 11 Epoch runs · 0 publisher rows · 8 external rows | 0 ECI tests · 13 raw benchmarks 8 Epoch runs · 0 publisher rows · 2 external rows | 20 ECI tests · 28 raw benchmarks 11 Epoch runs · 3 publisher rows · 24 external rows |
| ARC-AGI-2 · semi-privateReasoning · max ARC Prize leaderboard baseline ARC Prize official | — | 90.00%Δ unavailableClaude Fable 5.1 (Max) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $4.492349 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | 95.00%Δ unavailableGPT-6 Astra (Max) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $1.119468 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | — | — | — |
| ARC-AGI-2 · semi-privateReasoning · xhigh ARC Prize leaderboard baseline ARC Prize official | — | 90.00%Δ unavailableClaude Fable 5.1 (XHigh) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $3.120217 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | 93.33%Δ unavailableGPT-6 Astra (XHigh) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.828686 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | — | — | — |
| ARC-AGI-2 · semi-privateReasoning · high ARC Prize leaderboard baseline ARC Prize official | — | 88.75%Δ unavailableClaude Fable 5.1 (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $1.667306 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | 92.08%Δ unavailableGPT-6 Astra (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.668079 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | — | — | 91.25%Δ unavailableGemini 4 Argon (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.324876 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize |
| ARC-AGI-2 · semi-privateReasoning · medium ARC Prize leaderboard baseline ARC Prize official | — | 86.25%Δ unavailableClaude Fable 5.1 (Medium) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $1.223941 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | 92.08%Δ unavailableGPT-6 Astra (Medium) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.480093 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | — | — | 87.08%Δ unavailableGemini 4 Argon (Medium) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.137323 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize |
| ARC-AGI-2 · semi-privateReasoning · low ARC Prize leaderboard baseline ARC Prize official | — | 78.33%Δ unavailableClaude Fable 5.1 (Low) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.952327 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | 85.42%Δ unavailableGPT-6 Astra (Low) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.416242 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | — | — | 42.08%Δ unavailableGemini 4 Argon (Low) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.073811 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize |
| ARC-AGI-2 · semi-privateReasoning · none ARC Prize leaderboard baseline ARC Prize official | — | — | 59.58%Δ unavailableGPT-6 Astra (None) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.369674 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | — | — | — |
| ARC-AGI-2Reasoning · max Not reported Epoch bundle · settings unverified | — | 90.00%Setup unverified · no deltaclaude-fable-5-1_max External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | 95.00%Setup unverified · no deltagpt-6-astra_max External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| ARC-AGI-2Reasoning · xhigh Not reported Epoch bundle · settings unverified | — | 90.00%Setup unverified · no deltaclaude-fable-5-1_xhigh External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | 93.33%Setup unverified · no deltagpt-6-astra_xhigh External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| ARC-AGI-2Reasoning · high Not reported Epoch bundle · settings unverified | — | 88.75%Setup unverified · no deltaclaude-fable-5-1_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | 92.08%Setup unverified · no deltagpt-6-astra_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | — | — | 91.25%Setup unverified · no deltagemini-4-argon_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 |
| ARC-AGI-2Reasoning · medium Not reported Epoch bundle · settings unverified | — | 86.25%Setup unverified · no deltaclaude-fable-5-1_medium External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | 92.08%Setup unverified · no deltagpt-6-astra_medium External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | — | — | 87.08%Setup unverified · no deltagemini-4-argon_medium External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 |
| ARC-AGI-2Reasoning · low Not reported Epoch bundle · settings unverified | — | 78.33%Setup unverified · no deltaclaude-fable-5-1_low External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | 85.42%Setup unverified · no deltagpt-6-astra_low External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | — | — | 42.08%Setup unverified · no deltagemini-4-argon_low External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 |
| ARC-AGI-2Reasoning · none Not reported Epoch bundle · settings unverified | — | — | 59.58%Setup unverified · no deltagpt-6-astra_none External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| GPQA diamondKnowledge · max Epoch Inspect Epoch bundle | — | — | 95.77%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · gpqa_diamond.csv Reported standard error: ±1.37 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 94.32%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · gpqa_diamond.csv Reported standard error: ±1.49 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 94.19%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · gpqa_diamond.csv Reported standard error: ±1.44 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| GPQA diamondKnowledge · high Epoch Inspect Epoch bundle | — | — | — | — | — | 96.40%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · gpqa_diamond.csv Reported standard error: ±1.19 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| MirrorCodeCoding · high Epoch Inspect Epoch bundle | — | 73.30%Δ unavailableclaude-fable-5-1_high Epoch AI evaluation · epoch-run Run: 2026-09-10 · mirrorcode.csv Reported standard error: ±9.17 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 46.70%Δ unavailablegpt-6-astra_high Epoch AI evaluation · epoch-run Run: 2026-08-30 · mirrorcode.csv Reported standard error: ±11.86 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | 58.89%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-14 · mirrorcode.csv Reported standard error: ±11.71 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| FrontierSWECoding · max proximus Epoch bundle | — | 56.29%Δ unavailableclaude-fable-5-1_max https://www.frontierswe.com/ · external Run: Date not supplied · frontierswe_external.csv FrontierSWE V2 (mean@5 across 34 tasks, proximus harness, max reasoning effort). Site footnote: Claude Opus 5 used as fallback for tasks blocked by content filters. Inspect source ↗CC BY 4.0 | 65.51%Δ unavailablegpt-6-astra_max https://www.frontierswe.com/ · external Run: Date not supplied · frontierswe_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierSWECoding · high proximus Epoch bundle | — | — | — | — | — | 54.96%Δ unavailablegemini-4-argon_high https://www.frontierswe.com/ · external Run: Date not supplied · frontierswe_external.csv Inspect source ↗CC BY 4.0 |
| APEX-AgentsAgents · unknown Not reported Epoch bundle · settings unverified | — | 68.60%Setup unverified · no deltaclaude-fable-5-1_unknown External result; evaluator not specified · external Run: Date not supplied · apex_agents_external.csv Inspect source ↗CC BY 4.0 | 64.70%Setup unverified · no deltagpt-6-astra_unknown External result; evaluator not specified · external Run: Date not supplied · apex_agents_external.csv Inspect source ↗CC BY 4.0 | — | — | 82.40%Setup unverified · no deltagemini-4-argon_unknown External result; evaluator not specified · external Run: Date not supplied · apex_agents_external.csv Inspect source ↗CC BY 4.0 |
| APEX-AgentsAgents · max Not reported Epoch bundle · settings unverified | — | — | — | 68.30%Setup unverified · no deltagpt-6.1-terra_max External result; evaluator not specified · external Run: Date not supplied · apex_agents_external.csv Inspect source ↗CC BY 4.0 | — | — |
| APEX-AgentsAgents · high Not reported Epoch bundle · settings unverified | — | 59.70%Setup unverified · no deltaclaude-fable-5-1_high External result; evaluator not specified · external Run: Date not supplied · apex_agents_external.csv Inspect source ↗CC BY 4.0 | — | — | — | — |
| SimpleQA VerifiedKnowledge · max Epoch Inspect Epoch bundle | — | 70.80%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-01 · simpleqa_verified.csv Reported standard error: ±1.44 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 75.60%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · simpleqa_verified.csv Reported standard error: ±1.36 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 57.30%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · simpleqa_verified.csv Reported standard error: ±1.56 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 62.20%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · simpleqa_verified.csv Reported standard error: ±1.53 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| SimpleQA VerifiedKnowledge · high Epoch Inspect Epoch bundle | — | — | — | — | — | 76.00%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · simpleqa_verified.csv Reported standard error: ±1.35 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| HLEKnowledge · unknown Not reported Epoch bundle · settings unverified | — | — | 54.80%Setup unverified · no deltagpt-6-astra_unknown External result; evaluator not specified · external Run: Date not supplied · hle_external.csv Reported standard error: ±1.94 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| HLEKnowledge · xhigh Not reported Epoch bundle · settings unverified | — | 46.50%Setup unverified · no deltaclaude-fable-5-1_xhigh External result; evaluator not specified · external Run: Date not supplied · hle_external.csv Reported standard error: ±2.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — | — |
| ARC-AGIReasoning · max Not reported Epoch bundle · settings unverified | — | 97.50%Setup unverified · no deltaclaude-fable-5-1_max https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | 97.50%Setup unverified · no deltagpt-6-astra_max https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| ARC-AGIReasoning · xhigh Not reported Epoch bundle · settings unverified | — | 96.50%Setup unverified · no deltaclaude-fable-5-1_xhigh https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | 98.50%Setup unverified · no deltagpt-6-astra_xhigh https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| ARC-AGIReasoning · high Not reported Epoch bundle · settings unverified | — | 96.00%Setup unverified · no deltaclaude-fable-5-1_high https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | 98.50%Setup unverified · no deltagpt-6-astra_high https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | — | — | 98.50%Setup unverified · no deltagemini-4-argon_high https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 |
| ARC-AGIReasoning · medium Not reported Epoch bundle · settings unverified | — | 94.50%Setup unverified · no deltaclaude-fable-5-1_medium https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | 97.50%Setup unverified · no deltagpt-6-astra_medium https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | — | — | 98.50%Setup unverified · no deltagemini-4-argon_medium https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 |
| ARC-AGIReasoning · low Not reported Epoch bundle · settings unverified | — | 90.00%Setup unverified · no deltaclaude-fable-5-1_low https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | 96.50%Setup unverified · no deltagpt-6-astra_low https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | — | — | 76.50%Setup unverified · no deltagemini-4-argon_low https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 |
| ARC-AGIReasoning · none Not reported Epoch bundle · settings unverified | — | — | 86.00%Setup unverified · no deltagpt-6-astra_none https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| BalrogAgents · max Not reported Epoch bundle · settings unverified | — | — | 68.30%Setup unverified · no deltagpt-6-astra_max https://balrogai.com/ · external Run: Date not supplied · balrog_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| Chess PuzzlesReasoning · promax Epoch Inspect Epoch bundle | — | — | 76.00%Δ unavailablegpt-6-astra_promax Epoch AI evaluation · epoch-run Run: 2026-10-15 · chess_puzzles.csv Reported standard error: ±4.29 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| Chess PuzzlesReasoning · max Epoch Inspect Epoch bundle | — | 48.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±5.02 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 74.00%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.41 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | 45.00%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · chess_puzzles.csv Reported standard error: ±5.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| Chess PuzzlesReasoning · high Epoch Inspect Epoch bundle | — | — | — | — | — | 59.00%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.94 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| CritPtOther · unknown Not reported Epoch bundle · settings unverified | — | — | — | — | 24.57%Setup unverified · no deltakimi-k3.1-pro_unknown External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — |
| CritPtOther · max Not reported Epoch bundle · settings unverified | — | 29.71%Setup unverified · no deltaclaude-fable-5-1_max External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 31.71%Setup unverified · no deltagpt-6-astra_max External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 28.00%Setup unverified · no deltagpt-6.1-terra_max External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — | — |
| CritPtOther · xhigh Not reported Epoch bundle · settings unverified | — | 31.14%Setup unverified · no deltaclaude-fable-5-1_xhigh External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 31.43%Setup unverified · no deltagpt-6-astra_xhigh External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 26.57%Setup unverified · no deltagpt-6.1-terra_xhigh External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — | — |
| CritPtOther · high Not reported Epoch bundle · settings unverified | — | 30.29%Setup unverified · no deltaclaude-fable-5-1_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 28.86%Setup unverified · no deltagpt-6-astra_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 25.71%Setup unverified · no deltagpt-6.1-terra_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — | 27.14%Setup unverified · no deltagemini-4-argon_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 |
| CritPtOther · medium Not reported Epoch bundle · settings unverified | — | 29.14%Setup unverified · no deltaclaude-fable-5-1_medium External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 29.14%Setup unverified · no deltagpt-6-astra_medium External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 21.43%Setup unverified · no deltagpt-6.1-terra_medium External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — | — |
| CritPtOther · low Not reported Epoch bundle · settings unverified | — | 27.71%Setup unverified · no deltaclaude-fable-5-1_low External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 26.29%Setup unverified · no deltagpt-6-astra_low External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 15.14%Setup unverified · no deltagpt-6.1-terra_low External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — | — |
| CritPtOther · none Not reported Epoch bundle · settings unverified | — | — | 19.71%Setup unverified · no deltagpt-6-astra_none External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | 4.86%Setup unverified · no deltagpt-6.1-terra_none External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | — | — |
| CursorBenchOther · minimal Not reported Epoch bundle · settings unverified | — | — | — | — | — | 48.80%Setup unverified · no deltagemini-4-argon_minimal https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 |
| CursorBenchOther · max Not reported Epoch bundle · settings unverified | — | 51.80%Setup unverified · no deltaclaude-fable-5-1_max https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — | — |
| CursorBenchOther · xhigh Not reported Epoch bundle · settings unverified | — | 51.60%Setup unverified · no deltaclaude-fable-5-1_xhigh https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — | — |
| CursorBenchOther · high Not reported Epoch bundle · settings unverified | — | 49.20%Setup unverified · no deltaclaude-fable-5-1_high https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — | 52.90%Setup unverified · no deltagemini-4-argon_high https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 |
| CursorBenchOther · medium Not reported Epoch bundle · settings unverified | — | 46.80%Setup unverified · no deltaclaude-fable-5-1_medium https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — | 52.80%Setup unverified · no deltagemini-4-argon_medium https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 |
| CursorBenchOther · low Not reported Epoch bundle · settings unverified | — | 45.10%Setup unverified · no deltaclaude-fable-5-1_low https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — | 50.60%Setup unverified · no deltagemini-4-argon_low https://cursor.com/cursorbench · external Run: Date not supplied · cursorbench_external.csv Inspect source ↗CC BY 4.0 |
| DeepSWECoding · max mini-swe-agent Epoch bundle | — | — | 73.23%Δ unavailablegpt-6-astra_max https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DeepSWECoding · xhigh mini-swe-agent Epoch bundle | — | — | 74.12%Δ unavailablegpt-6-astra_xhigh https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DeepSWECoding · high mini-swe-agent Epoch bundle | — | — | 73.23%Δ unavailablegpt-6-astra_high https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DeepSWECoding · medium mini-swe-agent Epoch bundle | — | — | 72.79%Δ unavailablegpt-6-astra_medium https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DeepSWECoding · low mini-swe-agent Epoch bundle | — | — | 67.04%Δ unavailablegpt-6-astra_low https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DTBenchReasoning · unknown Not reported Epoch bundle · settings unverified | — | 97.60%Setup unverified · no deltaclaude-fable-5-1_unknown https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | 97.07%Setup unverified · no deltagpt-6-astra_unknown https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DTBenchReasoning · xhigh Not reported Epoch bundle · settings unverified | — | 97.60%Setup unverified · no deltaclaude-fable-5-1_xhigh https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | 97.33%Setup unverified · no deltagpt-6-astra_xhigh https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| DTBenchReasoning · high Not reported Epoch bundle · settings unverified | — | 96.53%Setup unverified · no deltaclaude-fable-5-1_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | 96.53%Setup unverified · no deltagpt-6-astra_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | — | — | 97.33%Setup unverified · no deltagemini-4-argon_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 |
| DTBenchReasoning · medium Not reported Epoch bundle · settings unverified | — | 97.07%Setup unverified · no deltaclaude-fable-5-1_medium https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | 97.07%Setup unverified · no deltagpt-6-astra_medium https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | — | — | 97.33%Setup unverified · no deltagemini-4-argon_medium https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 |
| DTBenchReasoning · low Not reported Epoch bundle · settings unverified | — | 94.13%Setup unverified · no deltaclaude-fable-5-1_low https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | 94.93%Setup unverified · no deltagpt-6-astra_low https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | — | — | 97.60%Setup unverified · no deltagemini-4-argon_low https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 |
| EBR-benchOther · max Epoch Inspect Epoch bundle | — | 57.14%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-08 · ebr_bench.csv Reported standard error: ±6.02 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 76.19%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · ebr_bench.csv Reported standard error: ±8.38 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 78.10%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · ebr_bench.csv Reported standard error: ±5.13 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — |
| EBR-benchOther · high Epoch Inspect Epoch bundle | — | — | — | — | — | 64.76%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · ebr_bench.csv Reported standard error: ±4.90 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| FrontierCodeCoding · max codex Epoch bundle | — | — | 53.26%Δ unavailablegpt-6-astra_max https://cognition.com/frontiercode · external Run: Date not supplied · frontiercode_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierCodeCoding · medium claude-code Epoch bundle | — | 50.91%Δ unavailableclaude-fable-5-1_medium https://cognition.com/frontiercode · external Run: Date not supplied · frontiercode_external.csv Inspect source ↗CC BY 4.0 | — | — | — | — |
| FrontierCodeCoding · medium devin-cli Epoch bundle | — | — | — | — | — | 47.55%Δ unavailablegemini-4-argon_medium https://cognition.com/frontiercode · external Run: Date not supplied · frontiercode_external.csv Inspect source ↗CC BY 4.0 |
| FrontierMath-ErdosReasoning · max Epoch Inspect Epoch bundle | — | 0.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-01 · frontiermath_erdos.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 2.94%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-28 · frontiermath_erdos.csv Reported standard error: ±2.06 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierMath-ErdosReasoning · high Epoch Inspect Epoch bundle | — | — | — | — | — | 1.47%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-20 · frontiermath_erdos.csv Reported standard error: ±1.47 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| FrontierMath-Tier-4-v2-PrivateReasoning · max Epoch Inspect Epoch bundle | — | 87.80%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-01 · frontiermath_tier_4_v2.csv Reported standard error: ±5.17 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 97.60%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tier_4_v2.csv Reported standard error: ±2.40 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 92.68%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · frontiermath_tier_4_v2.csv Reported standard error: ±4.12 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 65.85%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · frontiermath_tier_4_v2.csv Reported standard error: ±7.50 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| FrontierMath-Tier-4-v2-PrivateReasoning · xhigh Epoch Inspect Epoch bundle | — | — | 97.60%Δ unavailablegpt-6-astra_xhigh Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tier_4_v2.csv Reported standard error: ±2.40 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierMath-Tier-4-v2-PrivateReasoning · high Epoch Inspect Epoch bundle | — | — | 97.60%Δ unavailablegpt-6-astra_high Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tier_4_v2.csv Reported standard error: ±2.40 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | 78.05%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · frontiermath_tier_4_v2.csv Reported standard error: ±6.54 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| FrontierMath-Tier-4-v2-PrivateReasoning · medium Epoch Inspect Epoch bundle | — | — | 97.56%Δ unavailablegpt-6-astra_medium Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tier_4_v2.csv Reported standard error: ±2.44 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierMath-Tier-4-v2-PrivateReasoning · low Epoch Inspect Epoch bundle | — | — | 87.80%Δ unavailablegpt-6-astra_low Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tier_4_v2.csv Reported standard error: ±5.17 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierMath-Tier-4-v2-PrivateReasoning · none Epoch Inspect Epoch bundle | — | — | 82.93%Δ unavailablegpt-6-astra_none Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tier_4_v2.csv Reported standard error: ±5.95 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| FrontierMath-Tiers-1-3-v2-PrivateReasoning · max Epoch Inspect Epoch bundle | — | 90.18%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-01 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±1.77 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 93.68%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±1.44 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 90.88%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±1.71 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 78.25%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.45 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| FrontierMath-Tiers-1-3-v2-PrivateReasoning · high Epoch Inspect Epoch bundle | — | — | — | — | — | 85.96%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.06 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| Furniture AssemblyReasoning · max Epoch Inspect Epoch bundle | — | 70.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-10 · furniture_assembly.csv Reported standard error: ±5.72 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 80.00%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-09-10 · furniture_assembly.csv Reported standard error: ±4.93 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 79.17%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · furniture_assembly.csv Reported standard error: ±5.08 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 55.83%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · furniture_assembly.csv Reported standard error: ±6.41 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| Furniture AssemblyReasoning · high Epoch Inspect Epoch bundle | — | — | — | — | — | 72.50%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · furniture_assembly.csv Reported standard error: ±5.63 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| GDP.pdfOther · max Not reported Epoch bundle · settings unverified | — | 27.60%Setup unverified · no deltaclaude-fable-5-1_max https://surgehq.ai/benchmarks/gdp-pdf · external Run: Date not supplied · gdp_pdf_external.csv Inspect source ↗CC BY 4.0 | 34.20%Setup unverified · no deltagpt-6-astra_max https://surgehq.ai/benchmarks/gdp-pdf · external Run: Date not supplied · gdp_pdf_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| GDP.pdfOther · high Not reported Epoch bundle · settings unverified | — | 29.60%Setup unverified · no deltaclaude-fable-5-1_high https://surgehq.ai/benchmarks/gdp-pdf · external Run: Date not supplied · gdp_pdf_external.csv Inspect source ↗CC BY 4.0 | — | — | — | 28.40%Setup unverified · no deltagemini-4-argon_high https://surgehq.ai/benchmarks/gdp-pdf · external Run: Date not supplied · gdp_pdf_external.csv Inspect source ↗CC BY 4.0 |
| GSO-BenchCoding · unknown Not reported Epoch bundle · settings unverified | — | 88.24%Setup unverified · no deltaclaude-fable-5-1_unknown https://gso-bench.github.io/index.html · external Run: Date not supplied · gso_external.csv Inspect source ↗CC BY 4.0 | 79.41%Setup unverified · no deltagpt-6-astra_unknown https://gso-bench.github.io/index.html · external Run: Date not supplied · gso_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| LiveBench 2 Coding · 20261025Coding · max LiveBench 2 · 20261025 LiveBench 2 | — | 97.70%Δ unavailableclaude-fable-5.1 · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 93.70%Δ unavailablegpt-6-astra · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 94.30%Δ unavailablegpt-6.1-terra · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 81.80%Δ unavailablekimi-k3.1-pro · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | — |
| LiveBench 2 Coding · 20261025Coding · high LiveBench 2 · 20261025 LiveBench 2 | — | — | — | — | — | 89.70%Δ unavailablegemini-4-argon · high · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench |
| LiveBench 2 Long Agentic · 20261025Agents · max LiveBench 2 · 20261025 LiveBench 2 | — | 77.90%Δ unavailableclaude-fable-5.1 · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 77.50%Δ unavailablegpt-6-astra · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 48.60%Δ unavailablegpt-6.1-terra · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 39.10%Δ unavailablekimi-k3.1-pro · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | — |
| LiveBench 2 Long Agentic · 20261025Agents · high LiveBench 2 · 20261025 LiveBench 2 | — | — | — | — | — | 61.20%Δ unavailablegemini-4-argon · high · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench |
| LiveBench 2 Reasoning · 20261025Reasoning · max LiveBench 2 · 20261025 LiveBench 2 | — | 91.70%Δ unavailableclaude-fable-5.1 · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 92.70%Δ unavailablegpt-6-astra · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 88.70%Δ unavailablegpt-6.1-terra · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | 91.10%Δ unavailablekimi-k3.1-pro · max · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench | — |
| LiveBench 2 Reasoning · 20261025Reasoning · high LiveBench 2 · 20261025 LiveBench 2 | — | — | — | — | — | 95.10%Δ unavailablegemini-4-argon · high · LiveBench 2 20261025 LiveBench · independent-run Run: Date not supplied · 20261025/table.csv Benchmark edition 25 October 2026; individual run date not supplied. One run per model; mean task score. Failed/unmeasurable tasks score zero in the source. Source effort metadata is preserved. Inspect source ↗Attributed public leaderboard score facts · LiveBench |
| LMCAReasoning · unknown Not reported Epoch bundle · settings unverified | — | 65.46%Setup unverified · no deltaclaude-fable-5-1_unknown https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | 64.11%Setup unverified · no deltagpt-6-astra_unknown https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| LMCAReasoning · xhigh Not reported Epoch bundle · settings unverified | — | 65.06%Setup unverified · no deltaclaude-fable-5-1_xhigh https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | 64.37%Setup unverified · no deltagpt-6-astra_xhigh https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | — | — | — |
| LMCAReasoning · high Not reported Epoch bundle · settings unverified | — | 65.09%Setup unverified · no deltaclaude-fable-5-1_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | 63.84%Setup unverified · no deltagpt-6-astra_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | — | — | 63.69%Setup unverified · no deltagemini-4-argon_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 |
| LMCAReasoning · medium Not reported Epoch bundle · settings unverified | — | 65.32%Setup unverified · no deltaclaude-fable-5-1_medium https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | 62.93%Setup unverified · no deltagpt-6-astra_medium https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | — | — | 62.85%Setup unverified · no deltagemini-4-argon_medium https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 |
| LMCAReasoning · low Not reported Epoch bundle · settings unverified | — | 63.46%Setup unverified · no deltaclaude-fable-5-1_low https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | 62.19%Setup unverified · no deltagpt-6-astra_low https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | — | — | 62.92%Setup unverified · no deltagemini-4-argon_low https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 |
| Mystery Game PuzzlesReasoning · max Epoch Inspect Epoch bundle | — | 58.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-01 · mystery_game_puzzles.csv Reported standard error: ±4.96 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 84.00%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · mystery_game_puzzles.csv Reported standard error: ±3.68 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 74.00%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · mystery_game_puzzles.csv Reported standard error: ±4.41 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 42.00%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · mystery_game_puzzles.csv Reported standard error: ±4.96 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| Mystery Game PuzzlesReasoning · high Epoch Inspect Epoch bundle | — | — | — | — | — | 64.00%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · mystery_game_puzzles.csv Reported standard error: ±4.82 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| OTIS Mock AIME 2024-2025Reasoning · max Epoch Inspect Epoch bundle | — | 100.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-01 · otis_mock_aime_2024_2025.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 100.00%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-08-30 · otis_mock_aime_2024_2025.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 100.00%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-10-26 · otis_mock_aime_2024_2025.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 100.00%Δ unavailablekimi-k3.1-pro-together_max Epoch AI evaluation · epoch-run Run: 2026-11-06 · otis_mock_aime_2024_2025.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — |
| OTIS Mock AIME 2024-2025Reasoning · high Epoch Inspect Epoch bundle | — | — | — | — | — | 100.00%Δ unavailablegemini-4-argon_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · otis_mock_aime_2024_2025.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 |
| ProofBenchReasoning · unknown Not reported Epoch bundle · settings unverified | — | — | 99.00%Setup unverified · no deltagpt-6-astra_unknown https://www.vals.ai/benchmarks/proof_bench · external Run: Date not supplied · proofbench_external.csv Inspect source ↗CC BY 4.0 | — | 91.00%Setup unverified · no deltakimi-k3.1-pro_unknown https://www.vals.ai/benchmarks/proof_bench · external Run: Date not supplied · proofbench_external.csv Reported standard error: ±2.88 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 99.00%Setup unverified · no deltagemini-4-argon_unknown https://www.vals.ai/benchmarks/proof_bench · external Run: Date not supplied · proofbench_external.csv Inspect source ↗CC BY 4.0 |
| ProofBenchReasoning · max Not reported Epoch bundle · settings unverified | — | 100.00%Setup unverified · no deltaclaude-fable-5-1_max https://www.vals.ai/benchmarks/proof_bench · external Run: Date not supplied · proofbench_external.csv Inspect source ↗CC BY 4.0 | — | — | — | — |
| Remote Labor IndexAgents · unknown Not reported Epoch bundle · settings unverified | — | 17.92%Setup unverified · no deltaclaude-fable-5-1_unknown External result; evaluator not specified · external Run: Date not supplied · rli_external.csv Inspect source ↗CC BY 4.0 | 20.83%Setup unverified · no deltagpt-6-astra_unknown External result; evaluator not specified · external Run: Date not supplied · rli_external.csv Inspect source ↗CC BY 4.0 | — | — | 17.08%Setup unverified · no deltagemini-4-argon_unknown External result; evaluator not specified · external Run: Date not supplied · rli_external.csv Inspect source ↗CC BY 4.0 |
| SpireClimbOther · max Epoch Inspect Epoch bundle | — | 2.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-11-05 · spireclimb.csv Reported standard error: ±1.14 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 34.00%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-11-04 · spireclimb.csv Reported standard error: ±4.13 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 22.33%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-11-04 · spireclimb.csv Reported standard error: ±3.39 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — |
| SpireClimb (Slay the Spire 1, Ascension 20 Heart)Other · max Epoch Inspect Epoch bundle | — | 0.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-11-05 · spireclimb_sts1_a20_heart.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 33.33%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-11-04 · spireclimb_sts1_a20_heart.csv Reported standard error: ±6.14 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 10.00%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-11-04 · spireclimb_sts1_a20_heart.csv Reported standard error: ±3.91 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — |
| SpireClimb (Slay the Spire 2, Ascension 10)Other · max Epoch Inspect Epoch bundle | — | 4.00%Δ unavailableclaude-fable-5-1_max Epoch AI evaluation · epoch-run Run: 2026-09-25 · spireclimb_sts2_a10.csv Reported standard error: ±2.28 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 34.67%Δ unavailablegpt-6-astra_max Epoch AI evaluation · epoch-run Run: 2026-09-22 · spireclimb_sts2_a10.csv Reported standard error: ±5.53 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 34.67%Δ unavailablegpt-6.1-terra_max Epoch AI evaluation · epoch-run Run: 2026-11-03 · spireclimb_sts2_a10.csv Reported standard error: ±5.53 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — |
| Terminal-Bench 4.0Agents · max Terminal-Bench 4.0 · Claude Code Terminal-Bench 4.0 | — | 57.88%Δ unavailableClaude Fable 5.1 · Claude Code · max Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-01 · 4-0-0/c741608e-c94e-417d-b7a8-e67111a0c887 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 3.76 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — | — |
| Terminal-Bench 4.0Agents · max Terminal-Bench 4.0 · Codex Terminal-Bench 4.0 | — | — | 58.18%Δ unavailableGPT-6 Astra · Codex · max Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-03 · 4-0-0/5c537be4-7fc3-449b-8bfc-ceb9061c2535 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 2.79 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — |
| Terminal-Bench 4.0Agents · xhigh Terminal-Bench 4.0 · Claude Code Terminal-Bench 4.0 | — | 57.88%Δ unavailableClaude Fable 5.1 · Claude Code · xhigh Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-01 · 4-0-0/9e1c5e11-748f-4f2e-911a-070a396d5bf8 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 3.36 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — | — |
| Terminal-Bench 4.0Agents · xhigh Terminal-Bench 4.0 · Codex Terminal-Bench 4.0 | — | — | 57.88%Δ unavailableGPT-6 Astra · Codex · xhigh Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-03 · 4-0-0/16db8ad5-84aa-4588-b660-1ce68c0d45e2 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 2.72 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — |
| Terminal-Bench 4.0Agents · high Terminal-Bench 4.0 · Claude Code Terminal-Bench 4.0 | — | 54.55%Δ unavailableClaude Fable 5.1 · Claude Code · high Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-01 · 4-0-0/4643b23c-53e8-47ae-a3bd-9abc900e9588 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 3.44 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — | — |
| Terminal-Bench 4.0Agents · high Terminal-Bench 4.0 · Codex Terminal-Bench 4.0 | — | — | 57.88%Δ unavailableGPT-6 Astra · Codex · high Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-03 · 4-0-0/3475050c-bf5e-4261-a3f6-0af5350af13f Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 2.97 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — |
| Terminal-Bench 4.0Agents · medium Terminal-Bench 4.0 · Claude Code Terminal-Bench 4.0 | — | 53.94%Δ unavailableClaude Fable 5.1 · Claude Code · medium Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-01 · 4-0-0/83e6bf7e-aeb4-4116-9f30-ca8de96603fb Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 3.39 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — | — |
| Terminal-Bench 4.0Agents · medium Terminal-Bench 4.0 · Codex Terminal-Bench 4.0 | — | — | 54.24%Δ unavailableGPT-6 Astra · Codex · medium Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-03 · 4-0-0/f3c3d5a6-6424-4acb-bfcc-615c3f79f6dd Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 2.66 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — |
| Terminal-Bench 4.0Agents · low Terminal-Bench 4.0 · Claude Code Terminal-Bench 4.0 | — | 43.33%Δ unavailableClaude Fable 5.1 · Claude Code · low Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-01 · 4-0-0/e18d02ab-0973-4d86-872f-a353c8a7f344 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 3.61 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — | — |
| Terminal-Bench 4.0Agents · low Terminal-Bench 4.0 · Codex Terminal-Bench 4.0 | — | — | 50.61%Δ unavailableGPT-6 Astra · Codex · low Terminal-Bench public leaderboard · benchmark-publisher Run: 2026-09-03 · 4-0-0/b3ad58f3-b311-4d4d-875b-3158cf0309d6 Published model + agent result; agents differ across models. Not a controlled model-only evaluation. Reported trials: 330. Source 95% accuracy interval half-width: 2.75 pp; this is not a DC Power interval. Inspect source ↗Attributed public leaderboard score facts · Terminal-Bench | — | — | — |
| WeirdMLCoding · promax Not reported Epoch bundle · settings unverified | — | — | 93.57%Setup unverified · no deltagpt-6-astra_promax https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| WeirdMLCoding · max Not reported Epoch bundle · settings unverified | — | 92.90%Setup unverified · no deltaclaude-fable-5-1_max https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 93.28%Setup unverified · no deltagpt-6-astra_max https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| WeirdMLCoding · high Not reported Epoch bundle · settings unverified | — | 92.30%Setup unverified · no deltaclaude-fable-5-1_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | 92.87%Setup unverified · no deltagpt-6-astra_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | — | — | — |
| WeirdML v3Coding · xhigh claude_code Epoch bundle | — | 43.53%Δ unavailableclaude-fable-5-1_xhigh https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_v3_external.csv Inspect source ↗CC BY 4.0 | — | — | — | — |
| WeirdML v3Coding · xhigh codex_cli Epoch bundle | — | — | 54.05%Δ unavailablegpt-6-astra_xhigh https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_v3_external.csv Inspect source ↗CC BY 4.0 | 40.45%Δ unavailablegpt-6.1-terra_xhigh https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_v3_external.csv Inspect source ↗CC BY 4.0 | — | — |
| WeirdML v3Coding · high gemini_cli Epoch bundle | — | — | — | — | — | 41.12%Δ unavailablegemini-4-argon_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_v3_external.csv Inspect source ↗CC BY 4.0 |
Numerical benchmark differences are descriptive, not significance tests. No confidence interval is available for the DC aggregate; the ECI source interval applies only to ECI. External rows can contain model-developer reports. Public benchmark results do not establish performance in your application.
Filter by release date, provider, scores and cost
Date filters include verified release dates only. Scores and costs marked — do not match a numeric filter. Domain sorts show complete panels first, followed by partial summaries. Domain filters can include partial scores; inspect coverage before comparing them. Task cost uses 3,000 input and 1,000 output tokens.
437 matching models · “—” means insufficient comparable evidence. Prices are listed catalogue rates; open a model to check the provider. Listings do not confirm account access.
| Model | DC Power / coverage | Coding | Reasoning | Agents | DC Eco | DC Overall | Input / output | Context | Release / listing | Compare |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 anthropic · tools · Azure listed | 82.9100% fixed panel · 4 source families | 87.6 100% panel | 90.2 100% panel | 68.8 100% panel | 19.7 | 67.1 | $4 / $20 USD / 1M tokens | 1M | 2026-09-22 Release date | |
| Claude Fable 5.1 anthropic · tools · Azure listed | 80.9100% fixed panel · 4 source families | 85.5 100% panel | 87.5 100% panel | 67.9 100% panel | 9.0 | 62.9 | $10 / $50 USD / 1M tokens | 1M | 2026-09-01 Release date | |
| GPT-6 Astra openai · tools · Azure listed | 77.2100% fixed panel · 4 source families | 70.2 100% panel | 92.3 100% panel | 67.8 100% panel | 8.6 | 60.1 | $10 / $50 USD / 1M tokens | 1.05M | 2026-09-04 Release date | |
| GPT-6 Luna openai · tools · Azure listed | 51.7100% fixed panel · 4 source families | 55.6 100% panel | 71.9 100% panel | 23.6 100% panel | 47.9 | 50.8 | $0.1 / $0.5 USD / 1M tokens | 1.05M | 2026-09-22 Release date | |
| Grok 4.6 x-ai · tools · Azure listed | 56.8100% fixed panel · 4 source families | 65.1 100% panel | 76.4 100% panel | 24.1 100% panel | 25.8 | 49.0 | $2 / $6 USD / 1M tokens | 500K | 2026-08-12 Release date | |
| Aion 3.5 aion-labs · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $3 / $6 USD / 1M tokens | 262K | 2026-09-23 Release date | |
| Aion 3.5 Mini aion-labs · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.7 / $1.4 USD / 1M tokens | 262K | 2026-09-23 Release date | |
| Aion-2.0 aion-labs · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.8 / $1.6 USD / 1M tokens | 131K | 2025-12-21 Release date | |
| Aion-3.0 aion-labs · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $3 / $6 USD / 1M tokens | 131K | 2026-05-05 Release date | |
| Aion-3.0-Mini aion-labs · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.7 / $1.4 USD / 1M tokens | 131K | 2026-05-14 Release date | |
| Aion-RP 1.0 (8B) aion-labs | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.8 / $1.6 USD / 1M tokens | 33K | 2025-02-04 Release date | |
| Beam 1.0 501B A23B reflection · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.3 / $1 USD / 1M tokens | 1.05M | 2026-10-05 Release date | |
| Claude Fable 5 anthropic · tools · Azure listed | —53.5% fixed panel · Power withheld | 63.9 50% panel | 83.5 60% panel | 44.5 50% panel | — | — | $10 / $50 USD / 1M tokens | 1M | 2026-06-09 Release date | |
| Claude Haiku 4.5 anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $1 / $5 USD / 1M tokens | 200K | 2025-10-15 Release date | |
| Claude Haiku 4.5 (latest) anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $1 / $5 USD / 1M tokens | 200K | 2025-10-15 Release date | |
| Claude Haiku 5.5 anthropic · tools · Azure listed | —67.5% fixed panel · Power withheld | 91.4 50% panel | 74.2 100% panel | 40.0 50% panel | — | — | $0.1 / $0.5 USD / 1M tokens | 1M | 2026-10-07 Release date | |
| Claude Opus 4.1 (latest) anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $15 / $75 USD / 1M tokens | 200K | 2025-08-05 Release date | |
| Claude Opus 4.5 anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $5 / $25 USD / 1M tokens | 200K | 2025-11-24 Release date | |
| Claude Opus 4.5 (latest) anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $5 / $25 USD / 1M tokens | 200K | 2025-11-24 Release date | |
| Claude Opus 4.6 anthropic · tools | —10.5% fixed panel · Power withheld | — 0% panel | 59.4 30% panel | — 0% panel | — | — | $5 / $25 USD / 1M tokens | 1M | 2026-02-05 Release date | |
| Claude Opus 4.7 anthropic · tools · Azure listed | —28% fixed panel · Power withheld | 31.1 50% panel | 66.0 30% panel | — 0% panel | — | — | $5 / $25 USD / 1M tokens | 1M | 2026-04-16 Release date | |
| Claude Opus 4.8 anthropic · tools · Azure listed | —25.5% fixed panel · Power withheld | — 0% panel | 76.8 30% panel | 23.6 50% panel | — | — | $5 / $25 USD / 1M tokens | 1M | 2026-05-28 Release date | |
| Claude Opus 5 anthropic · tools · Azure listed | —36% fixed panel · Power withheld | — 0% panel | 83.1 60% panel | 51.8 50% panel | — | — | $5 / $25 USD / 1M tokens | 1M | 2026-07-24 Release date | |
| Claude Sonnet 4.5 anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $3 / $15 USD / 1M tokens | 200K | 2025-09-29 Release date | |
| Claude Sonnet 4.5 (latest) anthropic · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $3 / $15 USD / 1M tokens | 1M | 2025-09-29 Release date | |
| Claude Sonnet 4.6 anthropic · tools · Azure listed | —10.5% fixed panel · Power withheld | — 0% panel | 58.3 30% panel | — 0% panel | — | — | $3 / $15 USD / 1M tokens | 1M | 2026-02-17 Release date | |
| Claude Sonnet 5 anthropic · tools · Azure listed | —25.5% fixed panel · Power withheld | — 0% panel | 63.5 30% panel | 12.4 50% panel | — | — | $2 / $10 USD / 1M tokens | 1M | 2026-06-30 Release date | |
| Claude Sonnet 5.5 anthropic · tools · Azure listed | —82.5% fixed panel · Power withheld | 94.7 50% panel | 86.6 100% panel | 62.6 100% panel | — | — | $2 / $10 USD / 1M tokens | 1M | 2026-09-28 Release date | |
| Codestral (latest) mistral · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.3 / $0.9 USD / 1M tokens | 256K | 2024-05-29 Release date | |
| Codestral 25.08 mistralai · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.3 / $0.9 USD / 1M tokens | 256K | 2025-07-30 Release date | |
| Command A cohere · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $2.5 / $10 USD / 1M tokens | 256K | 2025-03-13 Release date | |
| Command A+ cohere · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.3 / $1.5 USD / 1M tokens | 192K | 2026-05-20 Release date | |
| Command R cohere · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.15 / $0.6 USD / 1M tokens | 128K | 2024-08-30 Release date | |
| Command R+ cohere · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $2.5 / $10 USD / 1M tokens | 128K | 2024-08-30 Release date | |
| Command R7B cohere · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.0375 / $0.15 USD / 1M tokens | 128K | 2024-12-13 Release date | |
| Crouton Alpha stealth · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0 / $0 USD / 1M tokens | 1.05M | 2026-11-03 Release date | |
| Cydonia 24B V4.1 thedrummer | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.3 / $0.5 USD / 1M tokens | 131K | 2025-09-27 Release date | |
| Daybreak Blue openai · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $4 / $20 USD / 1M tokens | 1.05M | 2026-08-07 Release date | |
| Daybreak Red openai · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $12.5 / $75 USD / 1M tokens | 400K | 2026-08-07 Release date | |
| Deep Research Max Preview (Apr-21-2026) google · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $2 / $12 USD / 1M tokens | 131K | 2026-04-21 Release date | |
| Deep Research Preview (Apr-21-2026) google · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $2 / $12 USD / 1M tokens | 131K | 2026-04-21 Release date | |
| DeepSeek Chat deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.32 / $0.89 USD / 1M tokens | 164K | 2025-12-01 Release date | |
| DeepSeek R1 0528 deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.5 / $2.15 USD / 1M tokens | 164K | 2025-05-28 Release date | |
| DeepSeek V3 0324 deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.29 / $1.14 USD / 1M tokens | 164K | 2025-03-24 Release date | |
| DeepSeek V3.1 deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.25 / $0.95 USD / 1M tokens | 164K | 2025-08-21 Release date | |
| DeepSeek V3.1 Terminus deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.27 / $1 USD / 1M tokens | 164K | 2025-09-22 Release date | |
| DeepSeek V3.2 deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.26 / $0.38 USD / 1M tokens | 164K | 2025-12-01 Release date | |
| DeepSeek V3.2 Exp deepseek · tools | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.27 / $0.41 USD / 1M tokens | 164K | 2025-09-29 Release date | |
| DeepSeek V4 Flash deepseek · tools · Azure listed | —0% fixed panel · Power withheld | — 0% panel | — 0% panel | — 0% panel | — | — | $0.091 / $0.182 USD / 1M tokens | 1.05M | 2026-04-24 Release date | |
| DeepSeek V4 Flash 0731 deepseek · tools | —21% fixed panel · Power withheld | — 0% panel | 61.4 60% panel | — 0% panel | — | — | $0.0434 / $0.3 USD / 1M tokens | 1.05M | 2026-07-31 Release date |
DC Score v3 / independent source families
One fixed panel. Every weight visible.
DC Power is our weighted aggregate of raw benchmark scores from LiveBench, Epoch AI, ARC Prize and Terminal-Bench. It is not an ECI rescaling. Power = 0.35 × Coding + 0.35 × Reasoning + 0.30 × Agents. These are explicit editorial preferences, not statistically learned weights or a probability of success.
| Domain | Power weight | Source panels within the domain |
|---|---|---|
| Coding | 35% | 50% LiveBench 2 Coding · 25 Oct 2026 50% Epoch AI original MirrorCode |
| Reasoning | 35% | 40% LiveBench 2 Reasoning · 25 Oct 2026 30% Epoch AI original reasoning panel 30% ARC Prize · ARC-AGI-2 semi-private |
| Agents | 30% | 50% LiveBench 2 Long Agentic · 25 Oct 2026 50% Terminal-Bench 4.0 · published model + agent |
Source balance. At full coverage, LiveBench contributes 46.5%, original Epoch AI runs 28%, ARC Prize 10.5% and Terminal-Bench 15%. One source does not gain weight by publishing more rows. ECI, vendor reports and external scores republished through Epoch have zero Power weight. Direct ARC and Terminal results enter once.
Epoch reasoning panel. Equal weights for original Epoch-run Chess Puzzles, FrontierMath tiers 1–3 v2 private and OTIS Mock AIME 2024–2025. All three are required. GPQA and other supplementary tests stay in the matrix; vendor-only GPQA reports do not become independent Epoch runs.
Missing evidence stays visible. Power, Eco and Overall require every fixed source panel. Domain scores can show a partial summary: available panel weights are renormalized, with coverage and missing inputs shown. A partial score is not a win against a complete score. The same complete panel is required for every ranked model.
Fixed editions and effort policy. LiveBench 2 edition 25 October 2026 and Terminal-Bench 4.0. Select known effort in this order: max, xhigh, high, medium, low, minimal, none, declared highest-supported. Never select the largest numerical result. A dated later run can replace an earlier run under the same setup; unresolved conflicts or different harnesses are missing.
Publisher independence is limited. Four separate source organizations do not imply blind or independently reproduced runs, independent funding, equal compute budgets or uncorrelated tests. Terminal-Bench measures a model with its published agent; Claude Code, Codex and other harnesses differ. Inspect the selected configurations. Public benchmark data cannot establish outcomes in your application.
No invented confidence interval. We do not convert ECI uncertainty into uncertainty for this aggregate. No statistically valid DC Power interval is available. Raw means are averaged on their original 0–100 scales; test difficulty and contamination can affect results. A small aggregate difference is descriptive, not proof of superiority.
Token cost C = (3,000 × input rate + 1,000 × output rate) / 1,000,000, uncached. DC Eco = Power / (1 + 100 × C), eligible only for complete Power ≥ 50 and known prices. At $0.01, Eco is half Power. DC Overall = 0.75 × Power + 0.25 × Eco. Use the comparison calculator for your workload and quality weight.
Cost is conditional. Prices are listed USD per million tokens. Reasoning tokens, tools, retries, cache writes, discounts and long-context surcharges are excluded. These are token-spend estimates, not measured cost per successful task. Quality benchmark effort does not establish a matching token workload.
Inspect Claude Fable 5.1: weights, inputs & missing evidence
Fixed-panel coverage: 100%. All ranked models use this complete panel. ECI contributes 0%.
Coding: 85.5 · 35% of Power · 100% coverage
LiveBench 2 Coding · 25 Oct 2026 · 50% of Coding = 17.5% of Power · 97.7
LiveBench 2 Coding · 20261025: 97.70% ↗claude-fable-5.1 · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench
Epoch AI original MirrorCode · 50% of Coding = 17.5% of Power · 73.3
MirrorCode: 73.30% ↗claude-fable-5-1_high · Epoch Inspect · Epoch AI evaluation
Reasoning: 87.5 · 35% of Power · 100% coverage
LiveBench 2 Reasoning · 25 Oct 2026 · 40% of Reasoning = 14.0% of Power · 91.7
LiveBench 2 Reasoning · 20261025: 91.70% ↗claude-fable-5.1 · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench
Epoch AI original reasoning panel · 30% of Reasoning = 10.5% of Power · 79.4
Chess Puzzles: 48.00% ↗claude-fable-5-1_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel
FrontierMath-Tiers-1-3-v2-Private: 90.18% ↗claude-fable-5-1_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel
OTIS Mock AIME 2024-2025: 100.00% ↗claude-fable-5-1_max · Epoch Inspect · Epoch AI evaluation · each input 33.33% of this panel
ARC Prize · ARC-AGI-2 semi-private · 30% of Reasoning = 10.5% of Power · 90.0
ARC-AGI-2 · semi-private: 90.00% ↗Claude Fable 5.1 (Max) · ARC Prize leaderboard baseline · ARC Prize official leaderboard
Agents: 67.9 · 30% of Power · 100% coverage
LiveBench 2 Long Agentic · 25 Oct 2026 · 50% of Agents = 15.0% of Power · 77.9
LiveBench 2 Long Agentic · 20261025: 77.90% ↗claude-fable-5.1 · max · LiveBench 2 20261025 · LiveBench 2 · 20261025 · LiveBench
Terminal-Bench 4.0 · published model + agent · 50% of Agents = 15.0% of Power · 57.9
Terminal-Bench 4.0: 57.88% ↗Claude Fable 5.1 · Claude Code · max · Terminal-Bench 4.0 · Claude Code · Terminal-Bench public leaderboard
Power = 0.35 × 85.5 + 0.35 × 87.5 + 0.30 × 67.9 = 80.9. Full precision is used before final rounding.
Reference token cost: $0.08 · Eco: 9.0 · Overall: 62.9
v3 replaces the previous ECI transform; scores across methodology versions are not directly comparable. Data refreshes every six hours, not on each page view. Benchmark editions are pinned for comparability; changing the test panel requires a methodology revision. Imports older than 24 hours are marked delayed. Catalogue releases continue to update independently. Azure listings do not confirm account quota or regional capacity.
OpenRouter catalogue ↗ · Public catalogue API; attributed metadata and list prices
models.dev ↗ · MIT
Epoch AI, Capabilities & benchmarking ↗ · CC BY 4.0; only original Epoch-run panel observations enter Power; ECI has zero weight
ARC Prize official leaderboard ↗ · Attributed public leaderboard facts; semi-private split; one direct Power panel, never counted through Epoch again
LiveBench 2 · 25 October 2026 ↗ · Attributed public score facts; original Coding, Reasoning and Long Agentic suites
Terminal-Bench 4.0 ↗ · Attributed public score facts; published model plus agent, not a model-only ablation
Microsoft Azure Retail Prices ↗ · Public retail meters; deployment eligibility is not confirmed
Epoch data: CC BY 4.0. Other publishers: attributed public leaderboard score facts. DARC.DEV defines the aggregate and makes no claim of source endorsement. No private account details are published.