| GPQA diamondKnowledge | 77.78%o4-mini-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-07 · gpqa_diamond.csv Reported standard error: ±2.96 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 75.25%o4-mini-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-07-11 · gpqa_diamond.csv Reported standard error: ±3.07 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 79.61%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-04-16 · gpqa_diamond.csv Reported standard error: ±2.40 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 10.69%o4-mini-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2025-11-16 · frontiermath.csv Reported standard error: ±1.82 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 18.97%o4-mini-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2025-11-13 · frontiermath.csv Reported standard error: ±2.31 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 24.83%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-11-13 · frontiermath.csv Reported standard error: ±2.54 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-2025-07-01-PrivateReasoning | 2.08%o4-mini-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2025-08-07 · frontiermath_tier_4.csv Reported standard error: ±2.08 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-2025-07-01-PrivateReasoning | 6.25%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-07-01 · frontiermath_tier_4.csv Reported standard error: ±3.53 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 16.14%o4-mini-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.18 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 28.77%o4-mini-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.69 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 36.14%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-06-11 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.85 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-v2-PrivateReasoning | 4.88%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-06-11 · frontiermath_tier_4_v2.csv Reported standard error: ±3.41 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| MATH level 5Reasoning | 97.83%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-04-16 · math_level_5.csv Reported standard error: ±0.33 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 73.33%o4-mini-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-07 · otis_mock_aime_2024_2025.csv Reported standard error: ±6.67 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 57.78%o4-mini-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-07-13 · otis_mock_aime_2024_2025.csv Reported standard error: ±7.45 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 81.67%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-04-16 · otis_mock_aime_2024_2025.csv Reported standard error: ±4.69 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| SimpleQA VerifiedKnowledge | 19.60%o4-mini-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-08-27 · simpleqa_verified.csv Reported standard error: ±1.26 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| SimpleQA VerifiedKnowledge | 18.80%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · simpleqa_verified.csv Reported standard error: ±1.24 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 13.00%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±3.38 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 22.00%o4-mini-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.16 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 15.00%o4-mini-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±3.59 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| Mystery Game PuzzlesReasoning | 5.00%o4-mini-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · mystery_game_puzzles.csv Reported standard error: ±2.19 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Aider polyglotCoding | 72.00%o4-mini-2025-04-16_high https://aider.chat/docs/leaderboards/ · external Run: Date not supplied · aider_polyglot_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 58.67%o4-mini-2025-04-16_high https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 41.83%o4-mini-2025-04-16_medium https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 21.33%o4-mini-2025-04-16_low https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| CadEvalOther | 62.00%o4-mini-2025-04-16_medium CadEval Dashboard · external Run: Date not supplied · cad_eval_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| Fiction.LiveBenchOther | 77.80%o4-mini-2025-04-16_medium Fiction.live leaderboard · external Run: Date not supplied · fictionlivebench_external.csv Assuming medium reasoning effort Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| GeoBenchOther | 64.00%o4-mini-2025-04-16_high GeoBench leaderboard · external Run: Date not supplied · geobench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| GeoBenchOther | 64.00%o4-mini-2025-04-16_medium GeoBench leaderboard · external Run: Date not supplied · geobench_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| GSO-BenchCoding | 3.60%o4-mini-2025-04-16_high https://gso-bench.github.io/index.html · external Run: Date not supplied · gso_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| SimpleBenchReasoning | 38.70%o4-mini-2025-04-16_high SimpleBench Leaderboard · external Run: Date not supplied · simplebench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| VPCTOther | 57.50%o4-mini-2025-04-16_medium VPCT leaderboard · external Run: Date not supplied · vpct_external.csv Assuming medium reasoning effort Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| WeirdMLCoding | 52.56%o4-mini-2025-04-16_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.02 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 6.11%o4-mini-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 2.36%o4-mini-2025-04-16_medium External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 1.67%o4-mini-2025-04-16_low External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| HLEKnowledge | 18.08%o4-mini-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · hle_external.csv Reported standard error: ±1.51 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| HLEKnowledge | 14.28%o4-mini-2025-04-16_medium External result; evaluator not specified · external Run: Date not supplied · hle_external.csv Reported standard error: ±1.37 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| CritPtOther | 0.60%o4-mini-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| GDPvalAgents | 25.30%o4-mini-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · gdpval_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| METR Time HorizonsOther | 63.93%o4-mini-2025-04-16_medium METR - Measuring AI Ability to Complete Long Tasks · external Run: Date not supplied · metr_time_horizons_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| LMCAReasoning | 26.50%o4-mini-2025-04-16_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| DTBenchReasoning | 77.60%o4-mini-2025-04-16_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| EnigmaEvalOther | 9.21%o4-mini-2025-04-16_high https://labs.scale.com/leaderboard/enigma_eval · external Run: Date not supplied · enigma_eval_external.csv Reported standard error: ±1.65 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| EnigmaEvalOther | 6.81%o4-mini-2025-04-16_medium https://labs.scale.com/leaderboard/enigma_eval · external Run: Date not supplied · enigma_eval_external.csv Reported standard error: ±0.83 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 1.67%o4-mini (Low) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.05 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | low / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 2.36%o4-mini (Medium) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.2311 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | medium / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 6.11%o4-mini (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.856 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | high / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|