| GPQA diamondKnowledge | 68.18%o3-mini-2025-01-31_low Epoch AI evaluation · epoch-run Run: 2026-07-20 · gpqa_diamond.csv Reported standard error: ±3.32 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 77.02%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2025-02-13 · gpqa_diamond.csv Reported standard error: ±2.59 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 74.27%o3-mini-2025-01-31_medium Epoch AI evaluation · epoch-run Run: 2025-01-31 · gpqa_diamond.csv Reported standard error: ±2.54 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 12.41%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2025-11-16 · frontiermath.csv Reported standard error: ±1.94 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 8.08%o3-mini-2025-01-31_medium Epoch AI evaluation · epoch-run Run: 2025-03-06 · frontiermath.csv Reported standard error: ±1.28 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-2025-07-01-PrivateReasoning | 4.17%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2025-07-01 · frontiermath_tier_4.csv Reported standard error: ±2.91 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 10.53%o3-mini-2025-01-31_medium Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±1.82 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 3.86%o3-mini-2025-01-31_low Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±1.14 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 18.60%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2026-06-11 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.31 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-v2-PrivateReasoning | 0.00%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2026-06-11 · frontiermath_tier_4_v2.csv Reported standard error: ±0.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| MATH level 5Reasoning | 96.49%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2025-02-13 · math_level_5.csv Reported standard error: ±0.43 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| MATH level 5Reasoning | 95.17%o3-mini-2025-01-31_medium Epoch AI evaluation · epoch-run Run: 2025-01-31 · math_level_5.csv Reported standard error: ±0.47 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 44.44%o3-mini-2025-01-31_low Epoch AI evaluation · epoch-run Run: 2026-07-20 · otis_mock_aime_2024_2025.csv Reported standard error: ±7.49 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 76.94%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2025-02-27 · otis_mock_aime_2024_2025.csv Reported standard error: ±5.24 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 63.89%o3-mini-2025-01-31_medium Epoch AI evaluation · epoch-run Run: 2025-02-25 · otis_mock_aime_2024_2025.csv Reported standard error: ±6.04 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| SimpleQA VerifiedKnowledge | 15.30%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2026-08-31 · simpleqa_verified.csv Reported standard error: ±1.14 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 11.00%o3-mini-2025-01-31_medium Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±3.14 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 15.00%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±3.59 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 6.00%o3-mini-2025-01-31_low Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±2.39 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| Mystery Game PuzzlesReasoning | 7.00%o3-mini-2025-01-31_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · mystery_game_puzzles.csv Reported standard error: ±2.56 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Aider polyglotCoding | 53.80%o3-mini-2025-01-31_medium https://aider.chat/docs/leaderboards/ · external Run: Date not supplied · aider_polyglot_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| Aider polyglotCoding | 60.40%o3-mini-2025-01-31_high https://aider.chat/docs/leaderboards/ · external Run: Date not supplied · aider_polyglot_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 34.50%o3-mini-2025-01-31_high ARC Prize Leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 22.33%o3-mini-2025-01-31_medium https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 14.50%o3-mini-2025-01-31_low ARC Prize Leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| CadEvalOther | 54.00%o3-mini-2025-01-31_medium CadEval Dashboard · external Run: Date not supplied · cad_eval_external.csv Assuming medium reasoning effort. Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| CybenchCoding | 22.50%o3-mini-2025-01-31_medium Cybench leaderboard · external Run: Date not supplied · cybench_external.csv "* Results from HAL leaderboard evaluation † The scores for OpenAI o3-mini and OpenAI o1-mini are inflated because HAL likely ran on a fork of the Inspect framework that leaked the answer to a task that both models completed successfully. Their Unguided % Solved scores have been adjusted downward by 2.5% (to 22.5% and 10% respectively) and their FSTs have been updated to reflect their most difficult tasks solved excluding the leaked task (42 min for o3-mini and 11 min for o1-mini)." Assuming m Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| Fiction.LiveBenchOther | 50.00%o3-mini-2025-01-31_medium Fiction.live leaderboard · external Run: Date not supplied · fictionlivebench_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| GSO-BenchCoding | 1.30%o3-mini-2025-01-31_high https://gso-bench.github.io/index.html · external Run: Date not supplied · gso_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| GSO-BenchCoding | 1.30%o3-mini-2025-01-31_low GSO Leaderboard · external Run: Date not supplied · gso_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| SimpleBenchReasoning | 22.80%o3-mini-2025-01-31_high SimpleBench Leaderboard · external Run: Date not supplied · simplebench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| WeirdMLCoding | 43.70%o3-mini-2025-01-31_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.02 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 2.99%o3-mini-2025-01-31_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 2.08%o3-mini-2025-01-31_medium External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 0.00%o3-mini-2025-01-31_low External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| CritPtOther | 0.29%o3-mini-2025-01-31_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| LMCAReasoning | 18.97%o3-mini-2025-01-31_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| DTBenchReasoning | 68.80%o3-mini-2025-01-31_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 0.00%o3-mini (Low) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.0623 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | low / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 2.08%o3-mini (Medium) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.2843 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | medium / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 2.99%o3-mini (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.5472 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | high / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|