| GPQA diamondKnowledge | 80.81%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-07 · gpqa_diamond.csv Reported standard error: ±2.81 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 79.80%o3-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-07-15 · gpqa_diamond.csv Reported standard error: ±2.86 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 81.82%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-04-16 · gpqa_diamond.csv Reported standard error: ±2.13 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 9.72%o3-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2025-11-17 · frontiermath.csv Reported standard error: ±1.75 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 18.69%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-11-16 · frontiermath.csv Reported standard error: ±2.30 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 16.90%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2025-11-16 · frontiermath.csv Reported standard error: ±2.20 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-2025-07-01-PrivateReasoning | 2.08%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-07-01 · frontiermath_tier_4.csv Reported standard error: ±2.08 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 33.33%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.80 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 29.82%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.71 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 19.30%o3-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-08-27 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.34 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| MATH level 5Reasoning | 97.77%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-04-16 · math_level_5.csv Reported standard error: ±0.33 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 84.44%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-07 · otis_mock_aime_2024_2025.csv Reported standard error: ±5.46 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 60.00%o3-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-07-15 · otis_mock_aime_2024_2025.csv Reported standard error: ±7.39 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 83.89%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2025-04-16 · otis_mock_aime_2024_2025.csv Reported standard error: ±4.38 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| SWE-Bench verifiedCoding | 62.32%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-02-12 · swe_bench_verified.csv Reported standard error: ±2.21 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| SimpleQA VerifiedKnowledge | 49.40%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · simpleqa_verified.csv Reported standard error: ±1.58 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 36.00%o3-2025-04-16_low Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.82 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 36.00%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.82 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 37.00%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.85 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| Mystery Game PuzzlesReasoning | 29.00%o3-2025-04-16_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · mystery_game_puzzles.csv Reported standard error: ±4.56 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Mystery Game PuzzlesReasoning | 23.00%o3-2025-04-16_medium Epoch AI evaluation · epoch-run Run: 2026-08-27 · mystery_game_puzzles.csv Reported standard error: ±4.23 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Epoch Inspect | Epoch AI run ↗ |
|---|
| Aider polyglotCoding | 76.90%o3-2025-04-16_medium Aider LLM Leaderboards · external Run: Date not supplied · aider_polyglot_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| Aider polyglotCoding | 81.30%o3-2025-04-16_high https://aider.chat/docs/leaderboards/ · external Run: Date not supplied · aider_polyglot_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| Aider polyglotCoding | 76.90%o3-2025-04-16_unknown https://aider.chat/docs/leaderboards/ · external Run: Date not supplied · aider_polyglot_external.csv Inspect source ↗CC BY 4.0 | unknown / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 60.83%o3-2025-04-16_high https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 53.83%o3-2025-04-16_medium https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 41.50%o3-2025-04-16_low ARC Prize Leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| CadEvalOther | 74.00%o3-2025-04-16_medium CadEval Dashboard · external Run: Date not supplied · cad_eval_external.csv Assuming medium reasoning effort. Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| DeepResearch BenchAgents | 45.20%o3-2025-04-16_medium https://evals.futuresearch.ai/#drb · external Run: Date not supplied · deepresearchbench_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| Fiction.LiveBenchOther | 88.90%o3-2025-04-16_medium Fiction.live leaderboard · external Run: Date not supplied · fictionlivebench_external.csv Assuming medium reasoning effort Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| GeoBenchOther | 60.00%o3-2025-04-16_high GeoBench leaderboard · external Run: Date not supplied · geobench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| GeoBenchOther | 74.00%o3-2025-04-16_medium GeoBench leaderboard · external Run: Date not supplied · geobench_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| GSO-BenchCoding | 8.82%o3-2025-04-16_high https://gso-bench.github.io/index.html · external Run: Date not supplied · gso_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| OSWorldAgents | 23.00%o3-2025-04-16_medium OS World Website · external Run: Date not supplied · os_world_external.csv Inspect source ↗CC BY 4.0 | medium / o3 (100 steps) | External · via Epoch ↗ |
|---|
| OSWorldAgents | 17.20%o3-2025-04-16_medium OS World Website · external Run: Date not supplied · os_world_external.csv Inspect source ↗CC BY 4.0 | medium / o3 (50 steps) | External · via Epoch ↗ |
|---|
| OSWorldAgents | 9.10%o3-2025-04-16_medium OS World Website · external Run: Date not supplied · os_world_external.csv Inspect source ↗CC BY 4.0 | medium / o3 (15 steps) | External · via Epoch ↗ |
|---|
| SimpleBenchReasoning | 53.10%o3-2025-04-16_high SimpleBench Leaderboard · external Run: Date not supplied · simplebench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| VPCTOther | 52.00%o3-2025-04-16_medium VPCT leaderboard · external Run: Date not supplied · vpct_external.csv Assuming medium reasoning effort Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| WeirdMLCoding | 52.42%o3-2025-04-16_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.02 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 6.53%o3-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 2.98%o3-2025-04-16_medium External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 1.99%o3-2025-04-16_low External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | low / Not reported | External · via Epoch ↗ |
|---|
| HLEKnowledge | 20.32%o3-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · hle_external.csv Reported standard error: ±1.58 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| HLEKnowledge | 19.20%o3-2025-04-16_medium External result; evaluator not specified · external Run: Date not supplied · hle_external.csv Reported standard error: ±1.54 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| CritPtOther | 1.40%o3-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| GDPvalAgents | 30.80%o3-2025-04-16_medium External result; evaluator not specified · external Run: Date not supplied · gdpval_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| CL-benchOther | 17.80%o3-2025-04-16_high External result; evaluator not specified · external Run: Date not supplied · cl_bench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| METR Time HorizonsOther | 63.62%o3-2025-04-16_unknown External result; evaluator not specified · external Run: Date not supplied · metr_time_horizons_external.csv Inspect source ↗CC BY 4.0 | unknown / Not reported | External · via Epoch ↗ |
|---|
| METR Time HorizonsOther | 65.44%o3-2025-04-16_medium METR - Measuring AI Ability to Complete Long Tasks · external Run: Date not supplied · metr_time_horizons_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| LMCAReasoning | 39.70%o3-2025-04-16_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| DTBenchReasoning | 84.80%o3-2025-04-16_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| EnigmaEvalOther | 13.09%o3-2025-04-16_medium https://labs.scale.com/leaderboard/enigma_eval · external Run: Date not supplied · enigma_eval_external.csv Reported standard error: ±1.92 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| EnigmaEvalOther | 11.91%o3-2025-04-16_high https://labs.scale.com/leaderboard/enigma_eval · external Run: Date not supplied · enigma_eval_external.csv Reported standard error: ±1.85 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 1.99%o3 (Low) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.2343 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | low / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 2.98%o3 (Medium) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.4787 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | medium / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 6.53%o3 (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.8339 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | high / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|