| GPQA diamondKnowledge | 88.89%gemini-3.5-flash_low Epoch AI evaluation · epoch-run Run: 2026-08-06 · gpqa_diamond.csv Reported standard error: ±2.24 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 86.36%gemini-3.5-flash_minimal Epoch AI evaluation · epoch-run Run: 2026-07-15 · gpqa_diamond.csv Reported standard error: ±2.45 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | minimal / Epoch Inspect | Epoch AI run ↗ |
|---|
| GPQA diamondKnowledge | 92.80%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-05-22 · gpqa_diamond.csv Reported standard error: ±1.64 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-2025-02-28-PrivateReasoning | 38.97%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-05-22 · frontiermath.csv Reported standard error: ±2.87 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-2025-07-01-PrivateReasoning | 14.58%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-05-25 · frontiermath_tier_4.csv Reported standard error: ±5.15 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tiers-1-3-v2-PrivateReasoning | 62.81%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-06-10 · frontiermath_tiers_1_3_v2.csv Reported standard error: ±2.87 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| FrontierMath-Tier-4-v2-PrivateReasoning | 26.83%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-06-10 · frontiermath_tier_4_v2.csv Reported standard error: ±7.01 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 88.89%gemini-3.5-flash_low Epoch AI evaluation · epoch-run Run: 2026-08-06 · otis_mock_aime_2024_2025.csv Reported standard error: ±4.74 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 80.00%gemini-3.5-flash_minimal Epoch AI evaluation · epoch-run Run: 2026-07-15 · otis_mock_aime_2024_2025.csv Reported standard error: ±6.03 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | minimal / Epoch Inspect | Epoch AI run ↗ |
|---|
| OTIS Mock AIME 2024-2025Reasoning | 95.56%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-05-25 · otis_mock_aime_2024_2025.csv Reported standard error: ±2.67 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| SWE-Bench verifiedCoding | 79.34%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-06-01 · swe_bench_verified.csv Reported standard error: ±1.84 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| SimpleQA VerifiedKnowledge | 66.20%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-08-27 · simpleqa_verified.csv Reported standard error: ±1.50 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 45.00%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±5.00 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 42.00%gemini-3.5-flash_low Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.96 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| Chess PuzzlesReasoning | 27.00%gemini-3.5-flash_minimal Epoch AI evaluation · epoch-run Run: 2026-10-13 · chess_puzzles.csv Reported standard error: ±4.46 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | minimal / Epoch Inspect | Epoch AI run ↗ |
|---|
| EBR-benchOther | 4.76%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-06-25 · ebr_bench.csv Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| Mystery Game PuzzlesReasoning | 28.00%gemini-3.5-flash_low Epoch AI evaluation · epoch-run Run: 2026-08-05 · mystery_game_puzzles.csv Reported standard error: ±4.51 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | low / Epoch Inspect | Epoch AI run ↗ |
|---|
| Mystery Game PuzzlesReasoning | 32.00%gemini-3.5-flash_high Epoch AI evaluation · epoch-run Run: 2026-07-27 · mystery_game_puzzles.csv Reported standard error: ±4.69 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Epoch Inspect | Epoch AI run ↗ |
|---|
| ARC-AGIReasoning | 92.50%gemini-3.5-flash_high https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGIReasoning | 48.83%gemini-3.5-flash_minimal https://arcprize.org/leaderboard · external Run: Date not supplied · arc_agi_external.csv Inspect source ↗CC BY 4.0 | minimal / Not reported | External · via Epoch ↗ |
|---|
| SimpleBenchReasoning | 76.70%gemini-3.5-flash_unknown SimpleBench Leaderboard · external Run: Date not supplied · simplebench_external.csv Inspect source ↗CC BY 4.0 | unknown / Not reported | External · via Epoch ↗ |
|---|
| WeirdMLCoding | 62.64%gemini-3.5-flash_high https://htihle.github.io/weirdml.html · external Run: Date not supplied · weirdml_external.csv Reported standard error: ±0.02 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| APEX-AgentsAgents | 27.50%gemini-3.5-flash_unknown External result; evaluator not specified · external Run: Date not supplied · apex_agents_external.csv Inspect source ↗CC BY 4.0 | unknown / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 72.08%gemini-3.5-flash_high External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2Reasoning | 8.89%gemini-3.5-flash_minimal External result; evaluator not specified · external Run: Date not supplied · arc_agi_2_external.csv Inspect source ↗CC BY 4.0 | minimal / Not reported | External · via Epoch ↗ |
|---|
| GBAEvalOther | 6.70%gemini-3.5-flash_unknown https://gbaeval.com/model/goose-gemini-3-5-flash · external Run: Date not supplied · gbaeval_external.csv Inspect source ↗CC BY 4.0 | unknown / Not reported | External · via Epoch ↗ |
|---|
| CritPtOther | 13.14%gemini-3.5-flash_high External result; evaluator not specified · external Run: Date not supplied · critpt_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| Surface Evolver BenchOther | 58.13%gemini-3.5-flash_medium https://yhenon.github.io/surface-evolver-llm-eval/ · external Run: Date not supplied · surface_evolver_bench_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| DeepSWECoding | 37.39%gemini-3.5-flash_medium https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | medium / mini-swe-agent | External · via Epoch ↗ |
|---|
| DeepSWECoding | 36.06%gemini-3.5-flash_high https://deepswe.datacurve.ai/ · external Run: Date not supplied · deepswe_external.csv Inspect source ↗CC BY 4.0 | high / mini-swe-agent | External · via Epoch ↗ |
|---|
| ProofBenchReasoning | 31.00%gemini-3.5-flash_high https://www.vals.ai/benchmarks/proof_bench · external Run: Date not supplied · proofbench_external.csv Reported standard error: ±4.65 pp. This is not a 90% interval. ProofBench v1.1 re-grade (vals.ai, 2026-08-14): corrected Lean formalizations + hardened grader (rejects sorry/admit shortcuts, larger elaboration budgets). Not comparable to v1.0-era values. Updated 2026-08-18 after multi-agent verification. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| LMCAReasoning | 47.15%gemini-3.5-flash_high https://conceptualreasoning.ai/lmca · external Run: Date not supplied · lmca_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| DTBenchReasoning | 94.67%gemini-3.5-flash_high https://conceptualreasoning.ai/dtbench · external Run: Date not supplied · dtbench_external.csv Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| GDP.pdfOther | 14.00%gemini-3.5-flash_unknown https://surgehq.ai/benchmarks/gdp-pdf · external Run: Date not supplied · gdp_pdf_external.csv Inspect source ↗CC BY 4.0 | unknown / Not reported | External · via Epoch ↗ |
|---|
| GDP.pdfOther | 14.00%gemini-3.5-flash_medium https://surgehq.ai/benchmarks/gdp-pdf · external Run: Date not supplied · gdp_pdf_external.csv Inspect source ↗CC BY 4.0 | medium / Not reported | External · via Epoch ↗ |
|---|
| EnigmaEvalOther | 25.41%gemini-3.5-flash_high https://labs.scale.com/leaderboard/enigma_eval · external Run: Date not supplied · enigma_eval_external.csv Reported standard error: ±2.45 pp. This is not a 90% interval. Inspect source ↗CC BY 4.0 | high / Not reported | External · via Epoch ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 8.89%Gemini 3.5 Flash (Minimal) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.1074 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | minimal / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|
| ARC-AGI-2 · semi-privateReasoning | 72.08%Gemini 3.5 Flash (High) ARC Prize official leaderboard · benchmark-publisher Run: Date not supplied · evaluations.json · v2_Semi_Private Reported benchmark cost: $0.8503 per task. Different from the token-workload estimate. Semi-private split only. Provider-adapter and custom-scaffold families are kept separate. Identical effort names do not imply identical compute budgets. Inspect source ↗Public leaderboard facts · ARC Prize | high / ARC Prize leaderboard baseline | ARC Prize ↗ |
|---|