DROPGoogle launches Gemini 3.8 Flash TTS and Flash-Lite TTS///DROPGemini Omni brings 1080p generation to Google Vids///BREAKClaude Opus 5.5 is here: 1M context, $4 input / $20 output///BREAKOpenAI adds GPT-6 Sol and Luna to the GPT-6 family///PAPEROpus 5.5's system card documents gains—and evaluation limits///BREAKGrok 4.7 launches at $2 input / $6 output per million tokens///DROPGrok Voice Transcribe 2.0 targets difficult real-world audio///TRENDGoogle steers new Gemini API projects to 3.8 Flash or 3.5 Flash-Lite///DROPGoogle launches Gemini 3.8 Flash TTS and Flash-Lite TTS///DROPGemini Omni brings 1080p generation to Google Vids///BREAKClaude Opus 5.5 is here: 1M context, $4 input / $20 output///BREAKOpenAI adds GPT-6 Sol and Luna to the GPT-6 family///PAPEROpus 5.5's system card documents gains—and evaluation limits///BREAKGrok 4.7 launches at $2 input / $6 output per million tokens///DROPGrok Voice Transcribe 2.0 targets difficult real-world audio///TRENDGoogle steers new Gemini API projects to 3.8 Flash or 3.5 Flash-Lite///
DARC.DEV
All guides
MethodologyLevel · Build#Cost#Optimization#Agents

Design a model router around quality, cost and current prices

Routing, context budgets and caching as a measurable system. Compare accepted-task cost while preserving a fixed quality threshold.

DCDariusz Czechowicz23 Sept 20263 min readEvaluation design
52
Methodology
Source-checked Sources & corrections
2 practical sections · checklist · sources

Evaluation design. Provider claims are attributed to the sources below. This page does not claim a new independent DARC benchmark.

For: Developers and teams putting AI into practical use

⚡ TL;DR — the 30-second version

  • Use externally checkable escalation criteria.
  • Measure accepted-task cost and retries.
  • Treat published token rates as inputs, not completed-task economics.

Split workloads by risk and verifiability. Extraction with a strict schema is easier to validate automatically than open-ended advice. Define escalation criteria that are checked by your application, rather than trusting an uncalibrated model confidence claim.

Current prices suggest useful candidates, not automatic winners: GPT-6 Luna targets high-volume tasks, Sol offers another capability tier, and DeepSeek's peak/off-peak rates can change a batch workload's economics. More expensive models can still cost less per accepted result when retries fall.

“Cost reduction is only a win if the work still meets its acceptance criteria.”
— Editorial interpretation

Measure output quality, retries, review time, cached tokens and all billing categories on the same fixed task set. Keep a direct-to-frontier control group and record the quality/cost trade-off instead of reporting savings alone.

Use the Token Thermostat cost workbench to explore a bounded text workload before running it. Its output is an estimate from stated inputs, not an invoice or a prediction of quality. Use the result analyzer once you have actual labeled runs.

Field guide / 01

Use cost per accepted task

Sum model, tool and retry charges for all attempts, then divide by the number of accepted results. Do not remove failed runs from the numerator. If no task is accepted, the ratio is undefined—not zero.

Keep human review time alongside the invoice figure. A lower token bill can simply move work from the model to an operator, which may or may not be the right trade-off.

Field guide / 02

Test routing as a separate policy

Freeze the model and prompt configurations, then compare direct-to-frontier work with a bounded escalation rule. Log which condition caused escalation and whether the final result improved. Model self-confidence is not enough unless it has been calibrated.

Include peak/off-peak pricing, context thresholds and cache behavior where applicable. The Lab's estimator exposes input assumptions and exclusions so a scenario estimate is not mistaken for a provider invoice.

Before you ship

0/5

Use this as a review aid. Checks reset when you leave this page.

Questions that come up

Is the cheapest token rate the cheapest workflow?

No. Output length, retries, cache reuse, tools and review effort determine the total. Compare accepted-task cost with a quality threshold.

Put it into practiceUse the cost workbench

Sources & verification

Last checked 23 Sept 2026. Publication dates and access conditions belong to the linked originals.

  1. 01GPT-6 Luna API model and pricingOpenAI↗
  2. 02GPT-6 Sol API model and pricingOpenAI↗
  3. 03DeepSeek models and pricingDeepSeek↗
  4. 04Introducing Claude Opus 5.5Anthropic↗
  5. 05LiteLLM — budgets, limits and virtual keysLiteLLM↗
  6. 06OpenAI — current prompt-caching behaviorOpenAI↗
Editorial policy and correction log →
#Cost#Optimization#Agents