Practical guide. Provider claims are attributed to the sources below. This page does not claim a new independent DARC benchmark.
For: Developers and teams putting AI into practical use
⚡ TL;DR — the 30-second version
- Set freshness and authority rules before tuning top-k.
- Evaluate correctness and citations together.
- Treat context limits and pricing as separate constraints.
Retrieval remains useful when the assistant needs facts that are private, current or too numerous to place into every prompt. Context engineering broadens that task: it includes what to retrieve, source authority, freshness, conversation state, tool results and the budget for all of it.
Begin with a realistic task set and answer provenance. Decide which source wins when documents disagree, what expires, and when the assistant should say that the available evidence is insufficient. If a newer runbook contradicts an old FAQ, a vector similarity score should not silently decide policy.
“A bigger context window is capacity, not a guarantee that every fact will be retrieved, trusted or used correctly.”
Large windows do not remove the design problem. Current provider documentation distinguishes total context from separate input and output limits, and pricing may change for very long prompts. Compress deliberately, keep citations inspectable and test whether omitted material matters to the answer.
Compare retrieval strategies on your own fixed evaluation set and publish the failures. This guide makes no claim that a specific architecture achieves 89% accuracy, or that retrieval is universally obsolete.
Field guide / 01
Write a context policy before tuning retrieval
For each source, define its authority, owner, access rules and expiry. A fresh external article should not override an internal policy solely because its embedding is more similar to the question. Keep conflicting evidence visible when the application cannot resolve it safely.
Set a budget for instructions, retrieved evidence, tool outputs and final answer space. If a category exceeds its budget, choose what to compress or omit deliberately and test the effect on answer quality.
Field guide / 02
Diagnose the first failing stage
If the right passage never reaches the candidate set, improve ingestion or retrieval. If it is retrieved but removed during compression, fix context assembly. If it reaches the model and the answer still contradicts it, examine instructions, ambiguity and generation behavior.
Label these stages in traces and evaluation reports. One aggregate accuracy number can conceal three different engineering problems with different fixes.
Before you ship
0/5Use this as a review aid. Checks reset when you leave this page.
Questions that come up
Can I just put every document into a million-token prompt?
Sometimes a large context is useful, but cost, latency, conflicting evidence and access constraints remain. Evaluate it against a retrieval baseline rather than assuming the window solves the problem.
Sources & verification
Last checked 23 Sept 2026. Publication dates and access conditions belong to the linked originals.
- 01Claude models overviewAnthropic↗
- 02GPT-6 Astra API model and pricingOpenAI↗
- 03Gemini 3.8 Flash model specificationsGoogle↗
- 04LlamaIndex — response and retrieval evaluationLlamaIndex↗
- 05Qdrant — hybrid semantic and lexical searchQdrant↗
