A polished answer can conceal a weak source, missing retrieval or an evaluator that rewards verbosity. This edition is about retaining enough evidence to know what improved and what failed.
01 / Curate before you summarize
Gemini Notebook's study features make source-based work more convenient, but the source set still needs an owner. Ask for disagreement and unknowns, then verify the passages supporting important conclusions.
One practical moveWrite a one-page evidence brief with five inspected citations.
02 / Retrieval and generation fail differently
A good answer model cannot reliably use a passage that never entered its context. Compare lexical, dense and fused retrieval on exact IDs and paraphrases, then test the permission filters used in production.
One practical moveLabel ten queries with their required supporting evidence.
03 / Traces are sensitive data too
Langfuse and Phoenix can connect a result to model, retrieval and tool steps. The same traces may copy private source text and credentials. Redact at the producer and keep tracing secrets server-side.
One practical moveTest redaction with a synthetic secret before exporting production traces.
04 / A green eval is only useful if red is possible
promptfoo's CLI documents evaluation failures and exit behavior. Test the integration by deliberately breaking a known assertion. Keep critical failures separate from an aggregate pass rate.
One practical moveMake one regression fail your release pipeline and inspect its artifact.
A compact evidence trail makes future model changes cheaper to evaluate. Save the task, source, version and failure—not just the final answer.