DROPGoogle launches Gemini 3.8 Flash TTS and Flash-Lite TTS///DROPGemini Omni brings 1080p generation to Google Vids///BREAKClaude Opus 5.5 is here: 1M context, $4 input / $20 output///BREAKOpenAI adds GPT-6 Sol and Luna to the GPT-6 family///PAPEROpus 5.5's system card documents gains—and evaluation limits///BREAKGrok 4.7 launches at $2 input / $6 output per million tokens///DROPGrok Voice Transcribe 2.0 targets difficult real-world audio///TRENDGoogle steers new Gemini API projects to 3.8 Flash or 3.5 Flash-Lite///DROPGoogle launches Gemini 3.8 Flash TTS and Flash-Lite TTS///DROPGemini Omni brings 1080p generation to Google Vids///BREAKClaude Opus 5.5 is here: 1M context, $4 input / $20 output///BREAKOpenAI adds GPT-6 Sol and Luna to the GPT-6 family///PAPEROpus 5.5's system card documents gains—and evaluation limits///BREAKGrok 4.7 launches at $2 input / $6 output per million tokens///DROPGrok Voice Transcribe 2.0 targets difficult real-world audio///TRENDGoogle steers new Gemini API projects to 3.8 Flash or 3.5 Flash-Lite///
DARC.DEV
All guides
GuideLevel · Build#Observability#Privacy#Langfuse

Trace your AI application without creating a second data leak

A minimal observability plan with useful spans, version metadata, cost accounting and redaction before export.

DCDariusz Czechowicz23 Sept 20263 min readPractical guide
10
Guide
Source-checked Sources & corrections
4 practical sections · checklist · sources

Practical guide. Provider claims are attributed to the sources below. This page does not claim a new independent DARC benchmark.

For: Developers operating LLM applications

Before you start: A request or job identifier · An inventory of sensitive data

⚡ TL;DR — the 30-second version

  • Trace steps and decisions, not only final text.
  • Redact before telemetry leaves the application.
  • Treat trace stores and evaluation datasets as sensitive systems.

When an answer is wrong, you need to know whether the failure came from retrieval, the model, a tool, permissions or a downstream consumer. A single log line containing the final response cannot explain that chain. Structured traces can—but they can also copy private documents and credentials into a second system.

Langfuse and Phoenix both document model-aware tracing and evaluation workflows. Choose an observability backend after defining the information you actually need and the data you are not allowed to retain.

“A perfect trace of an unsafe data flow is still an unsafe data flow. Redaction happens before export.”
— Editorial interpretation

Field guide / 01

01 / Trace the user-visible job

Use one trace for a complete user job, with child spans for retrieval, model requests, tool calls, validation and writes. Connect retries to their original operation. A latency number is useful only when you can distinguish queueing, inference, tools and review.

Record model ID, prompt version, fixture or corpus version and final outcome. These identifiers let you compare deployments without retaining every raw prompt.

json
{
  "job_id": "opaque-internal-id",
  "model_id": "pinned-model-id",
  "prompt_version": "support-v3",
  "outcome": "review_required",
  "usage": { "input": 12000, "cached": 8000, "output": 600 },
  "redaction_policy": "support-minimal-v1"
}
Illustrative telemetry envelope. Token categories must be mapped to the selected provider's actual response.

Field guide / 02

02 / Redact at the producer

Do not rely on a dashboard to hide data already transmitted. Remove credentials, customer identifiers and unnecessary document text before sending telemetry. Store a controlled reference to the original material when a reviewer genuinely needs to inspect it.

The Langfuse SDK documentation says tracing credentials belong on the server. Exposing a secret key in a web app is not fixed by calling it an analytics key. Apply the same standard to evaluator traces and debugging exports.

Field guide / 03

03 / Separate usage from quality

A successful HTTP request means the provider returned a response. It says nothing about whether the answer was correct or a proposed action was permitted. Track validation outcomes, review decisions and task acceptance as separate fields.

Cost estimates should preserve the units the provider bills: ordinary input, cache reads, cache writes, output, searches or audio minutes. Reconcile estimates periodically with actual invoices and mark unknown charges rather than filling them with zero.

Field guide / 04

04 / Test deletion and access

Choose retention by data category and operational need. Test whether deleting a customer or document removes related raw trace content, evaluation copies and exports. Restrict who can read production traces, not just who can administer the dashboard.

Promote only privacy-reviewed examples into long-lived regression datasets. A failure can be worth learning from without keeping the customer's entire conversation forever.

Before you ship

0/5

Use this as a review aid. Checks reset when you leave this page.

Put it into practiceEvaluate a release on a fixed test set

Sources & verification

Last checked 23 Sept 2026. Publication dates and access conditions belong to the linked originals.

  1. 01Langfuse — LLM observabilityLangfuse↗
  2. 02Langfuse — server-side SDKs and secret handlingLangfuse↗
  3. 03Arize Phoenix — tracing and evaluationArize AI↗
  4. 04OpenAI — current prompt-caching behaviorOpenAI↗
Editorial policy and correction log →
#Observability#Privacy#Langfuse#Phoenix