Practical guide. Provider claims are attributed to the sources below. This page does not claim a new independent DARC benchmark.
For: Developers turning model responses into application data
Before you start: Basic JSON and HTTP familiarity
⚡ TL;DR — the 30-second version
- Represent missing values instead of forcing a guess.
- Validate transport, structure, domain rules and evidence separately.
- Keep extraction distinct from consequential side effects.
A schema solves one class of problem: making the response predictable enough for software to consume. It does not prove that an extracted date appeared in the document, that a category is appropriate or that a total is correct. A reliable pipeline separates format, domain validation and evidence.
Consider an invoice intake assistant. It must distinguish an absent due date from a guessed one, return the source span supporting each field, and route ambiguous cases to review. That requires a contract designed for uncertainty rather than a prompt that demands every field.
“Syntactically valid JSON can still be semantically wrong. Validate the claim, not only the brackets.”
Field guide / 01
01 / Make uncertainty representable
Define allowed values, nullability and an explicit review state before calling the model. Avoid arbitrary confidence percentages unless you have calibrated them against labeled data. An evidence excerpt and a reason for review are usually more useful to an operator.
Do not let schema design force fabrication. If a field can be absent in real inputs, the schema needs an absent state. Keep identifiers as strings when leading zeros matter.
{
"invoice_id": "INV-1042",
"due_date": null,
"currency": "EUR",
"total": "120.00",
"review_required": true,
"review_reason": "No explicit due date in source",
"evidence": { "total": "Total payable: EUR 120.00" }
}Field guide / 02
02 / Validate in layers
First handle transport errors, refusals and truncated responses. Next parse and validate the schema. Then apply deterministic domain rules: dates must parse, currencies must be allowed, and invoice arithmetic must reconcile within your defined tolerance.
Finally check provenance. Does the cited text actually occur in the input? Does it support the field? A syntactically correct number with a fabricated excerpt is a failed extraction, even if downstream code could accept it.
| Failure | Response | Retry? |
|---|---|---|
| Network timeout | Retry within a bounded policy | Usually, with idempotency |
| Malformed structure | Record provider/version and validate again | At most a limited repair |
| Missing evidence | Route to human review | Not by guessing |
| Contradictory totals | Preserve both values and flag | Only with new evidence |
Field guide / 03
03 / Keep extraction away from payment
Do not combine 'read the invoice' and 'pay it' into one unconstrained tool. The output should enter an ordinary approval flow with record ownership, duplicate detection and an audit trail. The same invoice submitted twice should not create two payments.
Use an idempotency key derived from an internal document identity and a stable operation identifier. A model-generated key is not a substitute for the application's own transaction semantics.
Field guide / 04
04 / Test the cases that break the happy path
Include blank documents, unreadable scans, duplicate invoices, different locale formats, refunds and adversarial text embedded in the file. Report field-level accuracy and the review burden, not only whether a JSON parser succeeded.
Version the prompt, schema, model and extraction code together. If you change an enum or nullability, test the consumers as well as the model. A migration that improves extraction but breaks a billing job is still a regression.
Before you ship
0/5Use this as a review aid. Checks reset when you leave this page.
Questions that come up
Should every bad output be sent back for repair?
No. Formatting problems may justify a bounded repair. Missing evidence or an unauthorized action should be rejected or reviewed, not repeatedly prompted into compliance.
Sources & verification
Last checked 23 Sept 2026. Publication dates and access conditions belong to the linked originals.
- 01GPT-6 Sol API model and pricingOpenAI↗
- 02GPT-6 Luna API model and pricingOpenAI↗
- 03Gemini 3.8 Flash model specificationsGoogle↗
- 04promptfoo — evaluations and red teamingpromptfoo↗