Token
A model-specific unit of input or output representation. It is not a universal word count; measure with the selected tokenizer or usage report.
[ REF — The field glossary ]
Thirty compact definitions for people using and building AI. Each makes a practical distinction—not a claim that the terminology is standardized across providers.
30 terms · definitions for practical use
A model-specific unit of input or output representation. It is not a universal word count; measure with the selected tokenizer or usage report.
The capacity available to a request. Some providers quote a total window, others separate input and output limits.
Running a trained model on inputs to produce outputs. It has latency, memory and compute costs even when the weights are downloadable.
A provider-specific control over reasoning behavior or computation. Labels are not standardized across models.
Reuse of computed prefix state. It is not a saved final answer and does not guarantee deterministic output.
Input work stored for possible reuse. Depending on the provider it has a distinct billing rate, not necessarily an additive fee.
Attention-related runtime state associated with processed context. It consumes memory and is distinct from model weights.
Retrieval-augmented generation: selecting external evidence and supplying it to a model at inference time.
A numeric representation used for similarity and retrieval. Similarity does not prove truth or authorization.
A second-stage relevance assessment over retrieved candidates. It adds its own cost and latency.
Combining retrieval signals such as lexical matching and semantic vectors. It must be evaluated on real query types.
Connecting an answer to supplied or retrieved evidence. The evidence itself still needs quality and permission checks.
Output constrained to a schema or format. It can be syntactically valid and semantically incorrect.
A model-proposed request for an external operation. The application must validate and authorize it before execution.
A system that repeatedly uses model decisions, tools and observations toward a goal, with explicit limits and stop states.
The software surrounding a model: prompts, tools, execution loop, retries, context management and controls.
Model Context Protocol: a standardized connection between hosts/clients and tool or resource servers. It does not replace application authorization.
An attempt to make untrusted content influence behavior outside its intended role, often by posing as instructions.
An execution boundary restricting resources or actions. It limits reach, not necessarily business-logic mistakes.
A property that repeated execution of the same logical operation does not repeat its external effect.
Persisted workflow state used for recovery or human review. It does not prove that an external write happened exactly once.
A defined test of behavior against a rubric or expected result. Its value depends on the task set, judge and run conditions.
Using a model to grade outputs. The judge needs calibration, bias checks and its own observability.
Accepted cases divided by attempted cases under a stated rubric. It is not a universal measure of safety or intelligence.
Total cost of all evaluated attempts divided by accepted results. Undefined when no results are accepted.
A released model checkpoint under a specific license. Not a promise of unrestricted use, open training data or free serving.
Representing weights or runtime state with reduced precision to change memory and performance trade-offs.
Mixture of Experts: an architecture that activates part of a larger model per token. Active parameters are not the full storage requirement.
Time to first token. It is only one part of end-to-end user latency; tools, queueing and speech output can dominate.
Traceable evidence of what a system did, how long it took and what it cost. It does not by itself judge correctness.