Practical guide. Provider claims are attributed to the sources below. This page does not claim a new independent DARC benchmark.
For: Builders who need control over where data is processed
Before you start: A machine with sufficient RAM/VRAM · An approved model license
⚡ TL;DR — the 30-second version
- Audit embeddings and tools as well as inference.
- Plan full memory requirements, not just active parameters.
- Test offline behavior and secure the serving endpoint.
Running a model on your own machine can keep inference local, but a document assistant may still call a cloud embedding service, a search API or a remote tool. Modern local-model products also expose cloud options. Audit the whole path from file ingestion to output.
The current Ollama docs explicitly distinguish local and cloud models. LM Studio exposes local APIs and MCP integrations. Open WebUI's air-gapped guidance lists the functions that still require external services. These are useful starting points for a deployment checklist.
“Local inference is a location. Privacy is a property of the entire workflow.”
Field guide / 01
01 / Inventory every network dependency
List model downloads, runtime updates, embeddings, rerankers, authentication, telemetry, web search and tool servers. Mark which are used during setup and which are used for every request. A one-time model download is different from uploading every document for embedding.
Repeat the exercise after adding plugins. A previously local workflow can change when a convenient connector is enabled.
Field guide / 02
02 / Budget weights and working memory
Estimate raw weight memory from parameter count and precision, then add runtime overhead and context state. Quantization reduces weight storage but does not remove KV cache or concurrency costs. An active-parameter count for an MoE model is not its full storage requirement.
Use the Lab memory estimator as a lower-bound planning tool, not a fit guarantee. Confirm with the runtime's actual loaded-memory measurement before committing to hardware.
Field guide / 03
03 / Secure the local API
Verify the bind address and authentication before making inference available to another device. An unauthenticated server on a trusted-looking network can still be reached by malicious local software or an overly broad network rule.
Separate ordinary users from model-management and administrative operations. Test whether an untrusted caller can download models, alter prompts or access another user's documents.
Field guide / 04
04 / Test offline and plan updates
Disconnect outbound access in a test environment and run representative tasks. Document which fail and why. Do not label the setup air-gapped if it requires a cloud model, remote search or external identity service for the core workflow.
Keep model/license versions, backup rules and a runtime update process. Local control includes the obligation to patch the software and delete data when it is no longer needed.
Before you ship
0/5Use this as a review aid. Checks reset when you leave this page.
Sources & verification
Last checked 23 Sept 2026. Publication dates and access conditions belong to the linked originals.
- 01Ollama — local and cloud models, API and integrationsOllama↗
- 02LM Studio — local APIs and SDKsLM Studio↗
- 03Open WebUI — offline operation and network boundariesOpen WebUI↗
- 04Qwen3.8-Flash-Next official model cardQwen↗
