llama.cpp / Ollama / GGUF (edge inference)
Quantized local/edge inference for open models
llama.cpp, its GGUF quantization format, and the Ollama distribution wrapper are the standard stack for running open-weight models locally on laptops, phones, and edge devices without a datacenter GPU. They made on-device and privacy-preserving inference practical for developers. This is the counterweight to cloud-only serving.
Format
GGUF quantization
Use
Local/edge inference
How it fits the stack
llama.cpp / Ollama / GGUF (edge inference) with what it depends on (above) and what it feeds (below). The figure renders as a crawlable diagram and upgrades to an interactive 3D graph as it scrolls into view.
llama.cpp / Ollama / GGUF (edge inference) in the AI stack. llama.cpp / Ollama / GGUF (edge inference) with its immediate upstream dependencies (top) and downstream dependents (bottom) in the AI value chain. Hover a node in 3D, or read the full relationships below.
Graph data (text) — 3 entities, 2 relationships
- llama.cpp / Ollama / GGUF (edge inference) —uses→ Llama
- llama.cpp / Ollama / GGUF (edge inference) —uses→ Qwen