Architectures · advanced · concept 67 of 176
Model Quantization & Compression
Reducing model size and computational cost by using lower-precision numbers (e.g., 32-bit → 4-bit floats). Enables running LLMs on consumer hardware. GPTQ, GGUF, and AWQ are popular methods.
Key terms
INT8INT4GPTQGGUFAWQMixed precision
Learn these first
Where you meet it in the real world
Running LLMs locally, mobile AI, edge deployment
Videos
▶ LLM Compression Explained: Build Faster, Efficient AI Models ↗
IBM Technology · YouTube
Guides and articles
Courses, papers, and more
This unlocks