Architectures · advanced · concept 107 of 311 · Updated Nov 4, 2026

🌫️Diffusion Language Models

Language models that generate text by refining a whole masked sequence over several steps instead of predicting one token at a time. Many tokens update in parallel, so sampling can be faster and earlier text can be revised. LLaDA trained an 8B masked diffusion model, and Inception's Mercury Coder Mini was measured at 1,109 tokens per second on an H100. Gemini Diffusion is Google DeepMind's experimental model of the same kind.