NLP & Language · advanced · concept 96 of 176
LLM Scaling Laws
The empirical finding that LLM performance improves predictably with more data, compute, and parameters. Chinchilla scaling laws (2022) showed that most models were undertrained relative to their size.
Key terms
ChinchillaCompute-optimalEmergent abilitiesPower law
Learn these first
Videos
▶ Lec 20. Scaling Laws ↗
MIT OpenCourseWare · YouTube
▶ Stanford CS336 Language Modeling from Scratch | Spring 2026 | Lecture 9: Scaling Laws ↗
Stanford Online · YouTube
Guides and articles
2026-06-24-scaling-laws ↗
Lil'Log (OpenAI researcher)
Courses, papers, and more
This unlocks