Math & Optimization · beginner · concept 51 of 311 · Updated Nov 4, 2026
Loss Function
A function that measures how wrong the model's predictions are. Training = minimizing the loss. Common losses: MSE (regression), cross-entropy (classification), contrastive (embeddings). The formal name for training-by-average-loss is empirical risk minimization.
Interactive · 3D
See cross-entropy fill a physical meter →
Confident wrongness costs 4.61; confident rightness costs 0.36. Watch it live in Train mode.
Top AI contributors on this
- Geoffrey HintonUniversity of Toronto (emeritus)
- Yoshua BengioLawZero / Université de Montréal
- Yann LeCunAMI Labs / NYU
- Andrew NgDeepLearning.AI / AI Fund
- Ilya SutskeverSafe Superintelligence Inc. (SSI)
- Andrej KarpathyAnthropic
Key terms
MSECross-entropyContrastive lossObjective functionEmpirical risk minimization
Learn these first
Videos
▶ But what is cross-entropy? | Compression is Intelligence Part 2 ↗
3Blue1Brown · YouTube
▶ Gradient descent, how neural networks learn | Deep Learning Chapter 2 ↗
3Blue1Brown · YouTube
Guides and articles
Courses, papers, and more