Math & Optimization · beginner · concept 35 of 176
Loss Function
A function that measures how wrong the model's predictions are. Training = minimizing the loss. Common losses: MSE (regression), cross-entropy (classification), contrastive (embeddings). The formal name for training-by-average-loss is empirical risk minimization.
Interactive · 3D
See cross-entropy fill a physical meter →
Confident wrongness costs 4.61; confident rightness costs 0.36. Watch it live in Train mode.
Key terms
MSECross-entropyContrastive lossObjective functionEmpirical risk minimization
Learn these first
Videos
▶ But what is cross-entropy? | Compression is Intelligence Part 2 ↗
3Blue1Brown · YouTube
▶ Gradient descent, how neural networks learn | Deep Learning Chapter 2 ↗
3Blue1Brown · YouTube
Guides and articles
Courses, papers, and more
This unlocks