Math & Optimization · beginner · concept 54 of 311 · Updated Nov 4, 2026
Linear Algebra for ML
The substrate everything runs on: data is vectors, models are matrices, and a forward pass is matrix multiplication. You need a working feel for dot products, matrix shapes, and eigenvectors, not proofs. GPUs exist because this one operation dominates all of AI. SVD, the workhorse matrix decomposition, is the machinery behind PCA and the low-rank idea LoRA reuses.
Top AI contributors on this
- Michael I. JordanInria Paris / UC Berkeley (emeritus)
- John HopfieldPrinceton University (emeritus)
- Cassie KozyrkovIndependent (formerly Google)
- Chelsea FinnStanford University / Physical Intelligence
- Daphne Kollerinsitro
- Sébastien BubeckOpenAI
Key terms
Where you meet it in the real world
Every single forward pass, embeddings, attention, PCA, graphics, and the GPU market
Videos
3Blue1Brown · YouTube
3Blue1Brown · YouTube
Guides and articles
3Blue1Brown
Mathematics for Machine Learning (free book)
3Blue1Brown (site)
Courses, papers, and more