Math & Optimization · advanced · concept 76 of 311 · Updated Nov 4, 2026
The Lottery Ticket Hypothesis
Frankle and Carbin (2018) found that a dense, randomly initialized network contains a sparse subnetwork (a winning ticket) which, trained alone from the same initial weights, matches the full network's accuracy in as few steps. They found tickets by iterative magnitude pruning, often at 10 to 20 percent of the original size. It won a best paper award at ICLR 2019; deep networks turned out to need rewinding to an early checkpoint, not step zero.
Top AI contributors on this
- Geoffrey HintonUniversity of Toronto (emeritus)
- Yoshua BengioLawZero / Université de Montréal
- Yann LeCunAMI Labs / NYU
- Andrew NgDeepLearning.AI / AI Fund
- Ilya SutskeverSafe Superintelligence Inc. (SSI)
- Andrej KarpathyAnthropic
Key terms
Learn these first
Where you meet it in the real world
Network pruning and compression, sparse training research, on-device model size reduction, understanding why overparameterized networks train well
Videos
Yannic Kilcher · YouTube
JumpTrading ELLIS UCL CSML Seminar Series · YouTube
Yannic Kilcher · YouTube
Guides and articles
Courses, papers, and more