MLOps & Infrastructure · intermediate · concept 219 of 311 · Updated Nov 4, 2026
Offline and Online Evaluation
Offline evaluation scores a model on held-out data before it ships. Online evaluation measures it on live traffic, usually through an A/B test or a shadow deployment. The two often disagree, because offline data is a frozen snapshot while real users react to what the model does. A recommender that wins on offline recall can lose on live click-through. Teams use offline tests as a cheap filter and an online test as the final verdict.
Top AI contributors on this
- Michael I. JordanInria Paris / UC Berkeley (emeritus)
- John HopfieldPrinceton University (emeritus)
- Cassie KozyrkovIndependent (formerly Google)
- Chelsea FinnStanford University / Physical Intelligence
- Daphne Kollerinsitro
- Sébastien BubeckOpenAI
Key terms
Learn these first
Where you meet it in the real world
Recommender and search ranking rollouts, LLM prompt and model upgrades, fraud model promotion, ad click prediction
Videos
San Francisco Bay ACM · YouTube
Stanford MLSys Seminars · YouTube
Evidently AI · YouTube
Guides and articles
Courses, papers, and more