Math & Optimization · intermediate · concept 59 of 311 · Updated Nov 4, 2026
Embeddings
Dense vector representations that capture semantic meaning. Words, images, and even users can be embedded into continuous vector spaces where similar items are close together.
Interactive · 3D
Vectors in meaning-space, staged in 3D →
The embedding wall: token IDs become points where distance means similarity.
Top AI contributors on this
- Yoshua BengioLawZero / Université de Montréal
- Ilya SutskeverSafe Superintelligence Inc. (SSI)
- Noam ShazeerOpenAI
- Alec RadfordIndependent / Thinking Machines Lab (adviser)
- Jürgen SchmidhuberKAUST / IDSIA
- Oriol VinyalsDiscovery Loop
Key terms
Word2VecGloVeEmbedding spaceCosine similarityDimensionality
Learn these first
Where you meet it in the real world
Search engines, recommendation systems, RAG, semantic similarity
Videos
▶ Transformers, the tech behind LLMs | Deep Learning Chapter 5 ↗
3Blue1Brown · YouTube
▶ Word Embedding and Word2Vec, Clearly Explained!!! ↗
StatQuest with Josh Starmer · YouTube
Guides and articles
Embeddings ↗
Anthropic
OpenAI Cookbook ↗
OpenAI
Vector embeddings ↗
OpenAI
The Illustrated Word2vec ↗
Jay Alammar
Embeddings: what they are and why they matter ↗
Simon Willison
What are embeddings? ↗
Vicki Boykis
Getting Started With Embeddings ↗
Hugging Face
This unlocks