Word2Vec & Word Embeddings

← Large Language Models (LLMs)Retrieval-Augmented Generation (RAG) →

Representing words as dense vectors where semantic relationships are captured geometrically. The famous result: king - man + woman ≈ queen. Predecessor to modern embeddings used in LLMs. Before neural embeddings, LSA reached similar ground statistically by factorizing the term-document matrix with SVD.

Key terms

Skip-gramCBOWGloVeCosine similarityEmbedding spaceLSA (latent semantic analysis)

Learn these first