Bag-of-Words & TF-IDF

← Sentiment Analysis Large Language Models (LLMs) →

The pre-neural way to turn text into numbers: count the words, weight the rare ones higher (TF-IDF), ignore the order. Crude and still load-bearing, since BM25, its direct descendant, powers the sparse half of hybrid retrieval in modern RAG stacks and remains the search baseline to beat.

Key terms

Term frequencyTF-IDFBM25Sparse vectorsN-grams

Learn these first