Topic

Transformers

The transformer is the neural network design behind today's large language models. Introduced in 2017, it replaced reading word by word with attention, so every word can look at every other word at once. That made training far easier to run in parallel and to scale up. The guide here lets you step through one in 3D.

1 guide on Transformers.

Reference Guide · 18 min read

How LLMs Work

Every stage a prompt passes through, modeled as an explorable 3D machine. 21 stages, a 21-step guided journey tracing one token from "The cat sat on the" to "mat", real parameter counts from GPT-3 to DeepSeek-V3, and the training story, pretraining, RLHF, and the reasoning-RL breakthrough.

LLM Transformers Attention Mixture of Experts Reasoning Models 3D Interactive AI Explainer

Keep learning Transformers

8: Deep Learning for Natural Language – Transformers, Self-Supervised Learning video · MIT OpenCourseWare
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Paper Explained) video · Yannic Kilcher
Attention in transformers, step-by-step | Deep Learning Chapter 6 video · 3Blue1Brown
Encoder-Only Transformers (like BERT) for RAG, Clearly Explained!!! video · StatQuest with Josh Starmer
Illustrated Guide to Transformers · The AI Hacker video · The AI Hacker
RoPE: Understanding Rotary Positional Embeddings in transformers video · Hugging Face
Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 6 - LLM Reasoning video · Stanford Online
Stanford CME295 Transformers & LLMs | Autumn 2025 | Lecture 8 - LLM Evaluation video · Stanford Online

From the AI Roadmap. Every link was checked.

3D Interactive AI Explainer Attention LLM Mixture of Experts Reasoning Models

All Guides

Transformers

How LLMs Work

Keep learning Transformers

Related topics