Architectures · intermediate · concept 85 of 311 · Updated Nov 4, 2026
GPT (Generative Pre-trained Transformer)
A decoder-only Transformer trained to predict the next token. Each GPT generation, from GPT-1 through GPT-5 and beyond, showed that scaling this simple idea produces increasingly capable AI. The architecture behind ChatGPT.
Top AI contributors on this
- Dario AmodeiAnthropic
- Sam AltmanOpenAI
- Jeff DeanDiscovery Loop
- Noam ShazeerOpenAI
- Alec RadfordIndependent / Thinking Machines Lab (adviser)
- Oriol VinyalsDiscovery Loop
Key terms
AutoregressiveNext-token predictionDecoder-onlyScaling laws
Learn these first
Videos
▶ Deep Dive into LLMs like ChatGPT ↗
Andrej Karpathy · YouTube
▶ Large Language Models explained briefly ↗
3Blue1Brown · YouTube
Guides and articles
Courses, papers, and more
This unlocks