Alignment Methods (RLHF, Constitutional AI, DPO)
The post-training techniques that make models usable & safe
RLHF, Anthropic's Constitutional AI/RLAIF, Direct Preference Optimization (DPO), and published model specs (OpenAI Model Spec) are the alignment and steering methods that turn a base model into a deployable assistant. Reasoning-era RL (RLVR, process rewards) extended this stack in 2025. These techniques are as decisive to product quality as pretraining scale.
Anthropic method
Constitutional AI
Simpler RLHF
DPO
How it fits the stack
Alignment Methods (RLHF, Constitutional AI, DPO) with what it depends on (above) and what it feeds (below). The figure renders as a crawlable diagram and upgrades to an interactive 3D graph as it scrolls into view.
Alignment Methods (RLHF, Constitutional AI, DPO) in the AI stack. Alignment Methods (RLHF, Constitutional AI, DPO) with its immediate upstream dependencies (top) and downstream dependents (bottom) in the AI value chain. Hover a node in 3D, or read the full relationships below.
Graph data (text) — 3 entities, 2 relationships
- Claude (Opus / Sonnet) —uses→ Alignment Methods (RLHF, Constitutional AI, DPO)
- GPT-5 / o-series —uses→ Alignment Methods (RLHF, Constitutional AI, DPO)