Core ML Concepts · advanced · concept 49 of 311 · Updated Nov 4, 2026

🎯Reward Models and Process Reward Models

A reward model is a network trained on preference labels to score a response; it stands in for the human during RL training. InstructGPT trained one on pairwise comparisons and optimized the policy against it. An outcome reward model judges only the final answer; a process reward model scores every step, so early mistakes show. OpenAI's Let's Verify Step by Step trained a PRM on 800,000 step labels and solved 78% of a MATH subset.