Safety, Ethics & Governance · advanced · concept 294 of 311 · Updated Nov 4, 2026

🔭Scalable Oversight

Scalable oversight is the problem of supervising an AI system that may outperform its supervisors on a task, so a person cannot check its answers. It matters because RLHF rests on humans judging outputs. Approaches include debate, where two models argue and a human judges, and weak-to-strong generalization. In a 2022 Anthropic study, people chatting with an unreliable model assistant beat both the model alone and their own unaided scores.