Architectures · advanced · concept 96 of 311 · Updated Nov 4, 2026

🏗️Knowledge Distillation

Training a smaller 'student' model to mimic a larger 'teacher' model's behavior. The student learns from the teacher's soft outputs (probabilities), capturing knowledge that's lost with hard labels alone.