Technology
On-Policy Distillation
On-Policy Distillation (OPD) is a post-training technique that aligns a student model by generating its own rollouts and receiving per-token feedback from a teacher model to eliminate exposure bias.
Standard knowledge distillation suffers from exposure bias because the student model is trained only on the teacher's distribution, causing errors to compound rapidly during inference. On-Policy Distillation (OPD) solves this by reorganizing the training loop: the student generates its own trajectories, and the teacher evaluates and corrects the student's actual outputs at the token level (often targeting a 97% to 99% probability mass alignment on student-visited states). This approach provides a dense, token-level learning signal that functions as an alternative to noisy sequence-level reinforcement learning rewards, making it a key optimization method for modern architectures like Qwen and DeepSeek.
Recent Talks & Demos
Showing 1-0 of 0
