# On-Policy Distillation Projects at AI Tinkerers

> Canonical HTML: https://aitinkerers.org/technologies/on-policy-distillation
> Markdown URL: https://aitinkerers.org/technologies/on-policy-distillation.md
> Technology record last updated: 2026-06-08T09:43:58Z
> Generated: 2026-09-22T07:46:42Z

On-Policy Distillation (OPD) is a post-training technique that aligns a student model by generating its own rollouts and receiving per-token feedback from a teacher model to eliminate exposure bias.

Standard knowledge distillation suffers from exposure bias because the student model is trained only on the teacher's distribution, causing errors to compound rapidly during inference. On-Policy Distillation (OPD) solves this by reorganizing the training loop: the student generates its own trajectories, and the teacher evaluates and corrects the student's actual outputs at the token level (often targeting a 97% to 99% probability mass alignment on student-visited states). This approach provides a dense, token-level learning signal that functions as an alternative to noisy sequence-level reinforcement learning rewards, making it a key optimization method for modern architectures like Qwen and DeepSeek.

- Official technology site: https://github.com/thinkwee/AwesomeOPD
- Public AI Tinkerers demos and talks: 0
- Result page: 1 of 1

## Recent Public Talks and Demos

No public projects are currently indexed for this technology.