Google DeepMind; University of Toronto
Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, Olivier Bachem
Language models;Distillation;RLHF
On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes