Google Research
Vaishnavh Nagarajan, Aditya Menon, Srinadh Bhojanapalli, Hossein Mobahi, Sanjiv Kumar
knowledge distillation;regularization;understanding;underfitting;theory
On student-teacher deviations in distillation: does it pay to disobey?