Google DeepMind, Mila; Google Research, University of Toronto; Google Research, Princeton University; Google Research
Yongchao Zhou, Kaifeng Lyu, Ankit Singh Rawat, Aditya Krishna Menon, Afshin Rostamizadeh, Sanjiv Kumar, Jean-François Kagy, Rishabh Agarwal
large language model;knowledge distillation;speculative decoding
DistillSpec: Improving Speculative Decoding via Knowledge Distillation