Carnegie Mellon University; Microsoft Research
Mikhail Khodak, Neil Tenenholtz, Lester Mackey, Nicolo Fusi
model compression;knowledge distillation;multi-head attention;matrix factorization
Initialization and Regularization of Factorized Neural Layers