Salesforce Research, Palo Alto, US; Department of Computer Science, EPFL, Switzerland
Akhilesh Deepak Gotmare, Nitish Shirish Keskar, Caiming Xiong, richard socher
deep learning heuristics;learning rate restarts;learning rate warmup;knowledge distillation;mode connectivity;SVCCA
A Closer Look at Deep Learning Heuristics: Learning rate restarts, Warmup and Distillation