National Center for Applied Mathematics and Department of Mathematics, Nanjing University; Calculation Consulting; Department of Computer Science, Dartmouth College; ICSI, LBNL, and Department of Statistics, University of California at Berkeley
Yefan Zhou, TIANYU PANG, Keqin Liu, charles martin, Michael Mahoney, Yaoqing Yang
Heavy-tail self-regularization;learning rate schedule
Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training