Department of Electrical and Computer Engineering, Rutgers University; Texas Advanced Computing Server, University of Texas at Austin; Department of Computer Science, University of Toronto
Mohammad Mozaffari, Sikan Li, Zhao Zhang, Maryam Mehri Dehnavi
machine learning;deep learning;optimizers;distributed training;second-order optimization;
MKOR: Momentum-Enabled Kronecker-Factor-Based Optimizer Using Rank-1 Updates