Meta Platforms, Inc., Bellevue, WA, 98004, USA; Department of Computer Science, George Mason University, Fairfax, VA 22030, USA; School of Computer Science, University of Birmingham, United Kingdom; Department of Mathematics, Texas A&M University, College Station, Texas 77840, USA
Mingrui Liu, Zhenxun Zhuang, Yunwen Lei, Chunyang Liao
Distributed Training;Gradient Clipping;Communication-Efficient;Optimization
A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks