Facebook AI Research, Montreal, Canada; Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA
distributed optimization;decentralized training methods;communication-efficient distributed training with momentum;large-scale parallel SGD
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum