Google Inc., Los Angeles, CA, USA; Decision Intelligence Lab, Alibaba US, Bellevue, WA, USA; Center for Machine Learning Research, Peking University, Beijing, P. R. China; AI for Science Institute, Beijing, P. R. China; National Engineering Labratory for Big Data Analytics and Applications, Beijing, P. R. China; Department of Mathematics, University of California, Los Angeles, CA, USA; Department of Mathematics, Princeton University, Princeton, NJ, USA
Lisang Ding, Kexin Jin, Bicheng Ying, Kun Yuan, Wotao Yin
DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm