Department of Computer Science & Princeton Language and Intelligence, Princeton University; Institute for Interdisciplinary Information Sciences, Tsinghua University; Institute for Interdisciplinary Information Sciences, Tsinghua University; Shanghai Qizhi Institute; Shanghai AI Laboratory
Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang
distributed training;Local SGD;local gradient methods;generalization;implicit bias;sharpness
A Quadratic Synchronization Rule for Distributed Deep Learning