Yandex, Russia; HSE University, Russia; University of Toronto, Canada; Vector Institute, Canada; MIPT, Russia; HSE University, Russia; Yandex, Russia
Max Ryabinin, Eduard Gorbunov, Vsevolod Plokhotnyuk, Gennady Pekhimenko
decentralized training;distributed training;optimization
Moshpit SGD: Communication-Efficient Decentralized Training on Heterogeneous Unreliable Devices