King Abdullah University of Science and Technology, Thuwal, Kingdom of Saudi Arabia; Meituan, Beijing, China
Hang Xu, Wenxuan Zhang, Jiawei Fei, Yuzhe Wu, TingWen Xie, Jun Huang, Yuchen Xie, Mohamed Elhoseiny, Panos Kalnis
SLAMB: Accelerated Large Batch Training with Sparse Communication