The University of Hong Kong; ByteDance
Yangrui Chen, Cong Xie, Meng Ma, Juncheng Gu, Yanghua Peng, Haibin Lin, Chuan Wu, Yibo Zhu
data parallelism;communication optimization;staleness mitigation
SAPipe: Staleness-Aware Pipeline for Data Parallel DNN Training