Center for Data Science, Peking University; National Engineering Laboratory for Big Data Analysis and Applications, Peking University; Institute for Computing and Digital Economy, Peking University; Baidu Inc.; School of Mathematics Sciences, Peking University
Chang Chen, Min Li, Zhihua Wu, Dianhai Yu, Chao Yang
TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training