cs.CL cs.AI
AquilaMoE: Efficient Training for MoE Models with Scale-Up and Scale-Out Strategies
Bo-Wen Zhang Liangdong Wang Ye Yuan Jijie Li Shuhao Gu Mengdi Zhao Xinya Wu Guang Liu Chengwei Wu Hanyu Zhao Li Du Yiming Ju Quanyue Ma Yulong Ao Yingli Zhao Songhe Zhu Zhou Cao Dong Liang Yonghua Lin Ming Zhang Shunfei Wang Yanxin Zhou Min Ye Xuekai Chen Xinyang Yu Xiangjun Huang Jian Yang