DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models
Dai, Damai Deng, Chengqi Zhao, Chenggang Xu, R.x. Gao, Huazuo Chen, Deli Li, Jiashi Zeng, Wangding Yu, Xingkai Wu, Y. Xie, Zhenda Li, Y.k. Huang, Panpan Luo, Fuli Ruan, Chong Sui, Zhifang Liang, Wenfeng