Department of Computer Science, University of California, Los Angeles, Los Angeles, CA 90095, USA; Machine Learning Department, Carnegie Mellon University, Pittsburgh, PA 15213, USA
Zixiang Chen, Yihe Deng, Yue Wu, Quanquan Gu, Yuanzhi Li
Towards Understanding the Mixture-of-Experts Layer in Deep Learning