MoE-I^2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition
Yang, Cheng Sui, Yang Xiao, Jinqi Huang, Lingyi Gong, Yu Duan, Yuanlin Jia, Wenqi Yin, Miao Cheng, Yu Yuan, Bo