LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin
Dou, Shihan Zhou, Enyu Liu, Yan Gao, Songyang Shen, Wei Xiong, Limao Zhou, Yuhao Wang, Xiao Xi, Zhiheng Fan, Xiaoran Pu, Shiliang Zhu, Jiang Zheng, Rui Gui, Tao Zhang, Qi Huang, Xuanjing