Wangxuan Institute of Computer Technology, Peking University; Center for Data Science, AAIS, Peking University; National Key Laboratory of General Artificial Intelligence; BIGAI, Beijing, China; Meta AI; Gaoling School of Artificial Intelligence, Renmin University of China; Meituan; Wangxuan Institute of Computer Technology, Peking University; Center for Data Science, AAIS, Peking University
Model Compression;LLM
Retrieval-based Knowledge Transfer: An Effective Approach for Extreme Large Language Model Compression