Beijing Academy of Artificial Intelligence, Beijing, China; Harbin Institute of Technology, Shenzhen, China
Yiqun Yao, Zheng Zhang, Jing Li, Yequan Wang
Progressive Learning;Large Language Models;Model Growth
Masked Structural Growth for 2x Faster Language Model Pre-training