Pattern Recognition Center, WeChat AI, Tencent Inc, China; Institute of Information Engineering, Chinese Academy of Sciences
Yuanxin Liu, Fandong Meng, Zheng Lin, Jiangnan Li, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou
BERT compression;sparse subnetwork;dataset bias;OOD generalization
A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models