National Key Laboratory for Multimedia Information Processing, Peking University; School of Software & Microelectronics, Peking University; National Key Laboratory for Multimedia Information Processing, Peking University; School of Computer Science, Peking University; Alibaba Group; Tencent Cloud AI
pretrained language model;adaptive sub-net Optimization
Bi-Drop: Enhancing Fine-tuning Generalization via Synchronous sub-net Estimation and Optimization