New York University, Facebook AI Research, CIFAR Azrieli Global Scholar; Department of Mathematical Sciences, KAIST, Daejeon, 34141, Republic of Korea
regularization;finetuning;dropout;dropconnect;adaptive L2-penalty;BERT;pretrained language model
Mixout: Effective Regularization to Finetune Large-scale Pretrained Language Models