University of Illinois at Urbana-Champaign; Microsoft
Yu Meng, Chenyan Xiong, Payal Bajaj, saurabh tiwary, Paul N Bennett, Jiawei Han, Xia Song
Language Model Pretraining
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators