University of Illinois at Urbana-Champaign; Microsoft
Yu Meng, Chenyan Xiong, Payal Bajaj, saurabh tiwary, Paul Bennett, Jiawei Han, XIA SONG
Language Model Pretraining
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining