Institute of Automation, Chinese Academy of Sciences, Beijing, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China; Wuhan AI Research, Wuhan, China; Institute of Automation, Chinese Academy of Sciences, Beijing, China; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
multilinguality;mutlilingual pre-training
Take a Closer Look at Multilinguality! Improve Multilingual Pre-Training Using Monolingual Corpora Only