Wang, Boxin Ping, Wei Xu, Peng McAfee, Lawrence Liu, Zihan Shoeybi, Mohammad Dong, Yi Kuchaiev, Oleksii Li, Bo Xiao, Chaowei An kumar, Anima Catanzaro, Bryan
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study