Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Wang, Boxin Ping, Wei Xu, Peng McAfee, Lawrence Liu, Zihan Shoeybi, Mohammad Dong, Yi Kuchaiev, Oleksii Li, Bo Xiao, Chaowei An kumar, Anima Catanzaro, Bryan