Google Research & DeepMind; Google Research
Yi Tay, Mostafa Dehghani, Jinfeng Rao, William Fedus, Samira Abnar, Hyung Won Chung, SHARAN NARANG, Dani Yogatama, Ashish Vaswani, Donald Metzler
transformers;attention;deep learning
Scale Efficiently: Insights from Pretraining and Finetuning Transformers