Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
Tay, Yi Dehghani, Mostafa Abnar, Samira Chung, Hyung Fedus, William Rao, Jinfeng Narang, Sharan Tran, Vinh Yogatama, Dani Metzler, Donald