University of Warsaw; OpenAI; Google Research
Sebastian Jaszczur, Aakanksha Chowdhery, Afroz Mohiuddin, LUKASZ KAISER, Wojciech Gajewski, Henryk Michalewski, Jonni Kanerva
machine learning;transformers;sparsity;scaling
Sparse is Enough in Scaling Transformers