ISTAustria; Google DeepMind
Elias Frantar, Carlos Riquelme Ruiz, Neil Houlsby, Dan Alistarh, Utku Evci
sparsity;scaling;optimal sparsity;efficiency;foundational models;transformers;structured sparsity;pruning
Scaling Laws for Sparsely-Connected Foundation Models