Google Research, New York City, USA
Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank Reddi, Ke Ye, Felix Chern, Felix Yu, Ruiqi Guo, Sanjiv Kumar
Transformers;Sparse;Calibration;Robustness;Label Noise;Efficiency
The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers