Carnegie Mellon University
Elan Rosenfeld, Andrej Risteski
neural network optimization;progressive sharpening;edge of stability;adaptive gradient methods;batch normalization
Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization