Department of Statistics and Actuarial Science, Department of Applied Mathematics, and Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, Canada; Department of Statistics and Data Science, University of Pennsylvania, Philadelphia, PA, USA; Department of Mathematics, Northwestern University, Evanston, IL, USA; Courant Institute, New York University, New York, NY, USA
Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath
stochastic gradient descent;Hessian;multi-layer neural networks;high-dimensional classification;Gaussian mixture model;XOR problem
High-dimensional SGD aligns with emerging outlier eigenspaces