LTCI, Télécom Paris, Institut Polytechnique de Paris; INRIA & ENS – PSL Research University; University of Toronto & Vector Institute; Bo˘gaziçi University
Melih Barsbey, Milad Sefidgaran, Murat Erdogdu, Gaël Richard, Umut Simsekli
heavy tails;stochastic gradient descent;deep learning;compression
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks