Condensed Matter Theory Center, University of Maryland, College Park; Institute for Physical Science and Technology, University of Maryland, College Park; Department of Physics, University of Maryland, College Park; Joint Quantum Institute, University of Maryland, College Park
Dayal Singh Kalra, Maissam Barkeshli
Optimization dynamics;Phase diagrams;learning rate transition;Catapult effect
Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width