Paper under double-blind review
Learning dynamics of deep neural networks;weight decay;normalization;effective learning rate;AdamW;Adam with L2 regularization;spherical motion dynamics;optimization;training;scale invariance;equilibrium;rotational;SGDM;Lion
Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks