Affiliation
optimization;adam;adamw;adaptive optimizers;grokking;training instability;instability;empirical;empirical science of deep learning;double descent
The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the \emph{Grokking Phenomenon}