Microsoft Dynamics 365 AI; University of Illinois, Urbana-Champaign; Microsoft Research; Georgia Tech
warmup;adam;adaptive learning rate;variance
On the Variance of the Adaptive Learning Rate and Beyond