Department of Mathematical and Statistical Sciences, University of Alberta; Department of Computing Science, University of Alberta
Kirby Banman, Garnet Liam Peet-Pare, Nidhi Hegde, Alona Fyshe, Martha White
optimization;momentum;stochastic gradient descent;non-iid sampling
Resonance in Weight Space: Covariate Shift Can Drive Divergence of SGD with Momentum