Microsoft Research India; University of Washington Seattle
Rahul Kidambi, Praneeth Netrapalli, Prateek Jain, Sham M Kakade
Stochastic Gradient Descent;Deep Learning;Momentum;Acceleration;Heavy Ball;Nesterov Acceleration;Stochastic Optimization;SGD;Accelerated Stochastic Gradient Descent
On the insufficiency of existing momentum schemes for Stochastic Optimization