Optimization for Neural Networks;Optimization for Representation Learning;Stochastic Optimization;Nonconvex;Quasi-Newton;Optimization for Deep Learning
Apollo: An Adaptive Parameter-wised Diagonal Quasi-Newton Method for Nonconvex Stochastic Optimization