RIKEN CBS; Columbia University; University of Toronto, Vector Institute; University of Tokyo, RIKEN AIP; Google Research, Brain Team
Shun-ichi Amari, Jimmy Ba, Roger Grosse, Xuechen Li, Atsushi Nitanda, Taiji Suzuki, Denny Wu, Ji Xu
generalization;second-order optimization;natural gradient descent;high-dimensional asymptotics
When does preconditioning help or hurt generalization?