Boston University; George Mason University; IBM T. J. Watson Research Center; Meta Platforms, Inc.
Michael Crawshaw, Mingrui Liu, Francesco Orabona, Wei Zhang, Zhenxun Zhuang
stochastic;optimization;noncovex;adam;transformer;clipping;signsgd;momentum;unbounded smoothness
Robustness to Unbounded Smoothness of Generalized SignSGD