University of Illinois Urbana-Champaign; Kempner Institute, Harvard University; Allen Institute for AI; University of Edinburgh
Tom Sherborne, Naomi Saphra, Pradeep Dasigi, Hao Peng
sharpness-aware minimization;sam;trust region;optimization;cross-lingual transfer;language modeling
TRAM: Bridging Trust Regions and Sharpness Aware Minimization