Meta AI; LTI, CMU; ISI, USC; SJTU
Xuezhe Ma, Chunting Zhou, Xiang Kong, Junxian He, Liangke Gui, Graham Neubig, Jonathan May, Luke Zettlemoyer
Neural Architecture;Attention;Exponential Moving Average
Mega: Moving Average Equipped Gated Attention