University of Texas at Austin; AI@Meta (FAIR); University of Washington; Carnegie Mellon University, AI@Meta (FAIR)
Yuandong Tian, Yiping Wang, Zhenyu Zhang, Beidi Chen, Simon Du
multilayer transformer;training dynamics;theoretical analysis;self-attention;interpretability;neural network understanding
JoMA: Demystifying Multilayer Transformers via Joint Dynamics of MLP and Attention