University of Washington; Meta AI (FAIR), Carnegie Mellon University; University of Washington, Zhejiang University; Meta AI (FAIR)
Yuandong Tian, Yiping Wang, Beidi Chen, Simon Du
transformer;training dynamics;theoretical analysis;self-attention;interpretability;neural network understanding
Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer