LIONS, EPFL; University of Wisconsin-Madison; University of Warwick; LIONS, EPFL
Yongtao Wu, Fanghui Liu, Grigorios Chrysos, Volkan Cevher
Transformer;convergence;scaling;initialization;over-parameterization
On the Convergence of Encoder-only Shallow Transformers