1Apple, 3EPFL; 1Apple; 1Apple, 2MIT
Enric Boix-Adsera, Etai Littwin, Emmanuel Abbe, Samy Bengio, Joshua Susskind
transformers;low-rank bias;incremental learning
Transformers learn through gradual rank increase