Peking University; Salesforce AI Research; UC Berkeley
Hengyu Fu, Tianyu Guo, Yu Bai, Song Mei
transformers;attention;deep learning theory;random features
What can a Single Attention Layer Learn? A Study Through the Random Features Lens