Department of Computer Science, The University of Tokyo
Tokio Kajitsuka, Issei Sato
Transformer;Self-Attention;Memorization;Universal Approximation Theorem;Contextual Mapping
Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?