Distributed and Parallel Software Lab, Huawei Technologies
Yury Nahshan, Joseph Kampeas, Emir Haleva
Neural Networks;Transformers;Self-Attention;Linear Attention;Scalable Transformers;Efficient Attention;Attention with Linear Complexity;Linearized Attention;Self-Attention Analysis
Linear Log-Normal Attention with Unbiased Concentration