Graduate School of AI, Korea Advanced Institute of Science and Technology; School of Computing, Korea Advanced Institute of Science and Technology; Graduate School of AI, Korea Advanced Institute of Science and Technology, DeepAuto.ai
Heejun Lee, Jina Kim, Jeff Willette, Sung Ju Hwang
Sparse Attention;Attention Estimation;Linear Attention;Transformers;NLP
SEA: Sparse Linear Attention with Estimated Attention Mask