Huawei Noah’s Ark Lab; University of Surrey; Fudan University
Jiachen Lu, Jinghan Yao, Junge Zhang, Xiatian Zhu, Hang Xu, Weiguo Gao, Chunjing XU, Tao Xiang, Li Zhang
Transformer
SOFT: Softmax-free Transformer with Linear Complexity