Center for Research on Intelligent Perception and Computing, CASIA, Beijing, China; National Laboratory of Pattern Recognition, CASIA, Beijing, China; University of Science and Technology of China, Hefei, China; Center for Research on Intelligent Perception and Computing, CASIA, Beijing, China; National Laboratory of Pattern Recognition, CASIA, Beijing, China; School of Artiﬁcial Intelligence, University of Chinese Academy of Sciences, Beijing, China; Center for Research on Intelligent Perception and Computing, CASIA, Beijing, China; National Laboratory of Pattern Recognition, CASIA, Beijing, China
Huaibo Huang, Xiaoqiang Zhou, Ran He
Vision Transformer;Efficient Self-attention;Orthogonality
Orthogonal Transformer: An Efficient Vision Transformer Backbone with Token Orthogonalization