National Engineering Laboratory for Speech and Language Information Processing, University of Science and Technology of China; Speech Lab, Alibaba Group
Chao-Hong Tan, Qian Chen, Wen Wang, Qinglin Zhang, Siqi Zheng, Zhen-Hua Ling
Transformer;Efficient Transformers;Token Mixing;Pooling;Linear;Long Range Arena;Transfer Learning;BERT;GLUE
PoNet: Pooling Network for Efficient Token Mixing in Long Sequences