AWS AI; University of Minnesota, Minneapolis & AWS AI; University of Wisconsin, Madison
Zhanpeng Zeng, Cole Hawkins, Mingyi Hong, Aston Zhang, Nikolaos Pappas, Vikas Singh, Shuai Zheng
efficient;transformer;roberta;T5;language modeling;question answering;summarization
VCC: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens