Amazon Web Service; Georgia Institute of Technology
Long-Range Transformer;Attention Mechanism;Mixed Attention Span
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer