Google DeepMind; Carnegie Mellon University; Google Research
Shanda Li, Chong You, Guru Guruganesh, Joshua Ainslie, Santiago Ontanon, Manzil Zaheer, Sumit Sanghai, Yiming Yang, Sanjiv Kumar, Srinadh Bhojanapalli
Transformers;positional encoding;long context;length generalization
Functional Interpolation for Relative Positions improves Long Context Transformers