Mila, McGill University; Facebook CIFAR AI Chair; ServiceNow Research; IBM Research; Mila, McGill University
Amirhossein Kazemnejad, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Payel Das, Siva Reddy
Transformers;Positional Encoding;Length Generalization
The Impact of Positional Encoding on Length Generalization in Transformers