Princeton University; Carnegie Mellon University
Ta-Chung Chi, Ting-Han Fan, Peter J Ramadge, Alexander Rudnicky
Transformer Language Modeling;Length Extrapolation;Kernel Method
KERPLE: Kernelized Relative Positional Embedding for Length Extrapolation