School of Computer Science, Peking University; National Key Laboratory for Multimedia Information Processing, Peking University; Microsoft Corporation
Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, Sujian Li
context window extension;efficiency;positional skip-wise training
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training