Microsoft Corporation (Internship at Microsoft Research); Microsoft Corporation
Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, Furu Wei
large language model;context compression;in-context autoencoder;pretraining;fine-tuning;Llama;GPT;memorization
In-context Autoencoder for Context Compression in a Large Language Model