Carnegie Mellon University; Microsoft Corporation
Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen
Transfer learning;Adaptation;Transformer;Fine-tuning;Low-rank;RoBERTa;DeBERTa;GPT-2;GPT-3
LoRA: Low-Rank Adaptation of Large Language Models