cs.DC cs.AI cs.LG I.2.11; C.4
LayerKV: Optimizing Large Language Model Serving with Layer-wise KV Cache Management
Yi Xiong Hao Wu Changxu Shao Ziqing Wang Rui Zhang Yuhong Guo Junping Zhao Ke Zhang Zhenxuan Pan