Zhenyu Zhang Ying Sheng Tianyi Zhou Tianlong Chen Lianmin Zheng Ruisi Cai Zhao Song Yuandong Tian Christopher Re Clark Barrett Zhangyang Wang Beidi Chen
H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models