cs.AI cs.DC
Tackling the Dynamicity in a Production LLM Serving System with SOTA Optimizations via Hybrid Prefill/Decode/Verify Scheduling on Efficient Meta-kernels
Mingcong Song Xinru Tang Fengfan Hou Jing Li Wei Wei Yipeng Ma Runqiu Xiao Hongjie Si Dingcheng Jiang Shouyi Yin Yang Hu Guoping Long