Department of EECS, UC Berkeley; Computer Science Department, Stanford University
Banghua Zhu, Ying Sheng, Lianmin Zheng, Clark Barrett, Michael Jordan, Jiantao Jiao
caching;model selection;large language models;foundation models;inference;bandit;regret
Towards Optimal Caching and Model Selection for Large Model Inference