Stanford University; NVIDIA
Deepak Narayanan, Keshav Santhanam, Peter Henderson, Peter Henderson, Rishi Bommasani, Tony Lee, Percy Liang
Systems for Machine Learning;Inference efficiency;Transformer models;Text generation APIs;Capability-efficiency tradeoffs
Cheaply Estimating Inference Efficiency Metrics for Autoregressive Transformer Models