National University of Singapore; HPC-AI Technology Inc.
Xuanlei Zhao, Shenggan Cheng, Guangyang LU, Haotian Zhou, Bin Jia, Yang You
Long Sequence Inference;Inference;Compiler;Activation Memory;Machine Learning Infrastructure;Low Resource Inference
AutoChunk: Automated Activation Chunk for Memory-Efficient Deep Learning Inference