Preferred Networks
Akifumi Imanishi, Zijian Xu, Masayuki Takagi, Sixue Wang, Emilio Castillo
Recomputation;Gradient checkpointing;Memory reduction;Computational graph optimization
A fast heuristic to optimize time-space tradeoff for large models