In our experiments, we use quantization to K-bit fixed point representations as a simple and computationally-inexpensive approximation strategy to validate our method.
experiments on CIFAR-10, CIFAR-100, and ImageNet show that our method yields performance close to exact training, while storing activations compactly with as low as 4-bit precision.