reported that a pruned AlexNet can be quantized to eight bits and five bits in convolutional layers and fully connected layers, respectively, without any loss of accuracy. Similar results are also observed in LENET-300-100, LENET-5, and VGG-16 [@han2015deep]. One may argue that some of these benchmark networks are known to have redundancy. However, recent works show that quantization works well even on networks that are designed to be extremely small and compact. SqueezeNet, which is a state-of-the-art compact network, can be quantized to 8-bit while preserving the original accuracy [@gysel2016hardware; @iandola2016squeezenet]. There are some representative works that can achieve little accuracy loss on ImageNet classification even using binary or ternary weights [@courbariaux2015binaryconnect; @rastegari2016xnor; @li2016ternary; @zhu2016trained]. More aggressively, some works also reduce the precision of activations, e.g. [@hubara2016quantized; @rastegari2016xnor; @faraone2018syq]. Although the classification accuracy loss can be minimized, the universal approximation property is apparently lost, as with limited output precision the network cannot achieve arbitrary accuracy. Accordingly, we do not include them in the discussion of this paper. The limit of quantization is still unknown while the state-of-the-art keeps getting updated. For example, VGG-16 is quantized to 3-bit while maintaining the original accuracy [@leng2017extremely]. Motivated by the great empirical success, the 