=============================

In this section, we first introduce the optimal bit-width problem and then show how a theoretical bound could potentially be used to estimate the optimal bit-width of a neural network.

Because of the natural need and desire of comparison with competitive approaches, most quantization techniques are evaluated on some popular reference networks, without modification of the network topology. On the one hand, the advancement of lossless quantization almost stalls at a bit-width between two and six [@han2015deep; @choi2016towards; @sze2017efficient; @blott2017scaling; @su2018accuracy; @faraone2018syq]. A specific bit-width depends on the compactness of the reference network and the difficulty of the task. On the other hand, the design space, especially the different combinations of topology and bit-width, is largely underexplored because of the complexity, resulting in sub-optimal results. A recent work by [@su2018accuracy] empirically validates the benefit of exploring flexible network topology during quantization. That work adds a simple variable of network expanding ratio, and shows that a bit-width of four achieves the best cost-accuracy trade-off among limited options in $\{1,2,4,8,16,32\}$. Some recent effort on using reinforcement learning to optimize the network hyper-parameters [@he2018amc] could potentially be used to address this issue. But the current design space is still limited to a 