the expressions of two bounds while treating $\lambda$ as a constant, we can show that, to attain the same approximation error bound $\epsilon$, the number of weights needed by a quantized ReLU network is no more than $\mathcal{O}(\log^5(1/\epsilon))$ times that needed by an unquantized ReLU network. Note that this factor is of much lower order than the lower bound $\Omega(\log^{-3}(1/\epsilon)\left(1/\epsilon\right)^{d/n})$. This little overhead introduced by weight quantization explains in part the empirical success on network compression and acceleration by quantization and also answers in part the questions as raised in Section \[sec:intro\]. Given the significant benefits of quantization in term of memory and computation efficiency, we anticipate that the use of quantization networks will continue to grow, especially on resource-limited platforms.

**Future Work:** There remain many other avenues for future investigation. For example, although we derived the first upper bound of quantized neural networks, the lower bound is still missing. If a tight lower bound of the network size is established, it could be combined with the upper bound to give a much better estimation of required resources and the optimal bit-width. We believe the trends associated with the bounds can also be useful and deserve some further investigation. For 