\[fig:bitwidth\], we also mark the input dimension of a few image data sets. It is apparent to see that the optimal bit width derived from $M(\lambda)$ is dominated by $d$ and lies between one and four for a wide range of input size. This observation is consistent with most existing empirical research results, hence showing the potential power of our theoretical bound derivation.

[0.5]{} ![Quantitative evaluation of the derivative of $M(\lambda)$ and the optimal bit-width $\log(\lambda_{opt})$. The derivative is scaled by $\log^{\frac{-\lambda}{\lambda-1}}(3n2^d)$ to fit in the same range. $\log^{\frac{-\lambda}{\lambda-1}}(3n2^d)$ is a positive monotonically increasing function and thus does not affect the trends too much. Note that $\lambda$ is the number of distinct weight values and thus $\log(\lambda)$ is the corresponding bit-width. It can be seen that $\epsilon$ and $n$ only affect $\log(\lambda_{opt})$ when $d$ is small ($<10^2$). We also mark the input dimension $d$ of various image data set and their corresponding $\log(\lambda_{opt})$. It shows that the optimal bit-width increases very slowly with $d$.](derivative.pdf "fig:"){width="\linewidth"}

[0.5]{} ![Quantitative evaluation of the derivative of $M(\lambda)$ and the optimal bit-width $\log(\lambda_{opt})$. The derivative is scaled by $\log^{\frac{-\lambda}{\lambda-1}}(3n2^d)$ to fit in the same range. $\log^{\frac{-\lambda}{\lambda-1}}(3n2^d)$ is a positive monotonically increasing function and thus does 