single variable per layer (such as the pruning ratio based on a reference network). How to estimate an optimal bit-width for a target task without training could be an interesting research direction in the future.

The memory bound expression as derived in this paper helps us to determine whether there is an optimal $\lambda$ that would lead to the lowest bound and most compact network (which can be translated to computation cost in a fully connected structure) for a given target function. For example, by dropping the lower-order term and ignoring the rounding operator, our memory bound can be simplified as $$M(\lambda)=\theta_1 \lambda
\log(\lambda)\log^{\frac{1}{\lambda-1}+1}(3n2^{d}/\epsilon)$$ where $\theta_1$ is a constant determined by $\epsilon,n,$ and $d$. We can find an optimal $\lambda$ that minimizes $M(\lambda)$: $$\lambda_{opt}=\operatorname*{argmin}_\lambda M(\lambda)$$ As is detailed in Appendix \[proofBitwidth\], we prove that there exists one and only one local minimum (hence global minimum) in the range of $[2,\infty)$ whenever $\epsilon<\frac{1}{2}$. We also show that $\lambda_{opt}$ is determined by $\log( 3n2^{d}/\epsilon)$, which can be easily dominated by $d$. Based on such results, we quantitatively evaluate the derivative of $M(\lambda)$, and based on which the optimal bit-width $\log(\lambda_{opt})$ under various settings in Figure \[fig:derivative\] and Figure \[fig:bitwidth\], respectively. In Figure 