and Proposition \[w\], we are able to prove the upper bound for general functions.

\[T1\] For any $f\in \mathcal{F}_{d,n}$ , given $\lambda$ distinct weights, there is a ReLU network with fixed structure that can approximate $f$ with any error $\epsilon\in(0,1)$, such that (1) the depth is $\mathcal{O}\left(\lambda\log^{\frac{1}{\lambda-1}}\left(1/\epsilon\right)+\log\left(1/\epsilon\right)\right)$; (2) the number of weights is $\mathcal{O}\left(\lambda \log^{\frac{1}{\lambda-1}+1}\left(1/\epsilon\right)\left(1/\epsilon\right)^{\frac{d}{n}}\right)$; (3) the number of bits needed to store the network is $\mathcal{O}\left(\lambda\log\left(\lambda\right) \log^{\frac{1}{\lambda-1}+1}\left(1/\epsilon\right)\left(1/\epsilon\right)^{\frac{d}{n}}\right)$.

The complete proof and the network constructed can be found in Appendix \[PT1\]. We first approximate $f$ by $f_2$ using the Taylor polynomial of order $n-1$ and prove the approximation error bound. Note that even when $f$ is non-differentiable (only first order weak derivative exists), the Taylor polynomial of order 0 at $\textbf{x}=\frac{\textbf{m}}{N}$ can still be used, which takes the form of $P_{\textbf{m}}=f(\frac{\textbf{m}}{N})$. Then we approximate $f_2$ by a ReLU network that is denoted as $f'$ with bounded error. After that, we present the quantized ReLU network that implements $f'$ and the complexity.

The discussion above focuses on nonlinear quantization which is a more general case compared to linear quantization. For linear quantization, which strictly determines the available weight values once $\lambda$ is given, we can use the same proof for nonlinear 