a constant; (5) the number of weights is $\mathcal{O}\left(r\right)$.

Build three sub-networks $f_s^r$ as described in Proposition \[p1\] and let $$\label{constructX}
\times'(x,y)=2\left(f_{s}^r\left(|x+y|/2\right)-f_{s}^r\left(|x|/2\right)-f_{s}^r\left(|y|/2\right)\right).$$ Then the statement (1) is followed by property (1) of Proposition \[p1\]. Using the error bound in Proposition \[p1\] and Equation (\[constructX\]), we get the error bound of $\times'$: $$\label{Xerror}
\epsilon_{\times'}\leq6\cdot2^{-2(r+1)}.$$ Since a sub-network $B_{abs}$ that computes $\sigma(x)+\sigma(-x)$ can be constructed to get the absolute value of $x$ trivially, we can construct $\times'(x,y)$ as a linear combination of three parallel $f_s^r$ and feed them with $\frac{|x|}{2},\frac{|y|}{2}$, and $\frac{|x+y|}{2}$. Then claims of statement (3), (4), and (5) are also obtained.

Approximation of weights  {#weights}
-------------------------

\[w\] Denote the design parameter that determines the approximation error bound as $t$. A connection with any weight $w\in[-1,1]$ can be approximated by a ReLU sub-network that has only $\lambda \geq 2$ distinct weights, such that (1) the sub-network is equivalent to a connection with weight $w'$ while the approximation error is bounded by $2^{-t}$ i.e., $|w'-w|<2^{-t}$; (2) the depth is $\mathcal{O}\left(\lambda t^{\frac{1}{\lambda-1}}\right)$; (3) the width is $\mathcal{O}(t)$; (4) the number of weights is $\mathcal{O}\left(\lambda t^{\frac{1}{\lambda-1}+1}\right)$.

Consider that we need a weight $w$ to feed the input $x$ to a unit in the next 