hidden neuron $j_1$ are [*i.i.d.*]{}, and the probability density function $f_1: {\mbox{\boldmath $J$}}_1 \rightarrow [ 0, +\infty )$ of the probability distribution which each weight $w_{j_1k}$ obeys is an even function ($k = 1, \cdots, d_x$). Consequently, $P\left({\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath $w$}}_{j_1} > 0 \right) = P\left({\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath $w$}}_{j_1} \leq 0 \right)$, which means $P\left({\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath $w$}}_{j_1} > 0 \right) = 1/2$.

\[For $H$\] This case corresponding to a deep nonlinear neural network with $H$ hidden layers, we show that if the case of $H-1$ holds, then case $H$ also holds. Assuming that the case of $H-1$ holds, then $$\begin{aligned}
\label{eqn4-4}
& &P\Big( [Z_i]_{(j,p)} = 1 \Big) \nonumber \\
&=& P \left( {\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath $w$}}_{j_1} > 0, \left[ \varphi( U_1^{(1)} ) \cdots \varphi(U_{d_1}^{(1)} ) \right] 
{\mbox{\boldmath $w$}}_{j_2} > 0, \right. \nonumber\\ 
& & \left. \cdots, \left[ \varphi( U_1^{(H-1)} ) \cdots \varphi(U_{d_{H-1}}^{(H-1)}) \right] 
{\mbox{\boldmath $w$}}_{j_H} > 0 \right) \ \ 
(\mbox{from Eq.} (\ref{eqn4-2})) \nonumber\\
&=& P \left( {\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath $w$}}_{j_1} > 0, \left[ \varphi( U_1^{(1)} ) \cdots \varphi(U_{d_1}^{(1)} ) \right] 
{\mbox{\boldmath $w$}}_{j_2} > 0, \right. \nonumber\\ 
& & \left. \cdots, \left[ \varphi( U_1^{(H-1)} ) \cdots \varphi(U_{d_{H-1}}^{(H-1)}) \right] 
{\mbox{\boldmath $w$}}_{j_H} > 0 \ \ \Big\vert \ {\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath 