(w_1 \cdots w_{d_0})$ for the sake of simplicity, $\vert \varphi(U_s^{(1)}) \vert \leq \vert U_s^{(1)} \vert 
=\vert {\mbox{\boldmath $X$}}_i^T {\mbox{\boldmath $w$}}_s \vert 
\leq \sum_{t=1}^{d_0} \vert x_t \vert \vert w_t \vert 
\leq \alpha \sum_{t=1}^{d_0} (1/d_0) = \alpha$. Hence, according to the assumption of mathematical induction, the first term of the right-hand-side of Eq. (\[eqn4-4\]) is equal to $(1/2)^{H-1}$. In addition, the second term of the right-hand-side of Eq. (\[eqn4-4\]) is equal to 1/2 from Eq.(\[eqn4-3\]). Therefore, $$\begin{aligned}
\label{eqn4-5}
P\Big( [Z_i]_{(j,p)} = 1 \Big) &=& \left(\frac{1}{2}\right)^{H-1} \cdot \frac{1}{2} \nonumber\\
&=& \frac{1}{2^H}, \end{aligned}$$ which means that the case of $H$ indeed holds. Therefore, by mathematical induction, Eq. (\[eqn4-1\]) holds for any $H \geq 1$. Theorem \[thm1\] states that assumption A1p-m holds. Because $$\begin{aligned}
\lim_{H \rightarrow +\infty} P \left( [Z_i]_{(j,p)} = 1 \right) 
&=& \lim_{H \rightarrow +\infty} \frac{1}{2^H} \nonumber\\
&=& 0, \end{aligned}$$ the probability that path $p$ is active decreases exponentially. It converges to zero as the number of hidden layers $H$ increases. The probability that path $p$ is active decreases by half when a hidden layer is added.

\[thm2\] For any training pattern $i$, any output neuron $j$, and any path $p$ from an input neuron to the output neuron $j$ in the Kawaguchi 