matrix ${\mbox{\boldmath $W$}}_k \in {\mbox{\boldmath $R$}}^{d_k \times d_{k-1} }$ that is used in the $p$-th path to the output neuron $j$.

The objective of the training is to find the parameters which minimize the error function defined as $$\label{eqn2-2}
L({\mbox{\boldmath $\Theta$}}) = \frac{1}{2} \sum_{i=1}^m 
E_{{\mbox{\boldmath $Z$}}} \Vert \hat{{\mbox{\boldmath $Y$}}}_i({\mbox{\boldmath $\Theta$}}, {\mbox{\boldmath $X$}}_i) - {\mbox{\boldmath $Y$}}_i \Vert^2$$ where $\Vert \cdot \Vert$ is the Euclidean norm, that is, $\Vert {\mbox{\boldmath $u$}}\Vert = \sqrt{u_1^2 + \cdots + u_N^2}$ for a vector ${\mbox{\boldmath $u$}}= (u_1 \cdots u_N)^T \in {\mbox{\boldmath $R$}}^N$, and $\hat{{\mbox{\boldmath $Y$}}}_i({\mbox{\boldmath $\Theta$}}, {\mbox{\boldmath $X$}}_i) \in {\mbox{\boldmath $R$}}^{d_y}$ is the actual output of the output layer of the deep nonlinear neural network for the $i$-th training pattern ${\mbox{\boldmath $X$}}_i$. The expectation in Eq. (\[eqn2-2\]) is made with respect to random vector ${\mbox{\boldmath $Z$}}= \{ [{\mbox{\boldmath $Z$}}_i]_{(j,p)} \}$.

The Kawaguchi model has been analyzed based on the two assumptions.

[**A1p-m**]{} [ *$P([{\mbox{\boldmath $Z$}}_i]_{(j,p)} = 1) = \rho$ for all $i$ and $(j,p)$ where $\rho \in {\mbox{\boldmath $R$}}$ is a constant. That is, $[{\mbox{\boldmath $Z$}}_i]_{(j,p)}$ is a Bernoulli random variable.*]{}

[**A5u-m**]{} [ *${\mbox{\boldmath $Z$}}$ is independent of the input ${\mbox{\boldmath $X$}}$ and the parameter ${\mbox{\boldmath $\Theta$}}$.*]{}

A1p-m and A5u-m are weaker ones of the two 