true input-output relationship with a finite size training data set $\{(X_n,Y_n)\}_{n=1}^{n_1}$, where $n_1$ is the sample size. The training error $E(w)$ of an LNN is given by $E(w) = \frac{1}{n_1} \sum_{n=1}^{n_1} \|Y_n-f(X_n,w)\|^2$, where $\|\cdot\|$ is the Euclidean norm of $\mathbb{R}^N$.

Since the minimization of the training error $E(w)$ leads to overfitting to a training data set, we adopt the L1 regularization method[@Ishikawa1990; @Tibshirani1996] to delete redundant connection weights and obtain a sparse solution. Here, the objective function to be minimized is given by $H(w) = \frac{n_1}{2}\ E(w)+\lambda \sum_{d,i,j} |\omega^d_{ij}|$, where $\lambda$ is a hyperparameter used to determine the strength of regularization. The minimization of such a function $H(w)$ with the stochastic steepest descent method can be executed by an iterative update of the parameters from the output layer to the input layer, which is called error back propagation[@Rumelhart1986; @Werbos1974]. The parameter update is given by $$\begin{aligned}
\Delta \omega^{d-1}_{ij} = -\eta (\delta^d_j o^{d-1}_i+\lambda \ \mathrm{sgn}(\omega^{d-1}_{ij})),\ \ 
\Delta \theta^d_j = -\eta \delta^d_j,\end{aligned}$$ where $\delta^{d_0}_j = (o^{d_0}_j-y_j)\ (o^{d_0}_j\ (1-o^{d_0}_j) +\epsilon_1)$, and $\delta^d_j = \sum_{k=1}^{l_{d+1}} \delta^{d+1}_k \omega^d_{jk}\ (o^d_j\ (1-o^d_j) +\epsilon_1)$ for $d=d_0-1,\cdots, 2$. Here, $y_j$ is the $j$-th output dimension value of a randomly chosen $n$-th sample $(X_n, Y_n)$, $\epsilon_1$ is a hyperparameter for 