the LNN convergence, and $\eta$ is the step size for training time $t$ that is determined such that $\eta(t)\propto 1/t$. In the experiments, we adopt $\epsilon_1=0.001$ and $\eta =0.7\times a_1 n_1/(a_1 n_1 +5t)$, where $a_1$ is the mean iteration number for LNN training per dataset.

Hierarchical Modular Representation of LNNs
===========================================

Determining Feature Vectors of Hidden Layer Units {#sec:features}
-------------------------------------------------

To apply hierarchical clustering to a trained LNN, we define a feature vector for each hidden layer unit. Let $v_k$ be the feature vector of the $k$-th hidden layer unit in a hidden layer. Such a feature vector should reflect the role of its corresponding unit in LNN inference. Here, we propose defining such a feature vector $v_k$ of the $k$-th hidden layer unit based on its correlations between each input or output dimension. In previous studies[@Watanabe2018arxiv; @Watanabe2018arxiv2], methods have been proposed for determining the role of a unit or a unit cluster based on the square root error. However, these methods can only provide us with knowledge about the magnitude of the effect of each input dimension on a unit and the effect of a unit on each output dimension, not information about how a hidden layer unit is 