Graying the black box: Understanding [DQN]{}s. In [*Proceedings of the 33rd International Conference on Machine Learning*]{}, pages 1899–1908, 2016.

Appendix 1: Comparison with Clustering Method Based on Non-negative Matrix Factorization {#appendix-1-comparison-with-clustering-method-based-on-non-negative-matrix-factorization .unnumbered}
========================================================================================

Here, we show the effectiveness of our proposed method by comparing it with the clustering method based on non-negative matrix factorization (or NNMF), which was proposed in a previous study[@Watanabe2018arxiv2]. We applied this NNMF-based clustering method to the same data sets that we used in the experiments described in section \[sec:experiments\]. In the previous study[@Watanabe2018arxiv2], the feature vectors of the hidden layer units are defined by the magnitude of the effect of each input dimension value on a cluster and the effect of a cluster on each output dimension value, computed by the square root error of the unit output values. By definition, the elements of such feature vectors are all non-negative, which is a necessary condition for applying NNMF to the feature vectors.

We applied the NNMF-based clustering method to the trained network with exactly the same parameter as the network shown in Figures \[fig:mnist\_align\_lnn\] and \[fig:food\_align\_lnn\]. With the MNIST data set[@LeCun1998] and the data set of a consumer price index[@estat], respectively, we decomposed the trained 