and linear at the origin. He et al. (2015) proposed a weight initialization method (called [*He initialization*]{} here) with a normal distribution (either $N ( 0, 2 / n_i )$ or $N ( 0, 2 / n_{i+1} )$ ) for the neural networks with the ReLU (Rectified Linear Unit) activation function. The above two initialization methods are driven by experiments to monitor activations and back-propagated gradients during learning.

On the other hand, local minima of deep neural networks has been investigated theoretically in recent years. Local minima cause plateaus which have a strong negative influence on learning in deep neural networks. Dauphin et al. (2014) experimentally investigated the distribution of the critical points of a single-layer MLP and demonstrated that the possibility of existence of local minima with large error (i.e., bad or poor local minima) is very small. Choromanska et al. provided a theoretical justification for the work of Dauphin et al. (2014) on a deep neural network with ReLU units using the spherical spin-glass model under seven assumptions (Choromanska, Henaff, Mathieu, Arous & LeCun, 2015). Choromanska et al. also suggested that discarding the seven unrealistic assumptions remains an important open problem (Choromanska, LeCun & Arous, 2015). Kawaguchi (2016) 