valuable line of future research would be to extend the work of [@jordansaddle] to larger classes of degenerate critical points. One would like to establish which kinds of degenerate critical points can be problematic for gradient-based methods and which can kinds we can guarantee will not be. It would also be valuable to understand which of the problematic kinds of degenerate critical points in fact appear as critical points of the loss function $L$.

Our contribution
----------------

In this work, we establish some basic geometric properties of the locus of critical points of the loss function of deep nonlinear neural networks. We treat all three classes of critical points that gradient descent could with positive probability converge to — global minima, local non-global minima, and degenerate saddle points.

For networks of depth $\ell \geq 3$, with smooth activation function satisfying $\sigma(0)=0$, whether overparameterized or not, we show the existence of a set $S$ we call the star locus, which is a positive dimensional locus of critical points (there may be others.)

[**Theorem \[starlocus\]**]{} Every point in $S$ is a critical point of $L$.

Within the star locus we identify a set we call the core locus $C$, which is a 