star locus, is also a union of linear spaces, and looks something like the green set in this image.](lines_planes.pdf){width="2.2in"}

Note that the dimension of the core locus is $$\begin{aligned}
dim(C) = \text{Max}_{1 \leq i<j \leq \ell-1} d - \left( (m_{\ell-1}+1)m_\ell + m_{i-1}m_i + m_{j-1}m_j + \sum_{k=i}^{\ell-1} m_k \right)\end{aligned}$$

### Vanishing of derivatives

Now, we will prove that for deep networks, every point in the star locus $S$ is a critical point and every point in the core locus $C$ is a degenerate critical point.

\[starlocus\] Given a fully connected feedforward network of depth $\ell \geq 3$, with smooth activation function satisfying $\sigma(0) = 0$ and L2 loss function $L$, every point in the star locus $S$ is a critical point.

\[corelocus\] Given a fully connected feedforward network of depth $\ell \geq 4$, with smooth activation function satisfying $\sigma(0) = 0$ and L2 loss function $L$, every point in the core locus $C$ is a degenerate critical point. Furthermore, for any point $p$ in $C$, the Hessian of $L$ at $p$ has one positive eigenvalue and the remaining eigenvalues all vanish.

The loss function is $$\begin{aligned}
L(p) = \sum_{\alpha=1}^n (f_p(x_\alpha) - y_\alpha)^2.\end{aligned}$$

This means that differentiation with respect to any parameter 