& \dots & \frac{\partial }{\partial z_1}\frac{\partial}{\partial z_d} \\
\vdots & & \vdots \\
\frac{\partial }{\partial z_d}\frac{\partial }{\partial z_1} f & \dots & \frac{\partial^2}{\partial z_d^2} f \\
\end{pmatrix}$$

A point $p$ is a degenerate critical point if all the derivatives of $f$ vanish at $p$ and in addition the Hessian of $f$ does not have full rank at $p$. In other words, $$det(Hess(f)) (p) = 0.$$

Finally, we say that $p$ is a critical point of order $k$ if all of the derivatives of $f$ up to order $k$ vanish at $p$. For example, if $p$ is a critical point and the Hessian of $f$ at $p$ is identically zero, then $p$ is a critical point of order at least 2.

### Description

Consider a feedforward neural network as above, $$A_\ell \circ \sigma \circ ... \circ \sigma \circ A_1 = M_\ell(\sigma(M_{\ell-1}(.... ( \sigma (M_1 x + b_1) ... ) + b_\ell.$$

For any $1 \leq k \leq \ell-1$ we define the locus $S_k$ by the following formula: $$\begin{aligned}
S_k
=
\left\{
p = (M_1,b_1,\ldots,M_\ell,b_\ell)
\:\left|\:
M_\ell = M_k = 0,
b_\ell = \sum_{\alpha=1}^n y_\alpha,
b_k=\cdots=b_{\ell-1}=0
\right.\right\}.\end{aligned}$$

We define the star locus $S$ as the union of all these linear subspaces $S_k$, 