coordinates of this derivative are sum of terms of the form $$\begin{aligned}
\frac{\partial}{\partial w..^\ell} \left( \sum w..^\ell \sigma \left( \sum w..^{\ell-1} ... \sigma \left( \sum w..^2 \sigma \left( \sum w..^1 z. + b.^1 \right) + b.^2 \right) + ... + b.^{\ell-1} \right) + b.^\ell
\right) \\\end{aligned}$$ which are either 0 or of the form $$\begin{aligned}
\sigma \left( \sum w..^{\ell-1} ... \sigma \left( \sum w..^2 \sigma \left( \sum w..^1 z. + b.^1 \right) + b.^2 \right) + ... + b.^{\ell-1} \right) \\\end{aligned}$$

Every expression of this form vanishes on $S$, because every $w..^k = 0$, and for $i > k$, every $b.^i$ is also zero. So this becomes $$\begin{aligned}
\sigma \left( \sum w..^{\ell-1} \sigma \left( ... \sigma \left( \sum w..^{k+1} \sigma \left( 0 \right) + 0 \right) ... + 0 \right) \right) \\\end{aligned}$$ which vanishes because $\sigma(0) = 0$.

Hence for all the derivatives considered thus far, $\frac{\partial}{\partial *} f_p(x_\alpha) = 0$ for all $x_\alpha$. Consider the expression (\[onederivative\]) $$\frac{\partial}{\partial *} L(p) = \sum_{\alpha=1}^n 2 (f_p(x_\alpha) - y_\alpha) \frac{\partial}{\partial *} f_p(x_\alpha).$$ In this derivative, the second factor vanishes for every $\alpha$, hence $\frac{\partial}{\partial *} L(p)$ vanishes for all the parameters discussed.

We analyze the final group of derivatives, those with respect 