wide networks, where work of [@dulee], [@zhu], [@ntk], and many others proves that with high probability, gradient flow from random initializations converge to global minima. Our theoretical understanding of very wide networks is more limited, but has also seen substantial progress in the past few years.

In [@fullrank], [@quynhconnectedsublevel], [@ruoyumeasure0], and [@venturibruna], several groups showed in a number of different settings that the sublevel sets of $L$ are connected. In particular, this implies that in this setting all critical points of $L$ are degenerate. In this work, we build on their work and give lower bounds on the number of globally flat directions of the function $L$ at any critical point $p$.

[**Theorem \[dimlinuniform\].**]{} Consider a very wide feedforward neural network with smooth activation $\sigma$ and $L2$ loss $L$. For any critical point $p$ of $L$, the level set of $p$ contains a linear subspace of dimension $(m_{\ell-1}+1 - n) m_\ell$.

A corollary of this Theorem stated in the perhaps more familiar language of zero eigenvalues of the Hessian of $L$ is the following.

[**Corollary \[critzeroeigenuniform\].**]{} Consider a very wide feedforward neural network with smooth activation $\sigma$ and $L2$ loss $L$. For any critical point $p$ of $L$, $Hess(L)$ at 