under assumptions mild enough to include real-world architectures. The author studied the locus of global minima in [@cooper], where under mild assumptions we showed that for feedforward networks, the locus of global minima of $L$ forms a smooth, possibly disconnected manifold $M$ and computed its dimension.

Whether $M$ is connected has been studied both theoretically and empirically. In [@quynhconnectedsublevel], Ngyuen et al found that under stronger assumptions, it is possible to prove that the locus of global minima is in fact connected. Working in a different setting, in[@venturibruna] Venturi et. al. also prove the connectedness of the locus of global minima under strong assumptions.

In related work [@sanjeevepsilonconnected], Arora et al showed that under a different but again strong set of assumptions, any two dropout-stable minima are $\epsilon$-connected. It has also been empirically observed by several groups including [@LossSurfaces] and [@NoBarriers], that if one trains a neural net twice, two different global minima are found, but that it is often possible to construct a path between them along which the loss does not increase much along the path.

The geometry of the locus of local non-global minima is much less well understood than that of global minima. Kawaguchi showed that 