of this. For the remainder of this work, we assume that noise injections are of small variance and that as such, the higher order terms of Proposition \[app:accum\_noise\] and Theorem \[prop:explicit\_reg\] can be ignored. In the case of Proposition \[prop:accum\_noise\], for many of the functions that obey the extended envelope property (eg. $\mathrm{ReLU}$), which are locally linear, this result is locally exact. $\mathcal{O} ({{\boldsymbol{\mathbf{}}}}\gamma)$ is exactly zero if the variance of the noise is smaller than what is required to shift outside the linear region.

In Appendix \[app:exp\_reg\], we also show that the first set of terms in $R$ corresponds to the sum of the traces of the Gauss-Newton approximations of the Hessians ${{\boldsymbol{\mathbf{}}}}{H}_{k}({{\boldsymbol{\mathbf{}}}}{x})$, i.e the second order derivatives of the loss with respect to each layer’s activations. Smaller traces are measures of a smoother loss landscape and conversely larger traces indicate a ‘peakier’ and steeper landscape. Explicitly penalising this Hessian means that we are more likely to land in wider (smoother) minima, which has been shown, although this is a point of contention [@Dinh2017], to induce networks with better generalisation properties [@Keskar2019OnMinima; @Jastrzebski2017]. Though this offers one explanation for the regularisation offered by GNIs, we can glean more insight 