is $\nabla^2 \mathcal{L}({{\boldsymbol{\mathbf{}}}}{x})\rvert_{{{\boldsymbol{\mathbf{}}}}{h}_{L}({{\boldsymbol{\mathbf{}}}}{x})} \in \mathbb{R}^{N_L \times N_L}$, is easy to compute and has the form: $${{\boldsymbol{\mathbf{}}}}{H}_{L}({{\boldsymbol{\mathbf{}}}}{x})_{i,j} = 
\begin{cases}
{{\boldsymbol{\mathbf{}}}}{p}({{\boldsymbol{\mathbf{}}}}{x})_i(1 - {{\boldsymbol{\mathbf{}}}}{p}({{\boldsymbol{\mathbf{}}}}{x})_j) & i = j \\
-{{\boldsymbol{\mathbf{}}}}{p}({{\boldsymbol{\mathbf{}}}}{x})_i{{\boldsymbol{\mathbf{}}}}{p}({{\boldsymbol{\mathbf{}}}}{x})_j & i \neq j \\
\end{cases}$$ This Hessian is itself PSD and if we ignore the higher order terms in Theorem \[prop:explicit\_reg\] for small variance noise injections, the explicit regularisation term, which is PSD can be approximated by: $$R \approx \frac{1}{2} \mathbb{E}_{{{\boldsymbol{\mathbf{}}}}{x} \sim \mathcal{B}} \left[ \sum_{k=0}^{L-1} \sigma_k^2 \sum_{i,j}(\mathrm{diag}( {{\boldsymbol{\mathbf{}}}}{H}_{L}({{\boldsymbol{\mathbf{}}}}{x}))^T{{{\boldsymbol{\mathbf{}}}}{J}}^2_{k}({{\boldsymbol{\mathbf{}}}}{x}) )_{i,j} \right]
\label{eq:ce_reg_term}$$ where as before $\sigma^2_k$ is the variance of the noise ${{\boldsymbol{\mathbf{}}}}{\epsilon}_{k}$ injected at layer $k$. See Appendix \[app:exp\_reg\_classification\] for a detailed demonstration of this.

\

In Figure \[fig:marg\_approx\] we show that models trained with $R$ and GNIs have similar test-set loss and parameter Hessians throughout training, even when using ELU activations which test the first-order approximations for small variance noise we have made. This means that models trained with $R$ and GNIs have almost identical trajectories through the loss landscape and that they have smoother trajectories (smaller Hessian trace) than models trained without regularisation.

In both classification and regression the explicit regulariser penalises the norm of the Jacobians $\|{{\boldsymbol{\mathbf{}}}}{J}_k\|^2_F$. As we now show, this corresponds to a penalisation in the Fourier 