into the mechanism underpinning this regularisation by considering the cases of regression and classification models separately, which we do in the following section.

**Regularisation in Regression.** In the case of regression one of the most commonly used loss functions is the mean-squared error (MSE), which is defined for a point $n$ as follows: $$\mathcal{L}({{\boldsymbol{\mathbf{}}}}{x}) = ({{\boldsymbol{\mathbf{}}}}{y}_n - {{\boldsymbol{\mathbf{}}}}{h}_{L}({{\boldsymbol{\mathbf{}}}}{x}))^2
\label{eq:mse}$$ For this loss, the Hessians in Theorem \[prop:explicit\_reg\] are simply the identity matrix. Ignoring the higher order terms in Theorem \[prop:explicit\_reg\] for small variance noise injections, the explicit regularisation term, which is guaranteed to be PSD, is: $$R \approx \frac{1}{2} \mathbb{E}_{{{\boldsymbol{\mathbf{}}}}{x} \sim \mathcal{B}} \left[ \sum_{k=0}^{L-1} \sigma^2_k (\|{{\boldsymbol{\mathbf{}}}}{J}_{k}({{\boldsymbol{\mathbf{}}}}{x})\|^2_F) \right]
    \label{eq:mse_reg_term}$$

where $\sigma^2_k$ is the variance of the noise ${{\boldsymbol{\mathbf{}}}}{\epsilon}_{k}$ injected at layer $k$ and $||\cdot||_F$ is the Frobenius norm. See Appendix \[app:exp\_reg\_regression\] for a proof.

**Regularisation in Classification.** In the case of classification, we consider the case of a cross-entropy (CE) loss. Recall that we consider our network outputs ${{\boldsymbol{\mathbf{}}}}{h}_L$ to be the pre-$\mathrm{softmax}$ of logits of the final layer ${{\boldsymbol{\mathbf{}}}}{L}$. We denote ${{\boldsymbol{\mathbf{}}}}{p}({{\boldsymbol{\mathbf{}}}}{x})=\mathrm{softmax}({{\boldsymbol{\mathbf{}}}}{h}_L({{\boldsymbol{\mathbf{}}}}{x}))$. The loss is thus: $$\mathcal{L}({{\boldsymbol{\mathbf{}}}}{x}) = - \sum_{c=0}^M {{\boldsymbol{\mathbf{}}}}{y}_{n,c} \log ({{\boldsymbol{\mathbf{}}}}{p}({{\boldsymbol{\mathbf{}}}}{x}))_c),
\label{eq:cross_entropy}$$ where $c$ indexes over the $M$ possible classes of the classification problem. The hessian ${{\boldsymbol{\mathbf{}}}}{H}_{L}({{\boldsymbol{\mathbf{}}}}{x})$, which 