have that [@Novak]: $$\int_\Omega \|D f_{\theta}({{\boldsymbol{\mathbf{}}}}{x})\|^2_{L_2(\Omega)}d{{\boldsymbol{\mathbf{}}}}{x} = \int_\Omega (\prod_{j=1}^d{{\boldsymbol{\mathbf{}}}}{\omega}_j^2)|\mathcal{F}({{\boldsymbol{\mathbf{}}}}{\omega})|^2 d{{\boldsymbol{\mathbf{}}}}{\omega}, \Omega \subset \mathbb{R}^d
\label{eq:sobolev_norm}$$ where $\mathcal{F}$ is the Fourier transform of $f_{\theta}$ and $j$ indexes over ${{\boldsymbol{\mathbf{}}}}{\omega} = [\omega_1, \dots, \omega_d]$. In the case where the dataset contains finitely many points, the integral is approximated by a summation over datapoints and input dimensions, and in the case of SGD this summation occurs over a sampled batch. Further, in our case, the regularisation is applied to each intermediate set of activations and to each output neuron. Assuming differentiable and continuous activation functions, then the Jacobians in the explicit regulariser (equations and ) are equivalent to the weak derivative, giving: $$\begin{aligned}
 R &= \frac{1}{2}\mathbb{E}_{{{\boldsymbol{\mathbf{}}}}{x} \sim \mathcal{B}} \left[ \sum_{k=0}^{L-1} \sum_{i} \lambda_i \|D f^k_{\theta}({{\boldsymbol{\mathbf{}}}}{h}_{k}({{\boldsymbol{\mathbf{}}}}{x}))_i\|^2_{L_2(\Omega)} \right] \\
 &\approx \frac{1}{2}\sum_{k=0}^{L-1} \sum_{i} \int_\Omega \lambda_i (\prod_{j=1}^d{{\boldsymbol{\mathbf{}}}}{\omega}_j^2)|\mathcal{F}^k_i({{\boldsymbol{\mathbf{}}}}{\omega})|^2 d{{\boldsymbol{\mathbf{}}}}{\omega}, \Omega \subset \mathbb{R}^d
 \label{eq:sobolev_reg}\end{aligned}$$ where ${{\boldsymbol{\mathbf{}}}}{h}_0 ={{\boldsymbol{\mathbf{}}}}{x}$, $i$ indexes over output neurons, $f^k_{\theta}(\cdot)_i$ is the function from layer $k$ to the $i^{\mathrm{th}}$ network output, $d_k$ is the dimensionality of layer $k$, and $\mathcal{F}^k_{i}$ is the Fourier transform of $f^k_{\theta}(\cdot)_i$.

In our setting $\lambda_i = \sigma_k^2$ is constant for regression. For classification, $\lambda_i$ is data-dependent $\lambda_i = (\sigma_k^2\mathrm{diag}({{\boldsymbol{\mathbf{}}}}{H}_{L}({{\boldsymbol{\mathbf{}}}}{x}))^T)_i$ and becomes a part of the integral as in [@Tikhonov1977] and @Bishop1995. Generally our 