derived $R$ are equivalent to penalising the term associated with $\alpha = 1$ in the Sobolev norm with coefficient $\lambda_i$, and penalising the term associated with $\alpha = 0$ with coefficient 0 (see definition above for the Sobolev norm). This follows the Tikhonov regularisation definition of [@Tikhonov1977] and @Bishop1995 which allows for different coefficients to weight each term of the sum that defines the Hilbert space norm and allows for data-dependent $\lambda_i$.

As such, the explicit regulariser is the sum of the Fourier regularisers for each *subnetwork* within a network. Training with Gaussian noise is equivalent to a prior in Sobolev space which favours smooth functions with low-frequency components. We demonstrate this empirically in Figure \[fig:network\_fourier\] for models trained with noise and the explicit regulariser. Clearly, these models learn a function with low-frequency components, that doesn’t overfit to data.

![As in @Rahaman2019, we train 6-layer deep 256-unit wide ReLU networks trained to regress the function $\lambda(z) = \sum_i \sin(2\pi k_iz+\phi(i))$ with $k_i  \in(5,10,\dots,45,50)$. We train these networks with no noise (Baseline), with GNIs of variance $\sigma^2=0.1$ injected into each layer except the final layer (Noise), and with the explicit regulariser for regression in (Exp Reg). The first row shows the 