domain.

Fourier Domain Regularisation
=============================

In this section, we show that the regularisers we have derived are equivalent to Tikhonov regularisation in the Hilbert-Sobolev space with index $(1,2)$, which is formally defined as follows:

The Sobolev space of index (k,p), denoted $W^{k,p}(\Omega), \Omega \subset \mathbb{R}^d$, where $k$ is a non negative integer and $p\geq1$, is the space of locally integrable functions $f: \Omega \to \mathbb{R}$ such that for every $\alpha < k$ the weak derivative $D^{\alpha}f$ exists and $D^{\alpha}f \in L^p(\Omega)$. The norm in such a space is given by $\|v\|_{W^{k,p}(\Omega)} = (\sum_{|\alpha|_1 \leq k} \int_{\Omega} |D^{\alpha}f(x)|^p dx)^{\frac{1}{p}}$.

[@Hornik1991] have shown that neural networks with continuous activations, which have continuous derivatives up to order $k$, such as the sigmoid function, are universal approximators in the Sobolev spaces of order $k$. Recently [@Czarnecki2017] have shown that networks that use $\mathrm{ReLU}$-like activation functions are *also* universal approximators in the Sobolev spaces of order 1. Hence, we can view a neural network to be a parameter that indexes a function in the Sobolev space with index $(1,2)$, i.e. $f_{{{\boldsymbol{\mathbf{}}}}{\theta}} \in {W^{1,2}(\Omega)}$.

For a function, $f_{\theta}$, with a $d$-dimensional input and a single output if $f_{\theta} \in {W^{1,2}(\Omega)}, \Omega \subset \mathbb{R}^d$ holds, then we 