variance $\sigma^2=0.1$ injected into each layer except the final layer (Noise), and with the explicit regulariser for regression in (Exp Reg). The first row shows the Fourier spectrum (x-axis) of the networks (calculated using Lemmas 1 and 2 of @Rahaman2019) as training progresses (y-axis) averaged over 10 training runs. Colours show each frequency’s amplitude clipped between 0 and 1. The second row shows samples of randomly generated target functions and the function learnt by the networks. []{data-label="fig:network_fourier"}](figures/spectral_dynamics_exp_reg10x.png "fig:"){width="28.00000%"}\

GNIs and Model Sensitivity {#sec:sensitivity}
==========================

@Rahaman2019 showed empirically that functions biased towards lower frequencies in the Fourier domain are less sensitive to noisy data, and there is ample evidence demonstrating that models trained with noised data are less sensitive to perturbations by inducing larger classification margins [@Cohen2019; @Liu2018; @Li2018].

Given this, our aim is twofold. First, we want to establish a theoretical understanding of the link between a classifier’s Fourier spectrum and its classification margins. Second, because noisy data has already been shown to induce larger classification margins, we want to ascertain whether extending noise to all intermediate layers of a network further decreases network sensitivity.

Classifier Sensitivity
----------------------

As discussed previously, a model’s sensitivity to noise is intrinsically linked to 