this translates into a greater penalisation of $\|{{\boldsymbol{\mathbf{}}}}{J}_0({{\boldsymbol{\mathbf{}}}}{x})\|_F$. As such, noising each layer should induce a smaller $\|{{\boldsymbol{\mathbf{}}}}{J}_0({{\boldsymbol{\mathbf{}}}}{x})\|_F$, and larger classification margins than solely noising data. We support this empirically in Figure \[fig:adv\_attack\] and show that the regularisers $R$ replicate this effect.

Calibration
-----------

\

A neural network classifier gives a prediction $\hat{y}({{\boldsymbol{\mathbf{}}}}{x})$ with confidence $\hat{p}({{\boldsymbol{\mathbf{}}}}{x})$ (the probability attributed to that prediction) for a datapoint ${{\boldsymbol{\mathbf{}}}}{x}$. Perfect calibration consists of being as likely to be correct as you are confident: $p(\hat{y}=y|\hat{p}=r)=r, \quad \forall r\in[0,1]$

In Appendix \[app:fourier\_capacity\], we establish a connection between the Fourier domain and model capacity, which is a measure of model complexity, showing that models with low-frequency components correspond to less ‘complex’ models with lower capacity. @Guo2017 show that models with lower capacity are better calibrated and output a broader range of prediction probabilities. We show that this holds true for models trained with GNIs and with the explicit regulariser, in Figure \[fig:calibration\]. We leave establishing a formal connection between the Fourier domain and calibration for future work.

Related Work
============

Many variants of GNIs have been proposed to regularise neural networks. @Poole2014 extend this process to its logical conclusion and apply noise to all computational steps 