in a neural network layer. Not only is noise applied to the layer input it is applied to the layer output and to the pre-activation function logits. The authors allude to explicit regularisation but only derive a result for a single layer auto-encoder with a single noise injection. Similarly, @Bishop1995 derive an analytic form for the explicit regulariser induced by noise injections on *data* and show that such injections are equivalent to Tikhonov regularisation in an unspecified function space.

Recently @Wei2020 conducted similar analysis to ours, dividing the effects of Bernoulli dropout into *explicit* and *implicit* effects. Their work is built on that of @Mele1993 and @Helmbold2015 who perform this analysis for linear neural networks. @Arora2020 derive an explicit regulariser for Bernoulli dropout on the final layer of a neural network.

Conclusion
==========

In this work, we derived analytic forms for the explicit regularisation induced by Gaussian noise injections. Having characterised the explicit regulariser as a form of Tikhonov regularisation which penalises networks with high-frequency content in the Fourier space, we leverage this knowledge to show that noise injections produce wider classification margins and better calibration.

Impact Statement {#impact-statement .unnumbered}
================

This paper uncovers a new mechanism by which a 