the size of its classification margins. Intuitively a classification margin can be described as the minimal perturbation norm necessary to fool the classification function. @Sokolic2017 and @Jakubovitz2018 define a classification margin $M$ that is the radius of the largest metric ball centered on a point ${{\boldsymbol{\mathbf{}}}}{x}$ to which a classifier assigns ${{\boldsymbol{\mathbf{}}}}{y}$, the true label.

Consider a classifier that outputs a correct prediction for the true class $A$ associated with a point ${{\boldsymbol{\mathbf{}}}}{x}$. Then the first order approximation for the l2-norm of the classification margin $M$, which is the minimal perturbation necessary to fool a classifier, is lower bounded by: $$\begin{aligned}
M({{\boldsymbol{\mathbf{}}}}{x}) \geq \frac{({{\boldsymbol{\mathbf{}}}}{h}^A_{L}({{\boldsymbol{\mathbf{}}}}{x}) - {{\boldsymbol{\mathbf{}}}}{h}^B_{L}({{\boldsymbol{\mathbf{}}}}{x}) )}{\sqrt{2} \|{{\boldsymbol{\mathbf{}}}}{J}_0({{\boldsymbol{\mathbf{}}}}{x}))|_F}. 
\label{eq:radius_robust_frob}\end{aligned}$$ We have ${{\boldsymbol{\mathbf{}}}}{h}^A_{L}({{\boldsymbol{\mathbf{}}}}{x}) \geq {{\boldsymbol{\mathbf{}}}}{h}^B_{L}({{\boldsymbol{\mathbf{}}}}{x})$, where ${{\boldsymbol{\mathbf{}}}}{h}^A_{L}({{\boldsymbol{\mathbf{}}}}{x})$ is the $L^{th}$ layer activation (pre-softmax) associated with the true class $A$, and ${{\boldsymbol{\mathbf{}}}}{h}^B_{L}({{\boldsymbol{\mathbf{}}}}{x})$ is the second largest $L^{th}$ layer activation.

Networks that have lower-frequency spectrums and consequently have smaller norms of Jacobian (as in equation ), will have larger classification margins and will be less sensitive to perturbations. This broadly follows our prior discussion, but expresses it mathematically and explains the empirical observations of @Rahaman2019.

What does this entail for GNIs applied to each layer of a network ? We can view the 