\subsection{Evaluation Procedures}

One of the go-to~\cite{szegedy2014intriguing, kannan2018adversarial} datasets when it comes to
testing defenses is MNIST~\cite{lecun}. Modern papers~\cite{kannan2018adversarial,
tramèr2020ensemble} use ImageNet~\cite{ILSVRC15} as a training/assessment database. However we did
not get to considering ImageNet due to deadlines. So, we kept our evaluation to MNIST\footnote{MNIST
was suggested by Ali Varamesh as well; he was inspired by \cite{szegedy2014intriguing} and
\cite{goodfellow2015explaining}}. This dataset involves 70,000 images, 60,000 of which are meant for
training, and 10,000 for testing. We chose 4,200 of the 60k as validation images (used for early
stopping), and 55,800 used for the optimization procedure. The validation dataset was not subject to
an adversary. Moving on, each image is 28 pixels wide and tall, with a single channel that
represents grayscale data. The maximum value of a pixel is 255 (with the minimum at 0); however,
unlike what is common in imagery, 255 represents black, and decreasing values cause increasing
whiteness. This is an important point: \cite{szegedy2014intriguing}, for example, understandably,
has 0 and 255 be the black and white, respectively. In order to be consistent with testing, we
followed their assumption and did this same. Following \cite{szegedy2014intriguing}, in which they
claim to have trained their networks for perfect \textit{training set} classification, we
unsuccessfully tried to do the same, but got close\footnote{We had seen a reference that did not
achieve this either, implying that we are not alone and this might be normal, but have lost track of
the reference}.

We chose the ``FC-100-100-10'' network from \cite{szegedy2014intriguing} as our test network.
However, it was necessary in some cases to modify the network as Pairwise Difference and Angular are
not standard neurons. Further, in the latter case, we didn't use an activation at all; this is due
to cosine having curvature and putting out values between -1 and 1 (note that these values are
actually scaled depending on the value of its denominator constant). For Pairwise Difference, we
used sigmoid as the final output of the layer. Because FC-100-100-10 is fully-connected, this meant
that both Pairwise Difference and Angular took as input the output of the last layer or the image
itself. Finally, the Pairwise Difference (which was only used for the first layer due to its
pixel-oriented nature) and Angular-based networks used Batch Renormalization~\cite{ioffe2017batch},
and we trained a normal FC-100-100-10 network with Batch Renormalization for the purposes of an
ablation study (as we had found in the past that Batch Normalization~\cite{ioffe2015batch} acted as
a defense itself). In fact, the results for the network outside of ablation are interesting from an
adversarial standpoint, so we include them for that reason as well.

Our complete implementation, found at \cite{mycode}[Friendly], used Chainer~\cite{tokui2019chainer,
language, 2020NumPy-Array, cupy_learningsys2017}. The training procedures outside of what has been
described were very straightforward. We used momentum-based Stochastic Gradient Descent, and
learning rates varied to fit the model more appropriately; momentum was kept the same for all
models, as well as the batch size (20). The main reason for such a small number is that Pairwise
Difference can use up a lot of memory, and we thought it fair that all networks use the same batch
size for proper comparison. Early stopping was also employed, but none of our networks ever hit its
20 epoch limit (based on validation set performance) during the 100 epoch hard ``deadline'', so to
speak. Prediction loss was based on mean squared error, and it was formulated as
\[
    \frac{1}{s} \sum_{i=1}^{s} (\mathbf{c}_i - \mathbf{o}_i)^T (\mathbf{c}_i - \mathbf{o}_i)
\]
where $s$ is the number of samples in the batch, $\mathbf{c}$ is the correct confidence vector
(a.k.a.\ one-hot), and $\mathbf{o}$ is the outputs of the network for sample $i$. In the case of the
network whose derivative is penalized, that penalty is given a weight of 0.5, while the mean squared
error term has weight 1.0. See table \ref{settings} for the full list of hyperparameter settings
that vary between networks. Note that we tried only to follow through mostly with point 2 and
partially with point 1 in \ref{oggafsoscdtae}; time prevented us from exhausing all three points.
Specific to point 1, we do not achieve the properties ``any compelling threat model should at the
very least grant knowledge of the model architecture, training algorithm, and allow query
access''~\cite{athalye2018obfuscated}[§ 6.1] and ``[it] is not meaningful to restrict the
computational power of an adversary artificially (e.g., to fewer than several thousand attack
iterations)''~\cite{athalye2018obfuscated}[§ 6.1].
\begin{table}[th]
    \begin{center}
        \begin{tabular}{| c | c | c | c | c | c | c |}
            \hline
                                  &     S       &     RS      &   A     &     GMR     &      GMRES        &   PD     \\
            \hline
            Batch Renormalization &     no      &     yes     &   yes   &     no      &        no         &  yes     \\
            Activation function   &   sigmoid   &   sigmoid   &   N/A   &   sigmoid   &  Elastic Sigmoid  & sigmoid  \\
            Learning rate         &    0.002    &    0.002    & 0.0002  &    0.0009   &      0.002        &  0.002   \\
            \hline
        \end{tabular}
    \end{center}
    \caption{``S'' stands for ``Standard'', ``RS'' for ``Renormalized Standard'', ``A'' for
             ``Angular'', ``GMR'' for GMR, ``GMRES'' for the addition of Elastic Sigmoid to GMR, and
             ``PD'' for ``Pairwise Difference''}
    \label{settings}
\end{table}
As is stated in \cite{athalye2018obfuscated}, it is important to conduct tests using a separate
network for generating adversarial examples due to ``gradient masking'' (as a reminder, this is an
effect where the trained network can handle attacks that use the network's gradient, while not being
able to handle other kinds of attacks; this author only knows of gradient-based attacks which use a
different network, but there may be something he is missing). As a result, they recommend black-box
attacks, where attacks are performed not using any internal values (such as the gradient) from the
network being tested. As a result, we also show black-box results where we train
another\footnote{~\cite{madry2019deep, szegedy2014intriguing} are the inspirations for ``another''}
original, non-protected network provides the adversarial examples to each of the defended networks.
The results on the MNIST test set can be found in table \ref{defensiveproficiency}. We followed
\cite{madry2019deep} and used the $[0, 255]$ equivalent (77) of the 0.3 (out of 1.0) $L_\infty$
noise that they chose. We use $L_{\infty}$ as it is used throughout the literature
(\cite{goodfellow2015explaining, madry2019deep, tramèr2020ensemble}, etc) and is the easiest to
program for. To disclose discrepancies in performance under normal circumstances, we present table
\ref{clean}. All testing was performed \textit{not} using the values Batch Renormalization learned
(which turns it into Batch Normalization~\cite{ioffe2015batch}, according to \cite{ioffe2017batch}[§
3]) as we had very strange (at least from the perspective of our knowledge of Batch Renormalization)
performance issues otherwise; this issue may be the author's fault, but we did not get to resolving
it.

\begin{table}
    \begin{center}
        \begin{tabular}{| c | c | c | c | c | c | c |}
            \hline
                             &      S      &    RS     &    A     &   GMR     &   GMRES   &   PD     \\
            \hline
            White-box (FGSM) &  46.16\%    &  43.30\%  &  0.53\%  &  37.67\%  &  36.98\%  &  32.20\% \\
            Black-box (FGSM) &  45.28\%    &  45.27\%  &  56.35\% &  43.24\%  &  44.70\%  &  49.87   \\
            \hline
        \end{tabular}
    \end{center}
    \caption{Performance of each network when it comes to black-box and white-box attacks.
             Table \ref{settings}'s caption has the meaning of the top-row acronyms.}
    \label{defensiveproficiency}
\end{table}

\begin{table}
    \begin{center}
        \begin{tabular}{| c | c | c | c | c | c |}
            \hline
            S        &     RS    &     A     &   GMR    &   GMRES   &   PD     \\
            \hline
            95.86\%  &  97.80\%  &  85.40\%  &  96.09\% &  96.72\%  &  95.91\% \\
            \hline
        \end{tabular}
    \end{center}
    \caption{Same layout as table \ref{defensiveproficiency}, but the networks were not attacked.}
    \label{clean}
\end{table}

\subsection{Outcomes}

The results turned out to be poor. It is questionable whether or not we can make any claims
whatsoever. Angular's adversarial performance was just bad (even relative to a normal network
without defenses) when up against a white-box attack, a surprising find. It did do pretty well for
the black-box equivalent, but, again, its performance in table \ref{clean} makes this network nearly
useless considering that most scenarios in real life will not be adversarial. Further, such
comparisons are not fair because Angular is not in the same performance ``state'' as the other
networks; this is why ~\cite{szegedy2014intriguing} tries to keep performance across networks the
same. If we were to actually get the angular network's accuracy up enough, it could be that the
black-box benefits would disappear. It also turned out that there was no point to ablation because
there were no benefits of using Batch (Re)normalization. Speculation about the results can be found
in \ref{analysis}.


\subsection{Analysis}
\label{analysis}

The performance of an Angular-based network is simultaneously surprising and possibly easy to
explain. The first rationale could be that we did not pick an adequate constant to nullify the
aforementioned issue of many of zero-length vectors. None of these vectors actually ended up a zero
because each of them contained the whole image (a vector with all-zero values would contain no
number pixels), but the constant may cause issues regardless if not chosen properly. However, a more
likely scenario is that the scaling of the gradient with the addition of more elements is not as
good as hoped. As stated previously, ideally the gradient of each element in the input vector of an
Angular neuron contributes a normalized amount such that all partial derivatives in the vector add
up to, roughly speaking, the same value as when using a small patch as input. Such a neuron would
not be too dissimilar from Angular, but coming up with the formulation that we would want probably
entails integrating from the derivative that we want. We may even be able to drop normalizing the
weight vector by dividing it by $\left\| \mathbf{w} \right\|$ entirely, seeing as it is held
constant at time of attack. Though hardly fleshed-out, we would need to integrate something similar
to $\frac{\mathbf{a}}{\left\| \mathbf{a} \right\|_1}$ (this does not involve the weight vector, so
this integration is more complicated than what is shown).

As far as Pairwise Difference, the most likely explanation is that most pairs involve a comparison
between two zero-value pixels, which puts each subtraction squarely in the middle of the sigmoid
used for comparison. As a result, even FGSM using a max-norm of just 10 has a reduced 34.50\%
accuracy. It is possible that treating each comparison as a miniature neuron with two weights (or
even a bias) could be a fix for this issue because, as previously stated, this would still attempt
to address the issue of large vectors in dot products.

Renormalization appears to have been worse for the original network, which actually checks out. This
is due to the fact that \cite{ioffe2015batch} states that Batch Normalization~\cite{ioffe2015batch}
(Renormalization's predecessor) aims to keep the input to the activation function centered near, and
tight around, the origin (at least in the case of the sigmoid function) which is where the strongest
gradients lie. While this is a blessing for optimization during training, it's also a blessing for
any attacker who uses gradients in their attacks. It is possible that one conclusion that should be
drawn from this is that a Batch Renormalization-trained network in Batch Normalization mode appears
to be detrimental from a defensive standpoint.

Gradient Magnitude Reduction's lack of usefulness is concerning. The theory behind GMR, at least
intuitively, makes sense, and this may just come down to using the wrong weight in the loss term
involving the gradient w.r.t.\ the image. However, an important point to raise is that GMR does not
make any guarantees about reducing the gradient in any location other than at the point at which the
gradient is calculated. It may help to penalize even the second derivative (as it has a hand in
determining the first derivative in other locations), but the Universal Approximation
Theorem\cite{HORNIK1989359} makes it possible that even the second derivative may not be enough
seeing as there may be a valid third derivative, fourth derivative, etc. It is relevant to note that
adversarial training side steps this issue by essentially reducing the gradient at points far
(relatively speaking) from the normal datapoint, forcing (according to the evidence) all derivatives
(not just the first) to be close to zero. Elastic Sigmoid paired with GMR was ineffective as well,
for the same reasons.

Clearly, we have good reason to look further into all the defenses proposed; future work will need
to be responsible for doing so. We also need to consider Projected Gradient
Descent~\cite{madry2019deep} as an attack, as recommended by \cite{athalye2018obfuscated} because of
its effectiveness.