<p>Yes, both <code>nsl.AdversarialRegularization.perturb_on_batch()</code> and <code>cleverhans.attacks.FastGradientMethod.generate()</code> implement the Fast Gradient Sign Method in <a href="https://arxiv.org/abs/1412.6572" rel="nofollow noreferrer">Goodfellow et al. 2014</a>. And both offer parameters like epsilon and norm type to control perturbations. Since both <code>nsl</code> and <code>cleverhans</code> implements FGSM, the generated perturbations have no difference when the configurations are carefully specified. Yet some implementation details might be handled differently, especially in their default configuration. For example,</p>

<ul>
<li><code>cleverhans</code> by default takes model predictions as labels for generating adversarial perturbations, while <code>nsl</code> takes true labels.</li>
<li><code>cleverhans</code> usually expects the model to output logits (since the default <code>loss_fn</code> is <code>softmax_cross_entropy_with_logits</code>), while <code>nsl</code>'s models may output different things. In <code>nsl</code>'s adversarial training <a href="https://www.tensorflow.org/neural_structured_learning/tutorials/adversarial_keras_cnn_mnist" rel="nofollow noreferrer">tutorial</a>, the model outputs probability distributions.</li>
</ul>

<p>There could be more differences in other places. I can take a closer look if you could provide a concrete example.</p>

<p>Regarding adversarial training, <code>nsl.keras.AdversarialRegularization</code> takes the adversarial loss as regularization, which means the model is trained on both original and adversarial examples. <code>cleverhans.loss.CrossEntropy</code> also calculates loss on both original and adversarial examples, but the weighting scheme is a bit different. In <code>nsl</code> the original and adversarial examples are weighted as <code>1:multiplier</code>, while in <code>cleverhans</code> they are weighted as <code>(1-adv_coeff):adv_coeff</code>. Note that another training approach is deployed in some literatures, where the model is only trained on adversarial examples.</p>
Difference between adversarial training/perturbation with FGSM in Tensorflow nsl versus cleverhans
<p>I've implemented what I believe to be the same model training loop in both <a href="https://www.tensorflow.org/neural_structured_learning" rel="nofollow noreferrer">TensorFlow's neural structured learning</a> (<code>nsl</code>) and the <a href="https://github.com/tensorflow/cleverhans" rel="nofollow noreferrer"><code>cleverhans</code></a> library, and curiously, they show that models trained using adversarial training with the two libraries (via <code>nsl.AdversarialRegularization</code> and <code>cleverhans.attacks.FastGradientMethod</code>) do not achieve comparable performance. However, this question isn't about those specific results, so I don't attempt to replicate them here.</p>

<p>I'm curious more generally about what the implementation differences are for adversarial perturbation in <code>nsl.AdversarialRegularization.perturb_on_batch()</code> versus the <code>cleverhans</code> implementation of the same/similar functionality, which would be <code>FastGradientMethod.generate()</code>. </p>

<p>The <code>nsl</code> docs aren't especially clear, but they seem to imply that <code>nsl</code> is using the Fast Gradient Sign Method of <a href="https://arxiv.org/pdf/1412.6572.pdf" rel="nofollow noreferrer">Goodfellow et al. 2014</a>, which is supposedly the same method implemented in <code>FastGradientMethod</code>. For example, <code>nsl</code> refers to the Goodfellow et al. paper in the <a href="https://www.tensorflow.org/neural_structured_learning/tutorials/adversarial_keras_cnn_mnist" rel="nofollow noreferrer">adversarial training tutorial</a> and in some of the function <a href="https://github.com/tensorflow/neural-structured-learning/blob/c1fc41808e43784b971779cfeafa213807e708df/neural_structured_learning/lib/utils.py#L82" rel="nofollow noreferrer">docs</a>. Both libraries allow specification of similar parameters, e.g. an <code>epsilon</code> to control the level of perturbation and control over the norm used to constrain it. However, the differences in adversarially-trained performance lead me to believe that these libraries are not using the same underlying implementation. <code>nsl</code> is difficult to parse, so I am particularly curious what might be happening under the hood there.</p>

<p><strong>What are the differences in implementation in <code>nsl.AdversarialRegularization.perturb_on_batch()</code> and the <code>cleverhans.attacks.FastGradientMethod.generate()</code> which could cause different perturbations for the same inputs?</strong> Are there other differences in these functions which might contribute to differences in their performance (I am not interested in speed or efficiency, but in ways in which the results of the two perturbations might be different for the same model, epsilon, and norm).</p>