<p>Neural networks operate on a continuous space, and don't know what to do with a discrete space like words. That's why NLP tasks start by embedding the discrete word IDs into a continuous space.</p>

<p>Fast Gradient Sign Method, which clearly uses the gradient and also operates that continuous space, can get you as far as an adversarial embedding. But if you want an adversarial <em>example</em>, then you need to somehow go from that adversarial embedding to an adversarial word.</p>

<p>This paper on <a href="https://arxiv.org/abs/1801.04354" rel="nofollow noreferrer">Black-box Generation of Adversarial Text Sequences</a> describes one such idea.</p>

<blockquote>
  <p>Multiple recent studies [21, 25] defined adversarial perturbations
  on RNN-based text classifiers. [21] first chose the word at a random
  position in a text input, then used a projected Fast Gradient Sign
  Method to perturb the word’s embedding vector. The perturbed vector is projected to the nearest word vector in the word embedding
  space, resulting in an adversarial sequence (adversarial examples
  in the text case).</p>
</blockquote>

<p>But right after that quote they said this technique does not always generate great examples. Perhaps it will be suitable for your purposes, or perhaps you will want to dive deeper into the paper to see how that their black box idea works.</p>

<p>Or maybe you don't need to generate adversarial words, and an adversarial embedding is sufficient. If so, read on.</p>

<hr>

<p><strong>Older idea of mine, not backed by research.</strong></p>

<p>Another path forward is to generate the adversarial example on top of the embedding, instead of the indices the embedding is based on. That is:</p>

<ol>
<li>Run the embedding.</li>
<li>Feed it directly to the <code>answer</code> part of your model, which gives one half of your loss.</li>
<li>Update the <em>embedding</em> in an adversarial way. This will now work because you are working on the embeddings, which are floating point and suitable for the FGSM update.</li>
<li>Feed the adversarial example to your <code>answer</code> subnet, which gives the second half of your loss.</li>
</ol>

<p>This is straightforward to do in PyTorch, but unfortunately I do not know a convenient way to do so in Keras given the up-front requirement to <code>compile()</code> the model instead of leaving it in two pieces.</p>
Online adversarial training on NLP task
<p>I would like to implement a custom loss function for my neural network in order to consider also the contribution of adversarial examples during training, computed with the Fast Gradient Sign Method.</p>

<p><img src="https://i.stack.imgur.com/O5v7Q.png" alt="Custom Loss Function"></p>

<p>where <strong><em>J</em></strong> is a classic categorical cross-entropy computed wrt to the inputs. And <strong><em>x + delta</em></strong> is the adversarial example.</p>

<p><strong>Network Structure</strong></p>

<p>More in details, my network is the following:</p>

<pre><code>sentence = Input(shape=(story_maxlen,))
encoded_sentence = Embedding(vocab_size, embed_size, input_length=story_maxlen)(sentence)

question = Input(shape=(query_maxlen,))
encoded_question = Embedding(vocab_size, embed_size, input_length=query_maxlen)(question)

merged = concatenate([encoded_sentence, encoded_question], axis=1)
answer = LSTM(lstm_size, return_sequences=True)(merged)
answer = Dense(mlp_size, activation='tanh')(merged)
answer = Dropout(dropout_rate)(answer)
answer = Flatten()(answer)
answer = Dense(vocab_size, activation='softmax')(answer)

model = Model([sentence, question], answer)
model.compile(optimizer="adam", loss=my_loss_wrapper([sentence,question]), metrics=['accuracy'])
</code></pre>

<p>And then my custom loss function with also the function to generate the adversarial examples:</p>

<pre><code>def generate_advers(model, epsilon):

    x1 = input_tensor[0]
    x2 = input_tensor[1]
    answer = y_true

    x1 = tf.Variable(x1)
    x2 = tf.Variable(x2)

    with tf.GradientTape() as tape:
        tape.watch([x1, x2])

        proba = model([x1, x2])            
        loss = K.categorical_crossentropy(answer, proba[0])

    # Get the gradients of the loss w.r.t to the input.
    gradient = tape.gradient(loss, [x1, x2])

    g1 = gradient[0]
    g2 = gradient[1]

    signed_grad_st = tf.sign(g1)
    signed_grad_qu = tf.sign(g2)

    delta_1 = tf.multiply(signed_grad_st, epsilon)
    delta_2 = tf.multiply(signed_grad_qu, epsilon)

    x1_adv = tf.add(x1, delta_1)
    x2_adv = tf.add(x2, delta_2)

    proba_adv = model([x1_adv, x2_adv])

    loss_advers = K.categorical_crossentropy(label, proba_adv[0])

    return loss_advers

def my_loss_wrapper(input_tensor):

    def my_loss(y_true, y_pred):
        alpha = 0.05
        alpha_compl = 1.0 - alpha
        epsilon = 0.15

        loss_advers = generate_advers(model, epsilon)
        loss_advers = alpha_compl*loss_advers
        loss_true = K.categorical_crossentropy(y_true, y_pred)
        loss_true = alpha*loss_true

        total = loss_true + loss_advers
        return total

    return my_loss
</code></pre>

<p>Giving that my input is an encoded vector of vocabulary indices of the form:</p>

<pre><code>[1,5,4,3,6,9...]
</code></pre>

<p>I don't understand how to compute the gradient of the loss wrt to the input (it is always None), which is fundamental to implement the FGSM. Do you have any suggestions? Also, do you think I'm on the right way?</p>

<p><strong>Important</strong></p>

<p>I'm able to compute the gradient if and only if i <strong>remove</strong> the Embedding layer from the network. But then the problem is that I can't train my embeddings and so the accuracy does not increase. So I need the Embedding layer to be in the network.</p>