<p>Found an answer from <a href="https://arxiv.org/abs/1711.04436" rel="nofollow noreferrer">SQLNet</a>'s equation:</p>

<p><a href="https://i.stack.imgur.com/Yg21q.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Yg21q.png" alt="-"></a></p>
How to define the loss function or how to optimize if the target is a set?
<p>I use a full-connected network to get the whole words distribution from the last state of an encoder.</p>

<p>For example, there are 5 words in the vocabulary.</p>

<pre><code>P = [0.1, 0.1, 0.2, 0.2, 0,4]
</code></pre>

<p>And the ground truth is a words' set for this train data.</p>

<p>I sample 3 words from the 5 words and if the target set contains the 3 words , then I want the probability of the 3 words in <code>P</code> increase, for this state. </p>

<p>If one of the 3 word is not in the target set, then I want the probability of the word in <code>P</code> decrease, for this state.</p>

<p>So I wrote these code:</p>

<pre><code>reward = [0,0,0]
</code></pre>

<p>Suppose the first 3 words are sampled from <code>P</code>, and only the first 2 of the 3 words are in the target set. And the third word is not in the target set. Then</p>

<pre><code>reward = [1,1,-1]
</code></pre>

<p>Then I compute the negative sum and dot product of <code>reward</code> and sampled 3 <code>P2=[0.1, 0.1, 0.2]</code> as the loss</p>

<pre><code>loss = -sum(reward * P2.log())
</code></pre>

<p>But I fail to get the result: The top probability words can be selected from the vocabulary for every state.</p>