<p>The length of the predicted list is indeed not differentiable. You need to add an extra softmax output to the model predicting the length of the list, or add many sigmoid outputs predicting which entries should be included.</p>

<p>I wrote a paper about transcribing variable-length text sequences from images, and the appendix goes into a lot of detail with a worked example for how the math works:
<a href="http://arxiv.org/abs/1312.6082" rel="nofollow">http://arxiv.org/abs/1312.6082</a></p>
How to evaluate lists with predicted lengths? (`tensorflow.nn.top_k` with array of `k`s from another model)
<p>I am trying to predict medications given to patients. For each medication I have a column in the predictions (through softmax) indicating the probability that the patient will get this medication.</p>

<p>But obviously people can get several meds at once, so I have another model that tries to predict the number of different medications given.</p>

<p>I would like to evaluate them in a single TensorFlow call (I currently have a bunch of slow NumPy hacks), but I can't pass <code>tensorflow.nn.top_k</code> an array of <code>k</code>s (one for each patient, i.e. row), only a fixed integer - which doesn't work because different patients will get different numbers of meds.<br>
Ultimately I'm trying to <code>tensorflow.list_diff</code> between the actually prescribed medication indices and the predicted ones. And then maybe the <code>tensorflow.size</code> of it.</p>

<pre><code>tensorflow.list_diff(
    tensorflow.where(  # get indices of medications
            tensorflow.equal(medication_correct_answers, 1)  # convert 1 to True
    ),
    tensorflow.nn.top_k(  # get most likely medications
            medication_soft_max,  # medication model
            tensorflow.argmax(count_soft_max, 1)  # predicted count
    )[1]  # second element are the indices
)[:, 0]  # get unmatched medications elements
</code></pre>

<p><em>Bonus question: Would it be possible to train a model directly on this instead of two seperate cross entropies? It doesn't really look differentiable to me - or do only the underlying softmaxes need to be differentiable?</em></p>