<p><strong>CrossEntropy</strong> is not the best loss function when you deal with Top-k accuracy because cross-entropy may be prone to <strong>overfitting on small datasets or noisy labels</strong>.</p>
<p>As you have already pointed out, &quot;smooth loss&quot; functions are developed for top-k classification with SVM. To my knowledge, there is no a &quot;off-the-shelf&quot; loss function in Keras/TF that is best suited for top-k. However, I suggest you to try <strong>Smooth Surrogate Loss</strong> <strong>(SSL)</strong> presented in the <a href="https://arxiv.org/pdf/1802.07595.pdf" rel="nofollow noreferrer">article</a> and implemented in <em>Pytorch</em> to use with deep neural networks (see <a href="https://github.com/oval-group/smooth-topk" rel="nofollow noreferrer">Github</a>). It derives from multi-class SVMs as SSL creates a margin between the correct top-k predictions and the incorrect ones. The training time of SSL is comparatevely the same as in the case of cross-entropy thanking to a divide-and-conquer approach and the use of polynomials (see <a href="https://github.com/oval-group/smooth-topk/tree/master/topk/polynomial" rel="nofollow noreferrer">implementation</a>).</p>
What loss function to use in Keras when metric is SparseTopKCategoricalAccuracy/TopKCategoricalAccuracy?
<p>For multiclass classification problems, Keras and tf.keras have metrics like SparseTopKCategoricalAccuracy and TopKCategoricalAccuracy. However, if one uses loss functions like SparseCategoricalCrossentropy or CategoricalCrossentropy, they cannot achieve the max values for these two metrics.</p>
<p>What is a good loss function to use when one wants to maximize SparseTopKCategoricalAccuracy or TopKCategoricalAccuracy?</p>
<p>I understand that SparseTopKCategoricalAccuracy is not differentiable, just like Accuracy. I am trying to find a function that can approximate the smooth loss function and yield a higher number for SparseTopKCategoricalAccuracy.</p>