<p>The answer to this lies in the (admittedly very brief) description of what the tasks are about:</p>

<blockquote>
  <p>[<code>BertForMultipleChoice</code>] [...], e.g. for RocStories/SWAG tasks.</p>
</blockquote>

<p>When looking at the <a href="https://arxiv.org/pdf/1808.05326.pdf" rel="noreferrer">paper for SWAG</a>, it seems that the task is actually learning to <em>choose from varying options</em>. This is in contrast to your "classical" classification task, in which the "choices" (i.e., classes) <em>do not vary</em> across your samples, which is exactly what <code>BertForSequenceClassification</code> is for.</p>

<p>Both variants can in fact be for an arbitrary number of classes (in the case of <code>BertForSequenceClassification</code>), respectively choices (for <code>BertForMultipleChoice</code>), via changing the <code>labels</code> parameter in the config. But, since it seems like you are dealing with a case of "classical classification", I suggest using the <code>BertForSequenceClassification</code> model.</p>

<p>Shortly addressing the missing Softmax in <code>BertForSequenceClassification</code>: Since classification tasks can compute loss across classes indipendent of the sample (unlike multiple choice, where your distribution is changing), this allows you to use Cross-Entropy Loss, which factors in Softmax in the backpropagation step for <a href="https://medium.com/@zhang_yang/understanding-cross-entropy-implementation-in-pytorch-softmax-log-softmax-nll-cross-entropy-416a2b200e34" rel="noreferrer">increased numerical stability</a>.</p>
BertForSequenceClassification vs. BertForMultipleChoice for sentence multi-class classification
<p>I'm working on a text classification problem (e.g. sentiment analysis), where I need to classify a text string into one of five classes.</p>
<p>I just started using the <a href="https://huggingface.co/transformers/index.html" rel="noreferrer">Huggingface Transformer</a> package and BERT with PyTorch. What I need is a classifier with a softmax layer on top so that I can do 5-way classification. Confusingly, there seem to be two relevant options in the Transformer package: <a href="https://huggingface.co/transformers/model_doc/bert.html#bertforsequenceclassification" rel="noreferrer">BertForSequenceClassification</a> and <a href="https://huggingface.co/transformers/model_doc/bert.html#bertformultiplechoice" rel="noreferrer">BertForMultipleChoice</a>.</p>
<p><strong>Which one should I use for my 5-way classification task? What are the appropriate use cases for them?</strong></p>
<p>The documentation for <strong>BertForSequenceClassification</strong> doesn't mention softmax at all, although it does mention cross-entropy. I am not sure if this class is only for 2-class classification (i.e. logistic regression).</p>
<blockquote>
<p><em>Bert Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks.</em></p>
<ul>
<li><em><strong>labels</strong> (torch.LongTensor of shape (batch_size,), optional, defaults to None) – Labels for computing the sequence classification/regression loss. Indices should be in [0, ..., config.num_labels - 1]. If config.num_labels == 1 a regression loss is computed (Mean-Square loss), If config.num_labels &gt; 1 a classification loss is computed (Cross-Entropy).</em></li>
</ul>
</blockquote>
<p>The documentation for <strong>BertForMultipleChoice</strong> mentions softmax, but the way the labels are described, it sound like this class is for multi-label classification (that is, a binary classification for multiple labels).</p>
<blockquote>
<p><em>Bert Model with a multiple choice classification head on top (a linear layer on top of the pooled output and a softmax) e.g. for RocStories/SWAG tasks.</em></p>
<ul>
<li><em><strong>labels</strong> (torch.LongTensor of shape (batch_size,), optional, defaults to None) – Labels for computing the multiple choice classification loss. Indices should be in [0, ..., num_choices] where num_choices is the size of the second dimension of the input tensors.</em></li>
</ul>
</blockquote>
<p>Thank you for any help.</p>