<p>A Question Answering ot is basically a DL model that creates an answer by <em>extracting part of the context</em> (in you case what is called <code>text</code>). This means that the goal of the QAbot is to identify the <strong>start</strong> and the <strong>end</strong> of the answer.</p>
<hr />
<p>Basic functioning of a QAbot:</p>
<p>First of all, every word of the question and context is tokenized. This means it is (possibily divided into characters/subwords and then) convertend into a number. It really depends on the type of tokenizer (which means it depends on the model you are using, since you will be using the same tokenizer - it's what the third line of your code is doing). I suggest <a href="https://huggingface.co/course/chapter2/4?fw=pt" rel="nofollow noreferrer">this very useful guide</a>.</p>
<p>Then, the tokenized <code>question + text</code> are passed into the model which performs its internal operations. Remember when I told at the beginning that the model will identify the <code>start</code> and the <code>end</code> of the answer? Well, it does so by calculating for every token of the <code>question + text</code> the probability that that particular token is the start of the answer. This probabilities are the softmaxed version of the <code>start_logits</code>. After that, the same operations are performed for the end token.</p>
<p>So, this is what <code>start_scores</code> and <code>end_scores</code> are: the pre-softmax scores that every token is start and end of the answer, respectively.</p>
<hr />
<p>So, what are <code>start_position</code> and <code>stop_position</code>?</p>
<p>As stated <a href="https://huggingface.co/transformers/model_doc/roberta.html#robertaforquestionanswering" rel="nofollow noreferrer">here</a>, they are:</p>
<blockquote>
<p><code>start_positions</code> (<code>torch</code>.<code>LongTensor</code> of shape (<code>batch_size</code>,), optional) –
Labels for position (index) of the start of the labelled span for
computing the token classification loss. Positions are clamped to the
length of the sequence (<code>sequence_length</code>). Position outside of the
sequence are not taken into account for computing the loss.</p>
<p><code>end_positions</code> (<code>torch</code>.<code>LongTensor</code> of shape (<code>batch_size</code>,), optional) –
Labels for position (index) of the end of the labelled span for
computing the token classification loss. Positions are clamped to the
length of the sequence (<code>sequence_length</code>). Position outside of the
sequence are not taken into account for computing the loss.</p>
</blockquote>
<hr />
<p>Moreover, the model you are using (<code>roberta-base</code>, see <a href="https://huggingface.co/roberta-base" rel="nofollow noreferrer">the model on the HuggingFace repository</a> and the <a href="https://arxiv.org/abs/1907.11692" rel="nofollow noreferrer">RoBERTa official paper</a>) has <strong>NOT</strong> been fine-tuned for QuestionAnswering. It is &quot;just&quot; a model trained by using MaskedLanguageModeling, which means that the model has a general understanding of the english language, but it is not suitable for question asnwering. You can use it of course, but it would probably give non optimal results.</p>
<p>I suggest you use the same model, inthe version specifically fine-tuned on QuestionAnswering: <code>roberta-base-squad2</code>, see <a href="https://huggingface.co/deepset/roberta-base-squad2?context=Jim%20Henson%20was%20a%20nice%20puppet&amp;question=Who%20was%20Jim%20Henson%3F" rel="nofollow noreferrer">it on HuggingFace</a>.</p>
<p>In practical terms, you have to replace the lines where you load the model and the tokenizer with:</p>
<pre><code>tokenizer = RobertaTokenizer.from_pretrained('roberta-base-squad2')
model = RobertaForQuestionAnswering.from_pretrained('roberta-base-squad2')
</code></pre>
<p>This will give much more accurate results.</p>
<p>Bonus read: <a href="https://www.analyticsvidhya.com/blog/2020/07/transfer-learning-for-nlp-fine-tuning-bert-for-text-classification/" rel="nofollow noreferrer">what fine-tuning is and how it works</a></p>
Input/output format for Fine Tuning Huggingface RobertaForQuestionAnswering
<p>I'm trying to fine-tune &quot;RobertaForQuestionAnswering&quot; on my custom dataset and I'm confused about the input params it takes. Here's the sample code.</p>
<pre><code>&gt;&gt;&gt; from transformers import RobertaTokenizer, RobertaForQuestionAnswering
&gt;&gt;&gt; import torch

&gt;&gt;&gt; tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
&gt;&gt;&gt; model = RobertaForQuestionAnswering.from_pretrained('roberta-base')

&gt;&gt;&gt; question, text = &quot;Who was Jim Henson?&quot;, &quot;Jim Henson was a nice puppet&quot;
&gt;&gt;&gt; inputs = tokenizer(question, text, return_tensors='pt')
&gt;&gt;&gt; start_positions = torch.tensor([1])
&gt;&gt;&gt; end_positions = torch.tensor([3])

&gt;&gt;&gt; outputs = model(**inputs, start_positions=start_positions, end_positions=end_positions)
&gt;&gt;&gt; loss = outputs.loss
&gt;&gt;&gt; start_scores = outputs.start_logits
&gt;&gt;&gt; end_scores = outputs.end_logits
</code></pre>
<p>I'm not able to understand variables <strong>start_positions</strong> &amp; <strong>end_positions</strong> which are being given in the model as input and variables <strong>start_scores</strong> &amp; <strong>end_scores</strong> that are being generated.</p>