<p>Short answer to the question: yes, they are overfitting. Most of the important NLP data sets are not actually well-crafted enough to test what they claim to test, and instead test the ability of the model to find subtle (and not-so-subtle) patterns in the data.</p>
<p>The best tool I know for creating data sets that help deal with this is <a href="https://github.com/marcotcr/checklist/" rel="nofollow noreferrer">Checklist</a>. The corresponding paper, <a href="https://arxiv.org/abs/2005.04118" rel="nofollow noreferrer">&quot;Beyond Accuracy: Behavioral Testing of NLP models with CheckList&quot;</a> is very readable and goes into depth on this type of issue. They have a very relevant table... but need some terms:</p>
<blockquote>
<p>We prompt users to evaluate each capability with
three different test types (when possible): Minimum Functionality tests, Invariance, and Directional Expectation tests... A Minimum Functionality test (MFT), is a collection of simple examples (and labels) to check a
behavior within a capability. MFTs are similar to
creating small and focused testing datasets, and are
particularly useful for detecting when models use
shortcuts to handle complex inputs without actually
mastering the capability.</p>
</blockquote>
<blockquote>
<p>...An Invariance test (INV) is when we apply
label-preserving perturbations to inputs and expect
the model prediction to remain the same.</p>
</blockquote>
<blockquote>
<p>A Directional Expectation test (DIR) is similar,
except that the label is expected to change in a certain way. For example, we expect that sentiment
will not become more positive if we add “You are
lame.” to the end of tweets directed at an airline
(Figure 1C).</p>
</blockquote>
<p><a href="https://i.stack.imgur.com/s743t.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/s743t.png" alt="A table from the Checklist paper about duplicate issue identification" /></a></p>
Are transformer-based language models overfitting on the paraphrase identification task? What tools overcome this?
<p>I've been working on a sentence transformation task that involves paraphrase identification as a critical step: if we are confident enough that the state of the program (a sentence repeatedly modified) has become a paraphrase of a target sentence, stop transforming. The overall goal is actually to study potential reasoning in predictive models that can generate language prior to a target sentence. The approach is just one specific way of reaching that goal. Nevertheless, I've become interested in the paraphrase identification task itself, as it's received some boost from language models recently.</p>
<p>The problem I run into is when I manipulate sentences from examples or datasets. For example, in this <a href="https://huggingface.co/transformers/v2.6.0/usage.html#sequence-classification" rel="nofollow noreferrer">HuggingFace example</a>, if I negate either sequence or change the subject to Bloomberg, I still get a majority &quot;is paraphrase&quot; prediction. I started going through many examples in the <a href="https://github.com/wasiahmad/paraphrase_identification/blob/master/dataset/msr-paraphrase-corpus/msr_paraphrase_train.txt" rel="nofollow noreferrer">MSRPC</a> training set and negating one sentence in a positive example or making one sentence in a negative example a paraphrase of the other, especially when doing so would be a few word edit. I found to my surprise that various language models, like <code>bert-base-cased-finetuned-mrpc</code> and  <code>textattack/roberta-base-MRPC</code>, don't change their confidences much on these sorts of changes. It's surprising as these models claim an f1 score of <a href="https://huggingface.co/textattack/facebook-bart-large-MRPC/blob/main/eval_results_mrpc.txt" rel="nofollow noreferrer">0.918</a>+. The dataset is clearly missing a focus on negative examples and small perturbative examples.</p>
<p>My question is, are there datasets, techniques, or models that deal well when given small edits? I know that this is an extremely generic question, much more than is typically asked on StackOverflow, but my concern is in finding practical tools. If there is a theoretical technique, then it might not be suitable as I'm in the category of &quot;available tools define your approach&quot; rather than vice-versa. So I hope that the community would have a recommendation on this.</p>