# Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets
Via In-Context Learning

Namrata Shivagunde, Vladislav Lialin, and Anna Rumshisky
Department of Computer Science
University of Massachusetts Lowell
<EMAIL_ADDRESS>

###### Abstract

Language model probing is often used to test specific capabilities of these
models. However, conclusions from such studies may be limited when the probing
benchmarks are small and lack statistical power. In this work, we introduce
new, larger datasets for negation (NEG-1500-SIMP) and role reversal
(ROLE-1500) inspired by psycholinguistic studies. We dramatically extend
existing NEG-136 and ROLE-88 benchmarks using GPT3, increasing their size from
18 and 44 sentence pairs to 750 each. We also create another version of
extended negation dataset (NEG-1500-SIMP-TEMP), created using template-based
generation. It consists of 770 sentence pairs. We evaluate 22 models on the
extended datasets, seeing model performance dip 20-57% compared to the
original smaller benchmarks. We observe high levels of negation sensitivity in
models like BERT and ALBERT demonstrating that previous findings might have
been skewed due to smaller test sets. Finally, we observe that while GPT3 has
generated all the examples in ROLE-1500 is only able to solve 24.6% of them
during probing.

##  1 Introduction

Understanding the limitations of large language models (LMs) becomes ever more
important with their accelerated adoption and application to real-life tasks.
After the original discovery that large LMs could perform simple NLP tasks
without additional training Radford et al. (2019), the use of these models has
rapidly grown, as have their capabilities Brown et al. (2020); Sanh et al.
(2021); Chowdhery et al. (2022). As the community invests considerable effort
into creating, training, and deploying these models Zhang et al. (2022); Black
et al. (2021), it is important to understand the types of data and tasks they
might not be well-suited.

The field of analysis of pre-trained models, has grown rapidly in recent years
Zagoury et al. (2021); Liu et al. (2021); Lialin et al. (2022); Srivastava et
al. (2022); Rogers et al. (2020). Methods such as attention pattern analysis
Kovaleva et al. (2019); Kobayashi et al. (2020), linear probing Tenney et al.
(2019), and zero-shot probing Belinkov et al. (2020); Talmor et al. (2019);
Ettinger (2020); Lialin et al. (2022) allow us to evaluate specific
capabilities of pre-trained models. Zero-shot methods give us arguably the
most clear picture, as they directly probe what the model learned through the
upstream task and allow the researcher to target very specific skills such as
understanding of negation or role.

However, even though these methods do not require training data, producing a
good dataset for zero-shot evaluation of these language models is not an easy
task. We want these datasets to be clean, diverse and to have enough
statistical power to be useful for model comparison Card et al. (2020). Many
existing probing datasets struggle with at least one of these requirements.

Psycholinguistic datasets used in a study by Ettinger (2020) have been
particularly interesting in that they enabled a comparison between model
behavior and human response, including both N400 effects and well-reasoned
cloze judgments by human speakers. Despite being used in multiple studies
since Lialin et al. (2022); Rogers et al. (2020); Zhang et al. (2020), these
datasets are quite small, ranging in size from 18 sentence pairs in negation
(NEG-136-SIMP) to a maximum of 44 sentence pairs in the role-reversal dataset
(ROLE-88).

In our work, the NEG-136-SIMP and ROLE-88 datasets are dramatically extended.
For each of them, we follow the original dataset collection method Fischler et
al. (1983); Chow et al. (2016), with the exception of N400 amplitude (requires
electroencephalography) and cloze probability studies on humans Federmeier and
Kutas (1999). We extended negation dataset using two methods 1) a template-
based and 2) using GPT3111We use text-davinci-002 version of GPT3 in all our
experiments with human-in-the-loop.222Datasets and code are available on
https://github.com/text-machine-lab/extending_psycholinguistic_dataset.

To understand how well language models perform on these extended datasets, we
evaluated 22 models, including GPT3, following the methodology from Lialin et
al. (2022). Compared to the original test sets, we see a significant drop (up
to 58%) in accuracy for both Role and Negation tasks. At the same time most
models show higher sensitivity to negation compared to the original dataset.
Finally, while GPT3 has generated all of the examples in ROLE-1500, it is only
able to predict the correct answer in 24.6% cases (top-5 accuracy).
ALBERT-v2-xxlarge surpasses GPT3 by 4.6%.

##  2 Related Work

Recent research has shown that large language models are capable of generating
labeled data Anaby-Tavor et al. (2020); Papanikolaou and Pierleoni (2020);
Kumar et al. (2020); Meng et al. (2022); Mohapatra et al. (2020); Yang et al.
(2020). Most previous studies used GPT or GPT2 for dataset extension Schick
and Schütze (2021); Gao et al. (2022); Whitfield (2021); Anaby-Tavor et al.
(2019). Smith et al. (2022) attempted to generate labels for the unlabelled
dataset using GPT3. In our work, we generate the entire dataset, rather than
just the labels. Bonifacio et al. (2022) used GPT3 to generate questions,
sampling from 100k documents to create the prompts. Han et al. (2021) and Liu
et al. (2022) prompted GPT3 to generate synthetic translation and NLI
datasets, respectively.

Lialin et al. (2022) and Ettinger (2020) evaluated language models on smaller
datasets for negation and role reversal. We extend these datasets to around
1500 data points and evaluate 22 models, including GPT3. To our knowledge, our
work is the first to extend psycholinguistic datasets and use it to evaluate
an extended set of models.

##  3 Data generation

###  3.1 Negation Dataset: NEG-1500-SIMP

The existing negation dataset NEG-136-SIMP from Ettinger (2020) consists of 18
pairs of sentences. Each pair is made of one affirmative and one negated
sentence e.g. the sentences “A robin is a bird” (affirmative) and “A robin is
not a tree” (negated) form a pair. We extend this dataset using two methods:
template-based and generation-based. We create 770 sentence pairs with the
template-based method and generate 750 sentence pairs using GPT3. We will
refer to these datasets as NEG-1500-SIMP-TEMP and NEG-1500-SIMP-GEN,
respectively.

#### NEG-1500-SIMP-TEMP

Each pair in the original dataset follows a template. For affirmative
sentences, the template is “ _A/An{subject} is a/an {object}_”. Its negated
version is “ _A/An{subject} is not a/an {another object}_”. Battig and
Montague (1969) and Fischler et al. (1983) provide a template and a list of
objects (e.g., robin, pine) and the corresponding categories (e.g., bird,
tree) which are used to generate samples 333Fischler et al. (1983) refers to
these as “subjects” and ”objects”, respectively.. For instance, the category,
“bird” in the example, “A robin is a bird” is replaced with another category
(“tree”) to generate the negated example “A robin is not a tree”.

We experimented with using WordNet to generate from this template, but found
that it required a careful curation of hypernym hierarchies. Since our goal
was to reduce human effort and cost, we decided against it. We used a template
from Fischler et al. (1983) along with all the categories and subcategories
listed in Battig and Montague (1969) to create 1540 sentences (770 sentence
pairs). Similar to Ettinger (2020), we did not use multi-word category names.

#### NEG-1500-SIMP-GEN

We use human-in-the-loop and few-shot prompted GPT3 to generate 750 pairs
(1500 sentences). Each GPT3 prompt consists of an instruction and four
randomly selected in-context examples. We used GPT3-text-davinci-002 with
temperature 1, maximum tokens 150 and Top P set to 1. The cost of generating
the dataset is around $30. An example of the prompt is shown in the appendix
A. After generating samples from GPT3, the dataset was manually cleaned to
filter out poorly generated examples. Details about manual filtering in
mentioned in Appendix B.1.

Appendix D shows top 20 categories in the new negation datasets.
NEG-1500-SIMP-GEN has almost three times more examples in the “animal”
category than “tree” or “drink”. The distribution is more skewed compared to
the template-based dataset. All the categories from the original dataset
NEG-136-SIMP are represented in the top 20 (by frequency) categories in
NEG-1500-SIMP-GEN (except “insect”). However, only four out of nine categories
from the original data are present in the top 20 most frequent categories in
the template-generated NEG-1500-SIMP-TEMP.

###  3.2 Role-reversal Dataset: ROLE-1500

The original ROLE-88 dataset Ettinger (2020); Chow et al. (2016) is a role
reversal dataset consisting 88 sentences (44 sentence pairs). Here is an
example of a sentence pair: _“The librarian documented which journalist the
celebrities had avoided.” “The librarian documented which celebrities the
journalist had interviewed”_. The first sentence has two words representing
roles (here: journalist, celebrities), which are reversed in the second
sentence. The target verb (the last verb in the sentence) is always preceded
by the auxiliary “had”. This dataset was created to observe the effect of role
reversal on the target verb. We notice that all of the role words represent
human roles (chef, worker, etc.) or animals (whale, dog, etc.). There were
some pairs where the role reversal didn’t change the target word, but they
were semantically correct.

To extend ROLE dataset, we used the same method as NEG-1500-SIMP-GEN and
generated 1500 sentences with temperature set to 0.64. We call this new
dataset as ROLE-1500 and cost of generating it is $25. An example of the
prompt is shown in the appendix A. After generating samples from GPT3, the
dataset was manually cleaned. Details about manual filtering in mentioned in
Appendix B.2. Appendix D shows top 20 target verbs in ROLE-88 and ROLE-1500.
Except four, all of the top 20 verbs are common in both datasets.

##  4 Evaluation on Extended Datasets

Following methodology from Lialin et al. (2022), we evaluated 22 models on the
newly created negation and role reversal data (Figure 1). Both negation and
role reversal tasks were first converted into a masked language modeling task,
where the the category or the final verb was replaced with a mask token. For
GPT2 and GPT3, the task was converted into a causal language modeling task,
where the target was removed, and the model had to predict the target word.
The responses of the models were evaluated against the true target.

For GPT3, we also performed zero-shot evaluation with two prompts: with and
without instructions pre-pended to the input sentence (see Appendix A). We
used the GPT3-text-davinci-002 model via OpenAI API with the same parameters
as NEG-1500-SIMP-GEN. The temperature was set at 0.5, max_tokens at 1 and
logprobs at 5. GPT3 response was evaluated against gold label.

|  ROLE-88 |  NEG-136 SIMP(Aff) |  NEG-136 Sensitivity |  ROLE- 1500 |  NEG-1500 SIMP TEMP(Aff) |  NEG-1500 SIMP GEN(Aff) |  NEG-1500 SIMP-TEMP Sensitivity |  NEG-1500 SIMP-GEN Sensitivity
---|---|---|---|---|---|---|---|---
$\mathrm{BERT}_{\mathrm{base}}$ | 27.3 | 100.0 | 16.7 | 20.3 | 58.4 | 55.3 | 53.5 | 35.9
$\mathrm{BERT}_{\mathrm{large}}$ | 37.5 | 100.0 | 33.3 | 21.5 | 65.1 | 53.8 | 53.5 | 40.3
$\mathrm{RoBERTa}_{\mathrm{base}}$ | 46.6 | 94.4 | 66.7 | 23.0 | 62.1 | 44.0 | 64.4 | 71.5
$\mathrm{RoBERTa}_{\mathrm{large}}$ | 55.7 | 94.4 | 50.0 | 26.1 | 63.4 | 53.7 | 64.5 | 69.5
$\mathrm{DistilBERT}_{\mathrm{base}}$ | 28.4 | 94.4 | 27.8 | 19.3 | 57.3 | 52.8 | 44.7 | 41.5
$\mathrm{AlBERTv2}_{\mathrm{base}}$ | 26.1 | 33.3 | 38.9 | 15.3 | 10.0 | 11.7 | 37.3 | 35.9
$\mathrm{AlBERTv2}_{\mathrm{xlarge}}$ | 37.5 | 94.4 | 27.8 | 21.1 | 40.4 | 47.8 | 52.7 | 51.1
$\mathrm{AlBERTv2}_{\mathrm{xxlarge}}$ | 50.0 | 100.0 | 44.4 | 29.0 | 42.9 | 45.3 | 55.7 | 61.5
$\mathrm{T5}_{\mathrm{large}}$ | 36.4 | 94.4 | 50.0 | 18.2 | 60.4 | 49.8 | 44.0 | 0.0
$\mathrm{T5}_{\mathrm{xl}}$ | 44.3 | 83.3 | 66.7 | 21.5 | 60.9 | 60.9 | 68.2 | 70.0
$\mathrm{GPT2}_{\mathrm{base}}$ | 0.0 | 0.0 | 66.7 | 11.2 | 48.3 | 37.7 | 60.4 | 56.4
$\mathrm{GPT2}_{\mathrm{xl}}$ | 0.0 | 0.0 | 44.4 | 18.8 | 63.6 | 52.8 | 59.0 | 71.9
$\mathrm{GPT3}$ | 44.4 | 94.4 | 100.0 | 24.6 | 65.9 | 63.3 | 100.0 | 100.0
$\mathrm{GPT3}_{\mathrm{prompt}}$ | 38.6 | 72.2 | 100.0 | 24.4 | 55.9 | 55.2 | 100.0 | 100.0
Table 1: Zero-shot top-5 word prediction accuracy and sensitivity (top-5 over
the whole vocabulary). ROLE-88 and NEG-136 are from Lialin et al. (2022).
ROLE-1500 and NEG-1500 are the new datasets. The best result for each task is
in bold. “SIMP” stands for simple, “prompt” stands for prompting(adding
instruction). The negation task is evaluated in the affirmative form (Aff).
Sensitivity is the percentage of sentence pairs for which the top-1 prediction
changed. See Appendix E for accuracy and sensitivity of all models on extended
datasets.

Our evaluation metric is the top-5 prediction accuracy, following Ettinger
(2020). It is the percentage of the model responses where the gold label is
among the top five predictions from the model. Table 1 shows the results for
the original and extended datasets. We also included top-1, top-10, and top-20
accuracy predictions in Appendix F. For the negation dataset, we used model
sensitivity as an additional metric. Since we didn’t have N400 amplitude and
cloze probability for the extended datasets, we defined sensitivity as the
percentage of sentence pairs for which the top-1 prediction changed, when
“not” was added. Table 1 shows the sensitivity result. Results for all models
are shown in Appendix E. We could not compute sensitivity for ROLE-1500, since
in some sentence pairs the target word was the same.

##  5 Results

#### Models performance dropped substantially on extended datasets.

Model performance decreased by 40-50% on negation (affirmative-only) and by
20-30% on role reversal for most of the models. Even top-20 prediction
accuracy couldn’t reach the performance shown for the original datasets. For
NEG-1500-SIMP-TEMP, BERT, RoBERTa, DistilBERT and T5 performance decreased
around 30% each, while ALBERT-xxlarge-v2 had the biggest drop from 100% to
43%. Compared to NEG-1500-SIMP-TEMP, NEG-1500-SIMP-GEN shows 5-10% more
(absolute) drop in the performance. In ROLE-1500, the performance of BERT,
DistilBERT and T5 decreased by about 5-15% each. RoBERTa and ALBERT showed the
highest drop of 23-29% on this task. Increasing dataset size had a larger
effect on model accuracy for negation (especially NEG-1500-SIMP-GEN), than for
the role-reversal task. The models that performed best on the small datasets
did not remain in the lead on the extended. For example, in ROLE, Roberta-
large was the best model on the original dataset, but ALBERT-v2-xxlarge
surpassed it on the extended one. For negation, BERT and ALBERT performed the
best originally, but GPT3 surpassed them when the datasets became larger.

#### GPT3 cannot solve, but can generate such examples.

While GPT3 is unable to reliably solve these tasks, it is perfectly capable of
picking up the pattern and generating valid training examples. ROLE-1500 was
generated by GPT3, and yet, RoBERTa-large, and ALBERT-xxlarge perform better
on it than GPT3. There is a gap of around 4.5% between best model and GPT3.
Changing the prompt by adding instructions didn’t change the performance on
ROLE-1500, but decreased the accuracy by 8-10% on negation. Sensitivity to
negation remains the same for GPT3 with and without instruction prompts.

#### GPT3 responses contains variety of mistakes.

Approximately 2/3 of the generated ROLE data had to be manually filtered out.
14.3% of responses were single sentences without a pair. For example, the
model generated _“the athlete reported which coach the player had injured,”_
but not its pair _“the athlete reported which player the coach had trained”_.
7.4% responses were semantically incorrect. For example, _“the camper reported
which girl the book had read”_. Such responses were either removed or edited
to match the format. Similar kind of errors had to be manually filtered out in
the negation data as well.

#### Sampling in-context examples from a smaller dataset leads to more
duplicate responses from GPT3.

There were 44 sentence pairs to sample from in the role reversal data and only
18 pairs in the negation data. 62.5% of negation data generated with GPT3 were
duplicate entries, compared to 32.5% for the role reversal data. To generate
the same number of sentence pairs, we generated twice as many responses for
negation than for role reversal, which led to twice the manual labor required
for filtering.

#### Models are more sensitive to negation for extended dataset.

All models except ALBERT-v2-base, T5-large, and GPT2-base show higher
sensitivity to negation on both extended negation datasets compared to the
original one. For NEG-1500-SIMP-TEMP, the sensitivity for BERT and ALBERT-xl
increased by 36.8% and 21-25%, respectively. For RoBERTa-base, T5-large and
GPT2, sensitivity to negation dropped by 2.3-6%. The sensitivity to negation
on NEG-1500-SIMP-GEN either increased or remained the same compared to
original data for all models, except ALBERT-v2-base, T5-large and GPT2-base.
For BERT and RoBERTa the sensitivity increased by almost 20%. Results
demonstrate high levels of negation sensitivity in models like BERT and
ALBERT, suggesting that previous findings might have been skewed due to
smaller test sets.

##  6 Conclusion

We provide a new resource that dramatically extends existing probing data for
negation and role reversal. The psycholinguistic datasets NEG-136-SIMP and
ROLE-88 are extended to 1500 data points. We evaluate 22 models using the new
data, observing that the absolute accuracy drops for most models on both
tasks. However, most models do show an increased sensitivity to negation.
Strikingly, as seen in the role reversal task, we show that GPT3 may be
capable of generating the data for a task while failing to show top
performance on it.

## Limitations

The current method did not include the N400 amplitude and the cloze
probability to generate samples and was more based on a large language model
picking up the pattern and generating samples. The other limitation was the
diversity of datapoints generated using existing small dataset. We observed
that there is a cap on how many datapoints we can generate with a limited
number of original samples used via in-context learning. The less samples we
have the more duplication we get in GPT3 responses. We also need to be careful
in generating samples, as GPT3 responses can be expensive, especially when
using more in-context examples.

## Ethics Statement

Using large closed-source language models to generate evaluation data, while
highly effective, should be approached with caution. First, neural networks of
hundreds of billions of parameters are computationally expensive to infer and
potential carbon dioxide emissions should be taken into consideration. Second,
while API access to such models allows to avoid many technical challenges,
especially having enough resources to inference large models, it comes with
limitations. For example, GPT3 can only return the logprops of at most five
top tokens over the vocabulary. Also, the API provides no control over the
random seed used for sampling reducing the potential reprehensibility of the
research. Finally, without the direct access to the model weights, there is no
guarantee that this version of the model will be always accessible to
researchers in the future and won’t be restricted, modified, deleted, or lost.

## References

  * Anaby-Tavor et al. (2019) Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2019.  Not enough data? deep learning to the rescue!
  * Anaby-Tavor et al. (2020) Ateret Anaby-Tavor, Boaz Carmeli, Esther Goldbraich, Amir Kantor, George Kour, Segev Shlomov, Naama Tepper, and Naama Zwerdling. 2020.  Do not have enough data? deep learning to the rescue! _Proceedings of the AAAI Conference on Artificial Intelligence_ , 34:7383–7390.
  * Battig and Montague (1969) William F. Battig and William Edward Montague. 1969.  Category norms of verbal items in 56 categories a replication and extension of the connecticut category norms.  _Journal of Experimental Psychology_ , 80:1–46.
  * Belinkov et al. (2020) Yonatan Belinkov, Sebastian Gehrmann, and Ellie Pavlick. 2020.  Interpretability and analysis in neural NLP.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts_ , pages 1–5, Online. Association for Computational Linguistics.
  * Black et al. (2021) Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Rose Biderman. 2021.  Gpt-neo: Large scale autoregressive language modeling with mesh-tensorflow.
  * Bonifacio et al. (2022) Luiz Bonifacio, Hugo Abonizio, Marzieh Fadaee, and Rodrigo Nogueira. 2022.  Inpars: Data augmentation for information retrieval using large language models.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, and et. al Subbiah. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems_ , volume 33, pages 1877–1901. Curran Associates, Inc.
  * Card et al. (2020) Dallas Card, Peter Henderson, Urvashi Khandelwal, Robin Jia, Kyle Mahowald, and Dan Jurafsky. 2020.  With little power comes great responsibility.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9263–9274, Online. Association for Computational Linguistics.
  * Chow et al. (2016) Wing-Yee Chow, Cybelle Smith, Ellen Lau, and Colin Phillips. 2016.  A “bag-of-arguments” mechanism for initial verb predictions.  _Language, Cognition and Neuroscience_ , 31(5):577–596.
  * Chowdhery et al. (2022) Aakanksha Chowdhery, Sharan Narang, and Jacob Devlin et. al. 2022.  Palm: Scaling language modeling with pathways.  _ArXiv_ , abs/2204.02311.
  * Ettinger (2020) Allyson Ettinger. 2020.  What bert is not: Lessons from a new suite of psycholinguistic diagnostics for language models.
  * Federmeier and Kutas (1999) Kara D. Federmeier and Marta Kutas. 1999.  A rose by any other name: Long-term memory structure and sentence processing.  _Journal of Memory and Language_ , 41:469–495.
  * Fischler et al. (1983) Ira Fischler, Paul A Bloom, Donald G Childers, Salim E Roucos, and Nathan W Perry Jr. 1983.  Brain potentials related to stages of sentence verification.  _Psychophysiology_ , 20(4):400–409.
  * Gao et al. (2022) Jiahui Gao, Renjie Pi, Yong Lin, Hang Xu, Jiacheng Ye, Zhiyong Wu, Xiaodan Liang, Zhenguo Li, and Lingpeng Kong. 2022.  Zerogen+: Self-guided high-quality data generation in efficient zero-shot learning.
  * Han et al. (2021) Jesse Michael Han, Igor Babuschkin, Harrison Edwards, Arvind Neelakantan, Tao Xu, Stanislas Polu, Alex Ray, Pranav Shyam, Aditya Ramesh, Alec Radford, and Ilya Sutskever. 2021.  Unsupervised neural machine translation with generative language models only.
  * Kobayashi et al. (2020) Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020.  Attention is not only a weight: Analyzing transformers with vector norms.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 7057–7075.
  * Kovaleva et al. (2019) Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019.  Revealing the dark secrets of bert.  _arXiv preprint arXiv:1908.08593_.
  * Kumar et al. (2020) Varun Kumar, Ashutosh Choudhary, and Eunah Cho. 2020.  Data augmentation using pre-trained transformer models.  _CoRR_ , abs/2003.02245.
  * Lialin et al. (2022) Vladislav Lialin, Kevin Zhao, Namrata Shivagunde, and Anna Rumshisky. 2022.  Life after bert: What do other muppets understand about language?  In _ACL_.
  * Liu et al. (2022) Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2022.  Wanli: Worker and ai collaboration for natural language inference dataset creation.
  * Liu et al. (2021) Leo Z. Liu, Yizhong Wang, Jungo Kasai, Hannaneh Hajishirzi, and Noah A. Smith. 2021\.  Probing across time: What does roberta know and when?  _ArXiv_ , abs/2104.07885.
  * Meng et al. (2022) Yu Meng, Jiaxin Huang, Yu Zhang, and Jiawei Han. 2022.  Generating training data with language models: Towards zero-shot language understanding.
  * Mohapatra et al. (2020) Biswesh Mohapatra, Gaurav Pandey, Danish Contractor, and Sachindra Joshi. 2020.  Simulated chats for task-oriented dialog: Learning to generate conversations from instructions.  _CoRR_ , abs/2010.10216.
  * Papanikolaou and Pierleoni (2020) Yannis Papanikolaou and Andrea Pierleoni. 2020.  Dare: Data augmented relation extraction with gpt-2.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.
  * Rogers et al. (2020) Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020.  A primer in bertology: What we know about how bert works.
  * Sanh et al. (2021) Victor Sanh, Albert Webson, and Colin Raffel et. al. 2021.  Multitask prompted training enables zero-shot task generalization.  _ArXiv_ , abs/2110.08207.
  * Schick and Schütze (2021) Timo Schick and Hinrich Schütze. 2021.  Generating datasets with pretrained language models.
  * Smith et al. (2022) Ryan Smith, Jason A. Fries, Braden Hancock, and Stephen H. Bach. 2022.  Language models in the loop: Incorporating prompting into weak supervision.
  * Srivastava et al. (2022) Aarohi Srivastava, Abhinav Rastogi, and Abhishek B Rao et. al. 2022.  Beyond the imitation game: Quantifying and extrapolating the capabilities of language models.  _ArXiv_ , abs/2206.04615.
  * Talmor et al. (2019) Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019.  olmpics–on what language model pre-training captures.  _arXiv preprint arXiv:1912.13283_.
  * Tenney et al. (2019) Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019.  Bert rediscovers the classical nlp pipeline.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 4593–4601.
  * Whitfield (2021) Dewayne Whitfield. 2021.  Using gpt-2 to create synthetic data to improve the prediction performance of nlp machine learning classification models.
  * Yang et al. (2020) Yiben Yang, Chaitanya Malaviya, Jared Fernandez, Swabha Swayamdipta, Ronan Le Bras, Ji-Ping Wang, Chandra Bhagavatula, Yejin Choi, and Doug Downey. 2020.  G-DAUG: generative data augmentation for commonsense reasoning.  _CoRR_ , abs/2004.11546.
  * Zagoury et al. (2021) Avishai Zagoury, Einat Minkov, Idan Szpektor, and William W. Cohen. 2021.  What’s the best place for an ai conference, vancouver or ______: Why completing comparative questions is difficult.  In _AAAI_.
  * Zhang et al. (2022) Susan Zhang, Stephen Roller, and Naman Goyal et. al. 2022.  Opt: Open pre-trained transformer language models.  _ArXiv_ , abs/2205.01068.
  * Zhang et al. (2020) Yian Zhang, Alex Warstadt, Haau-Sing Li, and Samuel R. Bowman. 2020.  When do you need billions of words of pretraining data? _CoRR_ , abs/2011.04946.

##  Appendix A GPT3 prompt

###  A.1 ROLE-1500 generation prompt

An example prompt for generating ROLE-1500 dataset is given below. The roles
are colored orange and blue, whereas the target word is highlighted in green:

_The task is to reverse the role in the sentences. Generate more sentence like
this:_

_The journalist investigated which athlete the team had recruited,_ _The
journalist investigated which team the athlete had joined,_

_The detective interviewed which witness the doctor had suspected, The
detective interviewed which doctor the witness had seen._

_The teacher lectured which student the class had ignored, The teacher
lectured which class the student had left,_

_The police reported which criminal the witness had described, The police
reported which witness the criminal had robbed,_

_The doctor treated which patient the nurse had examined, The doctor treated
which nurse the patient had injured,_

###  A.2 NEG-1500-SIMP-GEN generation prompt

An example prompt for generating NEG-1500-SIMP-GEN dataset is given below.

_“The task is to generate affirmative sentences and its negation. The object
of the sentence should be a hypernym of the subject of the sentence. Generate
more sentence pairs like these: ”_

_A robin is a bird, A robin is not a tree,_

_An oak is a tree, An oak is not a flower,_

_A carrot is a vegetable, A carrot is not a bird,_

_A hammer is a tool, A hammer is not a tree,_ ”

###  A.3 ROLE-1500 evaluation prompts with and without instructions

Prompt with instruction for ROLE-1500:

_“The goal is to complete the given sentence with one English word. Avoid
punctuation or a new line or space. The librarian documented which journalist
the celebrities had”_.

The prompt without instruction: _“The librarian documented which journalist
the celebrities had”_. The model predicts the next word in sentence.
Instruction is same for ROLE-1500 and NEG-1500-SIMP-GEN. Only the in-context
example changes.

##  Appendix B Manual cleaning of the datasets

###  B.1 NEG-1500-SIMP

For NEG-1500-SIMP-GEN, to get 1500 cleaned samples, we generated 12000
sentences. 87.5% of the responses were rejected as they were a) duplicates
(62.5%) b) empty lines (15%) c) only negated sentences (8.75%) d) others
(1.25%). Others include sentences like “A toy is a toy” and “A toy is an
object”, as well as objects with multiple words and semantically wrong
sentences (e.g., “A flower is a rose”, “A cloud is a weather”). The cost of
generating NEG-SIMP-1500-GEN was $35.

###  B.2 ROLE-1500

After generating GPT3 responses to these prompts, the samples were manually
cleaned to get 1500 sentences (750 sentence pairs). We generated 4700
sentences (including empty, partial and no-pair sentences), out of which 68%
were removed, as a) 32.5% were duplicates, b) 14.3% missed on getting a pair
or were simple sentences, c) 8.5% were empty lines, d) 7.4% were semantically
wrong sentences or had a wrong role reversal (e.g., “The camper reported which
book the girl had eaten”), e) 5.3% were repeating instructions or partial
sentences. In some instances, generated examples included some repetition of
the role nouns present in the in-context examples.

##  Appendix C Rules to create NEG-1500-SIMP-GEN

An affirmative sentence and a negation sentence make a pair. Affirmative
sentences follow the format of “subject is an object” whereas its negation
form is “subject is not an object”. Affirmative sentences can be, for example
_“Robin is a bird”_ or _“Robin is an animal”_ , the subject word is _“Robin”_
, the object word is _“bird”_ and its immediate superordinate category name is
_“animal”_. The negation for any of these two affirmative sentences can be
_“Robin is not a plant” or “Robin is not a tree”_. The target completion (
e.g. _“plant” or “tree”_) has to be an object or superordinate word from
another category. Figure 1 in the Appendix 1 shows two categories (Animal and
Plant) together with its respective subject and object words Battig and
Montague (1969).

Figure 1: Sentence subject word (Robin). Affirmative sentence completion are
in green - object word(Bird) and super ordinate category name(Animal).
Negative sentence completion are in red - object word(Tree) and super ordinate
category name(Plant). Battig and Montague (1969)

##  Appendix D Top 20 target words for all datasets

Figure 2 and 6 show top 20 object words for original datasets NEG-136-SIMP and
ROLE-88. Figure 3, 4 and 5 depict top 20 object words for original new
extended datasets. Y axis represents the number of times the target words.

Figure 2: Top 20 categories for NEG-136-SIMP Figure 3: Top 20 categories for
NEG-1500-SIMP-TEMP Figure 4: Top 20 categories for NEG-1500-SIMP-GEN Figure 5:
Top 20 categories for ROLE-1500 Figure 6: Top 20 target words for ROLE-88

##  Appendix E Result for all models

Table 2 shows the top five accuracy for all the models.

|  ROLE-88 |  NEG-136 SIMP(Aff) |  NEG-136 Sensitivity |  ROLE- 1500 |  NEG-1500 SIMP TEMP(Aff) |  NEG-1500 SIMP GEN(Aff) |  NEG-1500 SIMP-TEMP Sensitivity |  NEG-1500 SIMP-GEN Sensitivity
---|---|---|---|---|---|---|---|---
$\mathrm{BERT}_{\mathrm{base}}$ | 27.3 | 100.0 | 16.7 | 20.3 | 58.4 | 55.3 | 53.5 | 35.9
$\mathrm{BERT}_{\mathrm{large}}$ | 37.5 | 100.0 | 33.3 | 21.5 | 65.1 | 53.8 | 53.5 | 40.3
$\mathrm{RoBERTa}_{\mathrm{base}}$ | 46.6 | 94.4 | 66.7 | 23.0 | 62.1 | 44.0 | 64.4 | 71.5
$\mathrm{RoBERTa}_{\mathrm{large}}$ | 55.7 | 94.4 | 50.0 | 26.1 | 63.4 | 53.7 | 64.5 | 69.5
$\mathrm{DistilBERT}_{\mathrm{base}}$ | 28.4 | 94.4 | 27.8 | 19.3 | 57.3 | 52.8 | 44.7 | 41.5
$\mathrm{AlBERTv1}_{\mathrm{base}}$ | 17.1 | 72.2 | 22.2 | 10.4 | 37.1 | 36.4 | 40.0 | 35.6
$\mathrm{AlBERTv1}_{\mathrm{large}}$ | 26.1 | 83.3 | 22.2 | 17.4 | 48.4 | 42.4 | 38.6 | 32.9
$\mathrm{AlBERTv1}_{\mathrm{xlarge}}$ | 34.1 | 55.5 | 55.6 | 19.5 | 22.3 | 26.0 | 77.0 | 62.5
$\mathrm{AlBERTv1}_{\mathrm{xxlarge}}$ | 53.4 | 72.2 | 55.6 | 28.5 | 38.7 | 39.3 | 59.9 | 61.7
$\mathrm{AlBERTv2}_{\mathrm{base}}$ | 26.1 | 33.3 | 38.9 | 15.3 | 10.0 | 11.7 | 37.3 | 35.9
$\mathrm{AlBERTv2}_{\mathrm{large}}$ | 29.5 | 83.3 | 22.2 | 18.8 | 36.6 | 36.2 | 31.7 | 33.1
$\mathrm{AlBERTv2}_{\mathrm{xlarge}}$ | 37.5 | 94.4 | 27.8 | 21.1 | 40.4 | 47.8 | 52.7 | 51.1
$\mathrm{AlBERTv2}_{\mathrm{xxlarge}}$ | 50 | 100.0 | 44.4 | 29.0 | 42.9 | 45.3 | 55.7 | 61.5
$\mathrm{T5}_{\mathrm{small}}$ | 9.1 | 44.4 | 0.0 | 5.8 | 33.6 | 25.3 | 16.8 | 19.6
$\mathrm{T5}_{\mathrm{base}}$ | 27.3 | 88.9 | 27.8 | 14.8 | 58.2 | 45.1 | 46.5 | 33.5
$\mathrm{T5}_{\mathrm{large}}$ | 36.4 | 94.4 | 50.0 | 18.2 | 60.4 | 49.8 | 44.0 | 0.0
$\mathrm{T5}_{\mathrm{xl}}$ | 44.3 | 83.3 | 66.7 | 21.5 | 60.9 | 60.9 | 68.2 | 70.0
$\mathrm{GPT2}_{\mathrm{base}}$ | 0.0 | 0.0 | 66.7 | 11.2 | 48.3 | 37.7 | 60.4 | 56.4
$\mathrm{GPT2}_{\mathrm{medium}}$ | 0.0 | 0.0 | 50.0 | 16.4 | 61.4 | 46.9 | 53.1 | 60.7
$\mathrm{GPT2}_{\mathrm{large}}$ | 0.0 | 0.0 | 38.9 | 16.8 | 61.7 | 51.7 | 57.9 | 69.2
$\mathrm{GPT2}_{\mathrm{xl}}$ | 0.0 | 0.0 | 44.4 | 18.8 | 63.6 | 52.8 | 59.0 | 71.9
$\mathrm{GPT3}$ | 44.4 | 94.4 | 100.0 | 24.6 | 65.9 | 63.3 | 100.0 | 100.0
$\mathrm{GPT3}_{\mathrm{prompt}}$ | 38.6 | 72.2 | 100.0 | 24.4 | 55.9 | 55.2 | 100.0 | 100.0
Table 2: Zero-shot top-5 word prediction accuracy and sensitivity (top-5 over
the whole vocabulary). ROLE-88 and NEG-136 are from Lialin et al. (2022).
ROLE-1500 and NEG-1500 are the new datasets. The best result for each task is
in bold. “SIMP” stands for simple, “prompt” stands for prompting. The negation
task is evaluated in the affirmative form (Aff). Sensitivity is the percentage
of sentence pairs for which the top-1 prediction changed.

##  Appendix F Top 1, 10 and 20 prediction accuracy for all models

Table 4 and 3 show top 1, 10 and 20 accuracy for new negation datasets. Table
5 depict top 1, 10 and 20 accuracy for new role reversal dataset.

NEG-1500-SIMP-TEMP |  Top 1 |  Top 10 |  Top 20
---|---|---|---
$\mathrm{BERT}_{\mathrm{base}}$ | 28.5 | 64.7 | 71.4
$\mathrm{BERT}_{\mathrm{large}}$ | 34.2 | 71.9 | 76.5
$\mathrm{RoBERTa}_{\mathrm{base}}$ | 22.2 | 68.7 | 73.0
$\mathrm{RoBERTa}_{\mathrm{large}}$ | 30 | 69.2 | 74.4
$\mathrm{DistilBERT}_{\mathrm{base}}$ | 28.6 | 67.8 | 75.6
$\mathrm{AlBERTv1}_{\mathrm{base}}$ | 14.3 | 43.9 | 49.5
$\mathrm{AlBERTv1}_{\mathrm{large}}$ | 13.5 | 56.9 | 63.3
$\mathrm{AlBERTv1}_{\mathrm{xlarge}}$ | 10.3 | 28.0 | 32.6
$\mathrm{AlBERTv1}_{\mathrm{xxlarge}}$ | 12.1 | 48.6 | 57.3
$\mathrm{AlBERTv2}_{\mathrm{base}}$ | 2.0 | 18.8 | 29.2
$\mathrm{AlBERTv2}_{\mathrm{large}}$ | 10.6 | 46.8 | 55.6
$\mathrm{AlBERTv2}_{\mathrm{xlarge}}$ | 24.7 | 45.2 | 50.0
$\mathrm{AlBERTv2}_{\mathrm{xxlarge}}$ | 14.8 | 53.4 | 62.8
$\mathrm{T5}_{\mathrm{small}}$ | 1.7 | 43.6 | 51.6
$\mathrm{T5}_{\mathrm{base}}$ | 21.4 | 65.6 | 71.3
$\mathrm{T5}_{\mathrm{large}}$ | 22.7 | 68.3 | 73.8
$\mathrm{T5}_{\mathrm{xl}}$ | 39.7 | 66.2 | 69.9
$\mathrm{GPT2}_{\mathrm{base}}$ | 0.0 | 57.1 | 63.6
$\mathrm{GPT2}_{\mathrm{medium}}$ | 0.0 | 69.5 | 73.2
$\mathrm{GPT2}_{\mathrm{large}}$ | 0.0 | 70.0 | 75.7
$\mathrm{GPT2}_{\mathrm{xl}}$ | 0.0 | 70.1 | 75.6
$\mathrm{GPT3}$ | 27.5 | N/A | N/A
$\mathrm{GPT3}_{\mathrm{prompt}}$ | 21.8 | N/A | N/A
Table 3: Zero-shot top-1,10 and 20 word prediction accuracy on NEG-1500-SIMP-TEMP. Top-5 is selected over the whole model vocabulary. The best result on each task is highlighted in bold. SIMP stands for simple, TEMP stands for template. Negation tasks are evaluated in the affirmative form (aff). GPT3 only produces top 5 predictions.  NEG-1500-SIMP-GEN |  Top 1 |  Top 10 |  Top 20
---|---|---|---
$\mathrm{BERT}_{\mathrm{base}}$ | 18.8 | 66.0 | 73.3
$\mathrm{BERT}_{\mathrm{large}}$ | 22.8 | 64.3 | 72.0
$\mathrm{RoBERTa}_{\mathrm{base}}$ | 15.4 | 51.6 | 60.1
$\mathrm{RoBERTa}_{\mathrm{large}}$ | 24.4 | 61.6 | 71.7
$\mathrm{DistilBERT}_{\mathrm{base}}$ | 22.5 | 61.9 | 70.5
$\mathrm{AlBERTv1}_{\mathrm{base}}$ | 11.2 | 46.0 | 53.6
$\mathrm{AlBERTv1}_{\mathrm{large}}$ | 12.9 | 50.5 | 59.6
$\mathrm{AlBERTv1}_{\mathrm{xlarge}}$ | 7.2 | 34.4 | 41.6
$\mathrm{AlBERTv1}_{\mathrm{xxlarge}}$ | 12.4 | 50.7 | 60.7
$\mathrm{AlBERTv2}_{\mathrm{base}}$ | 2.5 | 22.2 | 32.3
$\mathrm{AlBERTv2}_{\mathrm{large}}$ | 10.7 | 44.8 | 54.7
$\mathrm{AlBERTv2}_{\mathrm{xlarge}}$ | 24.0 | 56.5 | 64.5
$\mathrm{AlBERTv2}_{\mathrm{xxlarge}}$ | 15.3 | 55.3 | 54.1
$\mathrm{T5}_{\mathrm{small}}$ | 2.9 | 36.3 | 45.6
$\mathrm{T5}_{\mathrm{base}}$ | 15.1 | 65.5 | 65.3
$\mathrm{T5}_{\mathrm{large}}$ | 15.1 | 60.5 | 68.3
$\mathrm{T5}_{\mathrm{xl}}$ | 39.7 | 66.2 | 69.9
$\mathrm{GPT2}_{\mathrm{base}}$ | 16.4 | 48.0 | 60.0
$\mathrm{GPT2}_{\mathrm{medium}}$ | 26.1 | 56.1 | 66.9
$\mathrm{GPT2}_{\mathrm{large}}$ | 29.3 | 60.7 | 69.6
$\mathrm{GPT2}_{\mathrm{xl}}$ | 30.4 | 60.1 | 70.4
$\mathrm{GPT3}$ | 26.5 | N/A | N/A
$\mathrm{GPT3}_{\mathrm{prompt}}$ | 24.3 | N/A | N/A
Table 4: Zero-shot top-1,10 and 20 word prediction accuracy on NEG-1500-SIMP-GEN. Top-5 is selected over the whole model vocabulary. The best result on each task is highlighted in bold. SIMP stands for simple, negation tasks is evaluated in the affirmative form. GPT3 allow only till top 5 predictions.  ROLE-1500 |  Top 1 |  Top 10 |  Top 20
---|---|---|---
$\mathrm{BERT}_{\mathrm{base}}$ | 6.7 | 28.1 | 38.9
$\mathrm{BERT}_{\mathrm{large}}$ | 9.1 | 30.9 | 40.7
$\mathrm{RoBERTa}_{\mathrm{base}}$ | 0 | 33.9 | 47.3
$\mathrm{RoBERTa}_{\mathrm{large}}$ | 0 | 38.6 | 52
$\mathrm{DistilBERT}_{\mathrm{base}}$ | 6.5 | 27.3 | 36.9
$\mathrm{AlBERTv1}_{\mathrm{base}}$ | 2.8 | 16.2 | 25.3
$\mathrm{AlBERTv1}_{\mathrm{large}}$ | 4.8 | 24.2 | 32.7
$\mathrm{AlBERTv1}_{\mathrm{xlarge}}$ | 7.2 | 27.3 | 35.5
$\mathrm{AlBERTv1}_{\mathrm{xxlarge}}$ | 12.2 | 37.2 | 45.1
$\mathrm{AlBERTv2}_{\mathrm{base}}$ | 6.2 | 22.7 | 29.9
$\mathrm{AlBERTv2}_{\mathrm{large}}$ | 5.9 | 27.0 | 36.1
$\mathrm{AlBERTv2}_{\mathrm{xlarge}}$ | 8.8 | 29.6 | 37.9
$\mathrm{AlBERTv2}_{\mathrm{xxlarge}}$ | 11.6 | 37.1 | 45.1
$\mathrm{T5}_{\mathrm{small}}$ | 1.3 | 12.1 | 18.3
$\mathrm{T5}_{\mathrm{base}}$ | 4.7 | 21.9 | 28.5
$\mathrm{T5}_{\mathrm{large}}$ | 6.2 | 25.3 | 31.3
$\mathrm{T5}_{\mathrm{xl}}$ | 7.9 | 29.0 | 36.5
$\mathrm{GPT2}_{\mathrm{base}}$ | 0 | 18.9 | 27.1
$\mathrm{GPT2}_{\mathrm{medium}}$ | 0 | 25.2 | 36.5
$\mathrm{GPT2}_{\mathrm{large}}$ | 0 | 27.0 | 37.7
$\mathrm{GPT2}_{\mathrm{xl}}$ | 0 | 29.9 | 38.8
$\mathrm{GPT3}$ | 7.7 | N/A | N/A
$\mathrm{GPT3}_{\mathrm{prompt}}$ | 5.7 | N/A | N/A
Table 5: Zero-shot top-1,10 and 20 word prediction accuracy on ROLE-1500.
Top-5 is selected over the whole model vocabulary. The best result on each
task is highlighted in bold. GPT3 produces only top 5 predictions.