# GPTEval: A Survey on Assessments of ChatGPT and GPT-4

Rui Mao♠, Guanyi Chen♣, Xulang Zhang♠, Frank Guerin♢ Erik Cambria♠
♠Nanyang Technological University, Singapore
♣Hubei Provincial Key Laboratory of Artificial Intelligence and Smart
Learning,
National Language Resources Monitoring and Research Center for Network Media,
School of Computer Science, Central China Normal University, China
♢The University of Surrey, United Kingdom
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The emergence of ChatGPT has generated much speculation in the press about its
potential to disrupt social and economic systems. Its astonishing language
ability has aroused strong curiosity among scholars about its performance in
different domains. There have been many studies evaluating the ability of
ChatGPT and GPT-4 in different tasks and disciplines. However, a comprehensive
review summarizing the collective assessment findings is lacking. The
objective of this survey is to thoroughly analyze prior assessments of ChatGPT
and GPT-4, focusing on its language and reasoning abilities, scientific
knowledge, and ethical considerations. Furthermore, an examination of the
existing evaluation methods is conducted, offering several recommendations for
future research in evaluating large language models.

##  1 Introduction

ChatGPT (OpenAI, 2023b) has generated significant scholarly interest across
various disciplines due to its impressive dialogue-based task-processing
capabilities. This has enabled users to explore and evaluate its performance
across a wide range of tasks and disciplines, thereby sparking considerable
enthusiasm in the field of Artificial Intelligence (AI). While many
researchers have concentrated on evaluating ChatGPT and GPT-4 (OpenAI, 2023a)
within their specific domains of expertise, a comprehensive review
encompassing the assessments in multiple tasks and disciplines can offer a
holistic understanding of the strengths and limitations of these GPT models
(see Appendix A).

We focus on ChatGPT and GPT-4, because many evaluation reports have come to
similar conclusions – ChatGPT and GPT-4 are the state-of-the-art (SOTA) large
language models (LLMs) by now. The scope of our survey encompasses
quantitative evaluations carried out on ChatGPT or GPT-4, specifically
focusing on their language proficiency, scientific knowledge, and ethical
considerations.

Our main findings are summarized as follows:
a) ChatGPT and GPT-4 are strong in language understanding and generation,
adeptly engaging in user interactions through dialogues, enabling them to
tackle diverse NLP tasks and provide explanatory outputs. However, their
current status falls short of being a comprehensive AI, as their performance
lags behind expert models in numerous domains involving domain-specific
knowledge.

b) ChatGPT performs satisfactorily in general science knowledge and can answer
science questions that desire open responses. However, it can also make
mistakes, especially for questions that require multi-step reasoning. The
exceptional language proficiency poses challenges for users in assessing the
accuracy of factual information, giving rise to a range of ethical concerns.

c) Existing evaluation methods may be unreliable. The current evaluation
methods heavily depend on prompt engineering and benchmark datasets. Varying
prompts can yield disparate evaluation results. Additionally, the comparison
of expert systems often relies on (in-domain) datasets that were utilized for
training those systems. It remains uncertain whether the examined data, such
as public datasets and scientific knowledge, have been inadvertently exposed
during the training of ChatGPT and GPT-4. These factors may contribute to an
unfair comparison between LLMs and their respective baselines.

The contributions of this work are threefold: (1) We conduct a comprehensive
survey of recent assessments focusing on the language proficiency and
scientific knowledge of ChatGPT and GPT-4. (2) We compare their assessment
results across various tasks and disciplines to highlight the strengths and
weaknesses of the GPT models. (3) We critically analyze the existing
assessment methods employed, offering recommendations for future evaluation
studies and delivering our ethical considerations associated with the GPT
models.

##  2 Language and Reasoning Ability

###  2.1 Classic NLP Tasks

#### Dialogue.

Cabrera and Neubig (2023) compared several chatbots with a novel LLM
evaluation toolkit, Zeno Build. The performance was evaluated by Critique
metrics111https://docs.inspiredco.ai/critique/ such as ChrF (a character- and
word n-grams similarity-based metric, Popović, 2015), BERTScore (a BERT
embedding similarity-based metric, Zhang et al., 2019), and UniEval Coherence
(a coherence probability-based metric, Zhong et al., 2022). They found that
ChatGPT surpassed all the baselines, e.g., GPT-2 (Radford et al., 2019), LLaMa
(Touvron et al., 2023), Alpaca (Taori et al., 2023), Vicuna (Chiang et al.,
2023), MPT-Chat (MosaicML, 2023), and Cohere
Command222https://docs.cohere.com/docs/command-beta in the three evaluation
metrics. However, Cabrera and Neubig (2023) also highlighted that ChatGPT
exhibited vulnerabilities that were noticeable in various aspects, such as
occurrence of hallucinations, inadequate exploration for additional
information, and repetition of content. Although ChatGPT exceeded other LLMs,
Bang et al. (2023) observed that SOTA models still outperformed ChatGPT on
task-oriented dialogue, and open-domain knowledge-grounded dialogue, based on
automatic evaluation metrics.

#### Generation

was almost only evaluated on text-to-text generation tasks (but not data-to-
text). For machine translation (MT), an early assessment suggested SOTA MT
systems could defeat ChatGPT by a large margin (Bang et al., 2023). Later on,
by testing on more language pairs, more datasets, and better prompts, Hendy et
al. (2023) found ChatGPT yielded competitive performance for high-resource
languages, but still had limited capabilities for low-recourse languages. This
was further validated by Jiao et al. (2023), where the authors additionally
concluded that ChatGPT was not good at translating between distant languages
and translating sentences that contain domain biases or noise, compared to
commercial MT systems (e.g., DeepL and Google Translate), but the gap could be
narrowed for GPT-4. However, through a large-scale human evaluation and error
analysis by expert translators, Karpinska and Iyyer (2023) had very different
conclusions. They found that, when doing paragraph-level translation,
ChatGPT’s translations were overwhelmingly preferred compared to those from
Google Translate. ChatGPT could largely reduce errors, including
mistranslation, grammatical errors, inconsistency errors, and more. For
summarization, by testing on multiple summarization datasets in multiple
languages, Qin et al. (2023); Bang et al. (2023); Lai et al. (2023); Zhang et
al. (2023a) observed that ChatGPT largely underperformed SOTA systems in doing
either abstractive or extractive summarization. Interestingly, Zhang et al.
(2023a) found that its performance, especially the faithfulness, improved if
we asked ChatGPT to first extract salient sentences and then generate the
summarization, based on the extracted sentences, although it still lost to
SOTA. However, similar to what happened when assessing ChatGPT’s translation
ability, in a small human evaluation (i.e., 50 summaries rated by 2
annotators), annotators could not distinguish summaries generated by ChatGPT
from those by human beings (Soni and Wade, 2023). This conclusion should be
further confirmed by larger populations and rigorous experiments.

Although ChatGPT is often considered to perform well in generation tasks that
need creativity, there have not been many related assessments. Jentzsch and
Kersting (2023) asked ChatGPT to produce jokes and suggested that while
ChatGPT generated jokes, it struggled to produce “new” jokes. Approximately
90% of the generated jokes were repetitions of the same 25 jokes. Chu and Liu
(2023) conducted reader experiments and found that ChatGPT wrote more engaging
and persuasive short stories than its human counterparts. However, the
conclusion was the opposite if the aim was long stories. They suggested that
ChatGPT might focus on retaining information from the instruction when writing
a long story, which limited its flexibility.

#### Affective Computing.

Amin et al. (2023) compared ChatGPT to naive machine learning models on
Personality Detection, Sentiment Analysis, and Suicide Detection tasks. The
baselines were trained with task-specific datasets using RoBERTa embeddings,
Word2Vec, and Bag-of-Words (BoW) vector representations. ChatGPT yielded
better results on Sentiment Analysis, while lacking significant improvements,
compared to a basic BoW model. The baselines outperformed ChatGPT on the other
two examined tasks. Qin et al. (2023); Bang et al. (2023) found that fine-
tuned models exceeded ChatGPT on English sentiment analysis tasks. Kocoń et
al. (2023) also found that ChatGPT was weaker in emotion detection than other
SOTA models.

#### Information Retrieval.

Wei et al. (2023) examined ChatGPT on relation extraction, named entity
recognition (NER), and event extraction tasks, showing that the performance of
the basic version of ChatGPT was much weaker than supervised methods, while
largely exceeding 50 shot or fewer shot baselines. Then, they introduced a
multi-turn question answering (QA) framework, where the modified querying
process did not help ChatGPT to exceed the supervised baseline. Qin et al.
(2023) observed that ChatGPT yielded much weaker performance than fine-tuning-
based models on CoNLL03 NER dataset (Sang and De Meulder, 2003). Sun et al.
(2023) evaluated ChatGPT and GPT-4 on passage re-ranking tasks, showing that
GPT-4 outperformed SOTA supervised baselines across all three datasets,
including a multilingual dataset with 10 languages. ChatGPT slightly fell
behind SOTA methods, while largely exceeding BM25. Bubeck et al. (2023) found
that GPT-4 (77.4% accuracy) significantly outperformed the SOTA model (40.8%,
Payne, 2020) on a personally identifiable information detection task.

#### GPT as Human Annotator.

Wang et al. (2023c) compared ChatGPT with SOTA natural language generation
evaluation metrics, e.g., BERTScore, BARTScore (Yuan et al., 2021), ROUGE
(Lin, 2004), and more. ChatGPT slightly outperformed the strongest BARTScore-
based setup on the SummEval dataset (Fabbri et al., 2021) in coherence,
relevance, consistency, and fluency dimensions, while was surpassed by
BARTScore on the NewsRoom dataset (Grusky et al., 2018). ROUGE-1 yielded the
highest correlation scores on the sample- and dataset-level evaluations on the
RealSumm dataset (Bhandari et al., 2020). The above three datasets were used
for evaluating text summarization, although presenting inconsistent results.
ChatGPT achieved the largest improvements on the OpenMEVA-ROC (Guan et al.,
2021) story generation dataset. ChatGPT and baselines were comparable on the
BAGEL dataset (Mairesse et al., 2010) for informativeness, naturalness, and
quality evaluations. Liu et al. (2023c) found that GPT-4 could exceed other
metrics on the dialogue generation dataset from Mehri and Eskenazi (2020).
Kocmi and Federmann (2023) evaluated ChatGPT-generated machine translation
accuracy scores with its former versions and other SOTA scoring systems,
finding that ChatGPT was less accurate than those baselines. Gilardi et al.
(2023) observed that ChatGPT achieved higher intercoder agreements than MTurk
crowd-workers and educated annotators, and maintained the highest annotation
accuracy in tweet frame and stance annotation tasks. The relevance annotation
accuracy of ChatGPT was comparable to humans, while topic annotations were not
as useful as MTurk.

###  2.2 Multilingualism

#### Chinese Linguistic Test.

SuperCLUE benchmarks (Xu et al., 2023) compared ChatGPT-like foundation models
in regard to their basic language ability, professional ability, and Chinese-
featured ability. By 18th May 2023, they reported that GPT-4 and ChatGPT
achieved the second (76.67) and third best (66.18) results after humans
(96.50), exceeding other LLMs. GPT-4 and ChatGPT had better basic language
ability than the other two metrics in Chinese. Both models achieved human-like
accuracy on role-playing, chit-chatting, and coding. However, their ability to
understand Chinese poetry, literature, classical Chinese, and couplets was far
inferior to that of humans. Huang et al. (2023) also ranked GPT-4 and ChatGPT
as top-2 on a multi-discipline (52 subjects) Chinese evaluation.

#### Multilingual NLP Tasks

were examined by Lai et al. (2023), e.g., multilingual part-of-speech (PoS)
tagging, NER, relation classification, NLI, QA, commonsense reasoning, and
summarization. The researchers analyzed 36 languages and discovered that task-
specific fine-tuned models outperformed ChatGPT in the majority of examined
tasks, except PoS tagging. ChatGPT exhibited superior performance in English
tasks compared to tasks in other languages; for low- and extremely low-
resource languages, ChatGPT performed significantly worse than baselines.
Noticeably, despite the use of non-English languages in the target tasks,
ChatGPT improved its performance with English prompts. Wei et al. (2023) found
that direct usage of ChatGPT could not yield satisfying results in Chinese
information extraction. Wang et al. (2023b) tested ChatGPT and GPT-4 on
English-to-Chinese and English-to-German summarization, showing that although
ChatGPT and GPT-4 exceeded other LLM baselines on a zero-shot setup, they fell
behind a fine-tuned mBART-50 (Tang et al., 2021) on most of the examined
datasets. Bang et al. (2023) argued that ChatGPT generally yielded weak
performance on low-resource languages in language understanding and generation
, while achieving higher proficiency in comprehending non-Latin scripts
compared to its proficiency in generating them.

###  2.3 Reasoning

#### Logical Reasoning

was tested by Bang et al. (2023). They found 56 out of 60 answers correct
(with appropriate prompts) for deductive reasoning, i.e. applying general
rules to specific situations or cases. This was stronger than other types of
reasoning. 26 out of 30 were scored for abductive reasoning, i.e. forming
plausible explanations or hypotheses, based on limited evidence or incomplete
information. 33 out of 60 were scored for inductive reasoning, i.e., drawing
generalized conclusions from examples or specific observations.

#### Commonsense Reasoning.

Bang et al. (2023) tested ChatGPT via three commonsense datasets, showing that
80 out of 90 of ChatGPT’s predictions were correct. ChatGPT was able to give
good explanations of the reasoning steps to support its answer. However, Qin
et al. (2023); Laskar et al. (2023) showed that the commonsense reasoning
accuracy of ChatGPT was lower than fine-tuned baselines by a large margin.
Davis (2023) found significant flaws in common benchmarks for common sense,
including the CommonsenseQA dataset used by Bang et al. (2023), which he
explicitly addressed. Davis (2023) listed several examples of commonsense and
particularly physical reasoning failures that had been found shortly after the
release of ChatGPT, and pointed to others. However, there does not exist a
thorough assessment of the GPT models’ commonsense reasoning ability. More
generally, Davis (2023) pointed out that “many important aspects of
commonsense reasoning and commonsense knowledge are not tested in any existing
benchmark”. Bubeck et al. (2023) probed a small number of their own real-world
physical reasoning tasks with GPT-4, finding that it had good knowledge. They
concluded that the model was able to learn an understanding of the real-world
environment through the medium of text.

#### Causal Reasoning.

Bang et al. (2023) found that 24 out of 30 causes or effects could be
correctly identified. Gao et al. (2023) systematically evaluated event
causality identification, causal discovery, and causal explanation generation.
Compared to SOTA models, ChatGPT and GPT-4 yielded lower scores in causality
identification. They outperformed baseline models on the causal discovery,
although the compared models, e.g., BERT- (Devlin et al., 2019) and RoBERTa-
base Liu et al. (2019) were relatively weak. The generation of causal
explanations yielded inconsistent findings in terms of AVG-BLEU and ROUGE-l
metrics, while the human evaluation affirmed that both GPT models attained a
level of accuracy comparable to that of human performance. Kıcıman et al.
(2023) examined ChatGPT and GPT-4 on the causal discovery, counterfactual
reasoning, and actual causality inferring, finding that they outperformed
other LLMs and SOTA models largely on the first two tasks.

#### Psychological Reasoning

is the ability of humans to reason about other’s unobservable mental states
(a.k.a Theory of Mind (ToM)). Kosinski (2023) and Moghaddam and Honey (2023)
designed sets of False-Belief questions and quantified results suggested that
both ChatGPT and GPT-4 had ToM ability, but that was still inferior to a
human’s. However, Marcus and Davis (2023) pointed out flaws in the Kosinski
(2023) study because the test material was in the training data. Holterman and
van Deemter (2023) further tested GPT-3 and GPT-4 on more ToM tasks summarized
in Kahneman (2000). They acknowledged the potential problem of the test
material being in the training data, so they substituted various nouns in the
scenario. However, this is unlikely to be adequate to stop a neural model from
generalising from those examples. Borji (2023) found that chatGPT failed on a
variant of a classic ‘Sally-Anne Test’ (used to test children). Bubeck et al.
(2023) found that chatGPT answered correctly on a similar variant Sally-Anne,
and they further tested on a range of more advanced ToM scenarios, with
probing questions, e.g. to infer the counterfactual impact of actions on
mental states. They found that GPT-4 had superior abilities and suggested that
GPT-4 had a very advanced level of ToM.

#### Task-Oriented Reasoning.

Qin et al. (2023) evaluated dialogue, logical reasoning, complex yes/no QA,
symbolic reasoning (last letter concatenation and coin flip), date
understanding-, and tracking shuffled objects-oriented logical reasoning.
However, ChatGPT underperformed fine-tuned baselines on most of the tasks,
excluding the logical reasoning tasks. To ascertain whether ChatGPT relies on
profound comprehension of truth and logic in their reasoning or merely
exploits shallow memorized patterns, Wang et al. (2023a) proposed a
dialectical evaluation task, finding that despite displaying high confidence,
ChatGPT demonstrated an inability to hold its belief in the truth in a wide
range of reasoning tasks, e.g., mathematics, first-order logic, commonsense,
and generic reasoning.

#### Natural Language Inference (NLI)

aims to examine if a statement can be inferred, contradicted, or neutral,
compared to another statement. Liu et al. (2023b) compared ChatGPT and GPT-4
with RoBERTa. However, both models encountered difficulties when dealing with
novel and out-of-distribution data. They yielded relatively modest performance
on NLI that needed logical reasoning. Qin et al. (2023) also proved that the
NLI ability of ChatGPT was lower than that of supervised models. Ambiguity is
one of the difficulties of NLI. For example, whether “ _John and Anna are not
a couple_ ” contradicts “ _John and Anna are married_ ” depends on whether “
_married_ ” means “ _both married_ ” or “ _married to each other_ ”. Given an
ambiguous NLI premise, Liu et al. (2023a) asked ChatGPT and GPT-4 to either
generate disambiguations of a premise with respect to the hypothesis or
recognize disambiguation (i.e., deciding whether the disambiguation is an
interpretation of an ambiguous premise). Their human evaluation showed that
for the first task, GPT-4 achieved correctness at 32%, while, for the second
task it was at the level of random guessing, suggesting that resolving tricky
ambiguity remained challenging for ChatGPT.

##  3 Scientific Knowledge

###  3.1 Formal Science

#### Mathematics.

Frieder et al. (2023) proposed a new mathematical benchmark dataset at the
graduate level to test ChatGPT’s mathematical reasoning, related to textbook
exercises, Olympiad problems, proof completion, algebra and probability theory
problem solving, and theorem-proof and definition understanding. They found
that ChatGPT only achieved a passing grade (50% of points) on 6 out of 17
testing sets. ChatGPT performed badly on mathematical problem solving, e.g.,
Olympiad problems, textbook exercises, algebra, and probability theory, while
presenting comparatively better grades in definition understanding and proof
completion. It seems ChatGPT is not good at solving mathematical problems that
are weakly related to language memory or generation, because compared to the
textual understanding, e.g., theorem-proofs and definitions, the other takes
depend on multi-step reasoning. Qin et al. (2023) evaluated ChatGPT in
arithmetic reasoning, finding that ChatGPT achieved the highest score on 3 out
of 6 datasets, while fine-tuning-based methods exceeded ChatGPT on the rest of
the 3 datasets. Bubeck et al. (2023) showed that GPT-4 largely exceeded the
Minerva expert model (Lewkowycz et al., 2022), while ChatGPT fell behind
Minerva on average.

#### Computer Science.

Bordt and von Luxburg (2023) used an exam with 10 different exercises. ChatGPT
and GPT-4 achieved 20.5 and 24 points out of 40 full points. GPT-4 slightly
exceeded the average score (23.9) of 200 students. Bordt and von Luxburg
(2023) believed that passing the exam by ChatGPT should not be misconstrued as
an indication of its comprehension of computer science, because numerous
topics addressed in the exam were extensively documented and readily available
online. The coding skills of ChatGPT and GPT-4 largely exceeded the SOTA
baselines (Bubeck et al., 2023). GPT-4 even achieved human-level accuracy in
LeetCode tests, while ChatGPT yielded lower accuracy, reaching only half of
the average human performance. Li et al. (2023b) compared in-context learning
(ICL)-based ChatGPT to SOTA models on Text-to-SQL tasks. To further improve
the performance of the examined ICL models, Chain-Of-Thought (COT) and extra
knowledge evidence sentences were also incorporated. ChatGPT (40.08%) exceeded
the strongest baseline Codex (36.47%, Chen et al., 2021), while largely
lagging behind humans (92.96%) in execution accuracy.

###  3.2 Natural Science

#### Physics.

ChatGPT was evaluated as if it is a college student who needs to finish
homework, clicker questions, programming exercises, and exams in the first-
year Calculus-based Physics (Kortemeyer, 2023). Overall, ChatGPT achieved
53.05% after weighing different testing modules. This score met the minimum
requirement for course credit, yet it adversely affected the overall grade-
point average, falling below the necessary threshold for graduation. ChatGPT
showed outstanding performance in clicker and programming questions, achieving
scores higher than 90%. However, its performance in homework and exams was
subpar. Additionally, ChatGPT’s mathematical difficulties in the field of
physics lowered its overall score.

#### Chemistry.

Clark (2023) asked ChatGPT to finish two real chemistry exams with closed- and
open-response questions. 44% closed-response questions were correctly answered
by ChatGPT, although this is lower than the student average score (69%).
Conversely, when it came to open-response questions, ChatGPT’s performance was
even lower than that of the least successful student.

#### Medicine.

Gilson et al. (2023); Kung et al. (2023) showed that ChatGPT achieved college
student level on the United States Medical Licensing Examination (USMLE).
Antaki et al. (2023) found ChatGPT yielded low accuracy on neuro-ophthalmology
and high accuracy on general medicine, indicating it had not grasped
specialized medical knowledge well. Hirosawa et al. (2023) examined ChatGPT’s
diagnosis with 30 clinical vignettes, showing that the rate of top-10
differential suggestions covering the correct diagnosis reached 93.3%, while
the accuracy of top-1 suggestions was just 53.3% (physicians were 93.3%). Rao
et al. (2023) suggested that ChatGPT achieved an impressive final diagnosis
accuracy (76.9%), while its initial diagnosis accuracy was just 60.3%.

###  3.3 Social Science

#### Education.

Dahlkemper et al. (2023) aimed to investigate the extent to which students
possessed accurate perception regarding the scientific accuracy and the
linguistic quality of ChatGPT’s responses. They provided 102 physics students
with ChatGPT answers and (masked) expert answers to evaluate the scientific
accuracy and linguistic quality of both parties, perceived by the students,
based on three physics questions. Despite the fact that all responses
generated by ChatGPT in their study were incorrect, imperfect, or misleading,
the evaluation results indicated that when confronted with a difficult
question, students perceived ChatGPT’s scientific accuracy to be on par with
that of the expert solution, attributing this to the higher linguistic quality
of ChatGPT. However, in cases where the questions were relatively easy, both
the scientific accuracy and linguistic quality of the expert’s answers
surpassed that of ChatGPT.

#### Law.

ChatGPT was tested with four law class exams by Choi et al. (2023), including
Constitutional Law, Employee Benefits, Taxation, and Torts. ChatGPT performed
better on 12 essay questions than on the 95 multiple-choice questions.
Although it passed all four exams, it ranked almost the lowest among law
school students in each class. Liu et al. (2023b) examined ChatGPT and GPT-4
in Law School Admission Test (LSAT) and the Chinese Civil Service Examination
(CCSE). Compared with RoBERTa, the advantage of GPT-4 was much larger than
that of ChatGPT. GPT-4 even exceeded the average level of a human on LSAT.
Nevertheless, when compared to the human ceiling, significant disparities
persisted in both models.

#### Economics.

ChatGPT was examined with the Test of Understanding of College Economics
(TUCE, Geerling et al., 2023). It demonstrated proficiency by providing
accurate responses to 19/30 microeconomics questions and 26/30 macroeconomics
questions. Such performance placed it in the upper echelons, ranking within
the top 9% and top 1% among 3,255 and 2,789 college students who had
successfully finished a full semester of microeconomics and macroeconomics.
Xie et al. (2023) tested ChatGPT in stock predictions. ChatGPT showed weaker
performance than traditional machine learning algorithms in numerical feature-
based predictions. Through error analysis, it was determined that ChatGPT was
also weak in comprehending investor sentiment expressed in text.

##  4 Ethical Considerations

#### Fairness

asks models to be fair in terms of gender, race, language, culture, and more.
Seghier (2023); Yong et al. (2023) found that ChatGPT’s responses were much
worse in languages other than English (see § 2.2 ). Zhuo et al. (2023)
assessed two types of social biases, namely gender bias and race bias. They
concluded that, for text generation, although biases still existed, ChatGPT
had mitigated them to a large extent compared to its predecessors, and that,
for dialogue generation, ChatGPT could generate unbiased responses.

#### Robustness

requires models to maintain their performance when the inputs are different
from the training data. Such inputs could be noisy data, outliers and attacks.
For noisy data, Zhuo et al. (2023) tried two NLP datasets while Ye et al.
(2023) tried another 12 datasets. They found that although ChatGPT defeated
preceding LLMs, it was still far from perfection. Wang et al. (2023d) assessed
ChatGPT on outliers, with similar results. Besides generating bad responses,
the unsuccessful handling of such inputs may sometimes cause more serious
consequences, for example, data leakage or Denial of Service attacks. Peng et
al. (2023) assessed whether ChatGPT would suffer from SQL injection if it
served as a text-to-SQL interface (Li et al., 2023a), while the answer was
positive.

#### Reliability

requires the generated text to be faithful. Neural text generators hallucinate
(Ji et al., 2023) and, therefore, produce unfaithful texts. ChatGPT/GPT-4 is
no exception (Bang et al., 2023). Zhuo et al. (2023) assessed ChatGPT on fact-
based question-answering datasets and concluded that it was not improved
compared to its predecessors. This makes ChatGPT unreliable in tasks where
faithfulness is vital. For example, it might make up references when writing
scientific articles (Athaluri et al., 2023) and make up legal cases when
serving in the legal domain (Deroy et al., 2023).

#### Toxicity

asks models not to generate harmful, offensive and pornographic content. The
GPT models were designed to normally refuse to generate toxic content.
Nevertheless, Derner and Batistič (2023) found that through role-playing
ChatGPT still produced offensive content. A quantitative study by Zhuo et al.
(2023) showed that, by feeding toxic prompting, only 0.5% of the ChatGPT
responses were toxic. Nonetheless, this ability was not very robust. In line
with Derner and Batistič (2023), they also found that it was susceptible to
prompt injections achieved by role-playing.

##  5 Discussion

###  5.1 Comparing GPT versus Humans

In specific NLP tasks with rich training resources, ChatGPT and GPT-4 may not
perform as well as expert models (see Tables 1 and 3). There is also a certain
distance from humans (see Tables 2 and 4). However, when it comes to
scientific knowledge, extensive multidisciplinary testing has showcased their
superiority compared to earlier models (see Table 5). Notably, in computer
science and law exams, GPT-4 has achieved a level of accuracy that closely
matches or even surpasses the average human performance (see Table 6).
However, when we are quoted a study that shows an AI system performing at a
similar level to humans, these are typically based on average performance over
many cases. This average hides the fact that the AI is typically outperforming
on some specialist knowledge that is difficult for humans, and underperforming
on some examples that are relatively easy for humans. The same has been shown
in computer vision (Russakovsky et al., 2015). Thus, human-level average
accuracy achieved by AI is not equivalent to human performance and
intelligence.

The GPTs may exhibit instances of hallucinations and false information
(Cabrera and Neubig, 2023; Bang et al., 2023). This may be attributed to the
fact that the “next word prediction”-based pre-training only taught GPTs “what
is right” while neglecting “what is wrong”. To learn to generate a correct
last word, e.g., “bird” after the context “if an animal has wings and can fly,
it is likely a”, GPTs have to learn logic, commonsense, linguistics, and
science. However, without learning “if an animal has wings and can fly, it is
a penguin” is wrong, GPTs may struggle to distinguish penguins from other
birds by flight ability and yield penguins’ flying hallucinations, because
GPTs likely have learned “penguins are birds, having wings” from corpora. In
reality, incorrect examples are far more than the negations we can see from
corpora. False information may exhibit in ambiguous cases if GPTs do not know
the boundary between positive and negative examples. In contrast, humans learn
knowledge from both right and wrong applications to sidestep obvious fallacies
(NASEM, 2018).

The inference process of GPT models is also dissimilar to humans. Humans are
known to have two types of reasoning: “thinking fast” and “thinking slow”
Kahneman (2000). However, GPT models seem to only have thinking fast; they do
a feedforward propagation and perform relatively fast inference (Bubeck et
al., 2023). Humans in contrast can spend longer deliberating for hard
questions. This requires a longer process of iterative inference, not
necessarily dictated by a fixed number of iterations, but instead iterating
until the system achieves a satisfactory state (van Bergen and Kriegeskorte,
2020). Nevertheless, it is clear that GPT models can do a certain type of
reasoning. Especially, it has been shown that adding “Let’s think step by
step” to a prompt can allow GPT to use its own output as a sort of scratchpad
to help it chain together multiple steps to arrive at a solution (Bubeck et
al., 2023). In this way GPT is simulating “thinking slow”, however it is
limited to “linear” sequences of thought and has severe limitations in tasks
that require planning (ibid.); it does not backtrack to try other possible
alternatives. Such backtracking would require some sort of short-term memory
or workspace to remember what has been tried and what is yet to be tried.
These are well-known shortcomings of neural models (Minsky, 1991).

Bubeck et al. (2023) noted the context-dependence of GPT’s mathematical
knowledge; “changes in the wording of the question can alter the knowledge
that the model displays.” The general phenomenon of sensitivity to input
phrasing is well known from other language models also (Mao et al., 2022).
Similarly, Lai et al. (2023) found a huge drop in commonsense knowledge when
testing in languages other than English. This also explains why assessments by
different papers can arrive at contradictory conclusions about its knowledge.
It demonstrates that the internal representation in GPT suffers from severe
entanglement (Bengio et al., 2013). While more extensive training can cause it
to give correct responses in more contexts, it cannot change the fundamental
fact that knowledge is not disentangled from the text contexts in which it
appears; therefore there will always be the possibility that a change to the
text of a question, that preserves the semantics, will elicit a factually
incorrect response. Fixing this issue may require a hybrid system with
separate facts. A commonsense-based neurosymbolic AI framework, such as the
one proposed by Cambria et al. (2022) for sentiment analysis, moreover, can
help increase the explainability of the reasoning processes required for
decision-making, which is crucial for sensitive applications involving ethics,
privacy and health.

###  5.2 Evaluation

Concerns about the validity of NLP evaluations in general (Belz et al., 2023)
are applicable to this paper also. The following points are worth noting: 1)
ChatGPT as a commercial product is updated periodically. On one hand, this
means the “flaws” identified in early studies are fixed in later versions and
different studies about the same task are not fully comparable. On the other
hand, since the repair may be done by including datasets from previous
assessments in fine-tuning, the data leakage may make the assessment unfair
(Min et al., 2022; OpenAI, 2023a). This has been well-evidenced in Aiyappa et
al. (2023) and de Wynter et al. (2023). 2) The design of prompts highly
influences the results, leading to biased comparisons. Take MT as an example,
by seeking better prompts, Hendy et al. (2023) and Jiao et al. (2023) had very
different conclusions compared to Bang et al. (2023). In future studies, it is
necessary to ensure transparency in prompt design and facilitate a fair
comparison between LLMs and baselines. 3) The factors that matter in previous
NLP evaluations are still valid, which may include the choice of evaluation
corpora and metrics, the design of human evaluations, the task formulation and
so on. For instance, Martínez (2023) argued that OpenAI’s assessment of GPT-4
(OpenAI, 2023a) on the Uniform Bar Exam is misleading because they incorrectly
included test-takers who re-took the exam in their comparison. Zhang et al.
(2023b) misconcluded that GPT-4 had solved all MIT math and computer science
curricula because they improperly used GPT-4 as the judge. 4) The assessments
of some key abilities, e.g., creativity and logical reasoning, lack either
objective criteria or large-scale benchmarks. Consequently, the evaluation can
only capture a portion of the overall capabilities of GPT. People seem to
commonly believe that AI will cause mass unemployment (Hatzius et al., 2023).
However, in these fields, the evaluation is also very limited. It would be
expected to see what kind of leading results the GPT models achieved compared
with humans in the field of work that can be replaced by AI.

###  5.3 Ethics

Several works have found that human perception about the reliability of
ChatGPT’s output can be misled by its seemingly scientific language style
(Gudibande et al., 2023; Dahlkemper et al., 2023). Given AI-generated content,
it would be necessary to allow users to retrieve source references that were
generated by humans to improve response reliability. Caution should be also
exercised when employing AI-generated data for training new AI models, as this
practice carries the risk of introducing irreparable flaws into the resultant
models (Shumailov et al., 2023). Even if the Reinforcement Learning from Human
Feedback (RLHF, OpenAI, 2023b) can somewhat mitigate biased, inaccurate, and
toxic responses, and enhance human-centric output preferences, RLHF may be
misled by human-biased feedback, e.g., system gaming (Leike et al., 2017),
positive reward cycles (Ho et al., 2015), and tangled social norms and
cultural context (Liu, 2023). There are also concerns regarding the non-
transparency of the training set of ChatGPT, which could potentially lead to
data leakage, including trade secrets or personal privacy, as user inputs
might be utilized for fine-tuning (Min et al., 2022). These limitations
appearing in the training and inferring processes may raise additional biases
and ethical concerns.

## Limitations

Several assessment papers are pre-print, and these papers may contain non-
authoritative viewpoints and findings. Despite our efforts to filter out
assessments with evident shortcomings, it is important to acknowledge the
potential presence of unverified information in these papers. As argued in §
5.2, ChatGPT and GPT-4 are commercial products. The current assessments and
the conclusion upon the assessments may be not suitable and reproducible in
their later versions.

## References

  * Aiyappa et al. (2023) Rachith Aiyappa, Jisun An, Haewoon Kwak, and Yong-Yeol Ahn. 2023.  Can we trust the evaluation on ChatGPT?  _arXiv preprint arXiv:2303.12767_.
  * Amin et al. (2023) Mostafa M Amin, Erik Cambria, and Björn W Schuller. 2023.  Will affective computing emerge from foundation models and general artificial intelligence? A first evaluation of ChatGPT.  _IEEE Intelligent Systems_ , 38(2):15–23.
  * Antaki et al. (2023) Fares Antaki, Samir Touma, Daniel Milad, Jonathan El-Khoury, and Renaud Duval. 2023\.  Evaluating the performance of ChatGPT in ophthalmology: An analysis of its successes and shortcomings.  _Ophthalmology Science_ , page 100324.
  * Athaluri et al. (2023) Sai Anirudh Athaluri, Sandeep Varma Manthena, VSR Krishna Manoj Kesapragada, Vineel Yarlagadda, Tirth Dave, and Rama Tulasi Siri Duddumpudi. 2023.  Exploring the boundaries of reality: Investigating the phenomenon of artificial intelligence hallucination in scientific writing through ChatGPT references.  _Cureus_ , 15(4).
  * Bang et al. (2023) Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023.  A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity.  _arXiv preprint arXiv:2302.04023_.
  * Belz et al. (2023) Anja Belz, Craig Thomson, and Ehud Reiter. 2023.  Missing information, unresponsive authors, experimental flaws: The impossibility of assessing the reproducibility of previous human evaluations in nlp.  In _The Fourth Workshop on Insights from Negative Results in NLP_ , pages 1–10.
  * Bengio et al. (2013) Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013.  Representation learning: A review and new perspectives.  _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 35(8):1798–1828.
  * Bhandari et al. (2020) Manik Bhandari, Pranav Narayan Gour, Atabak Ashfaq, Pengfei Liu, and Graham Neubig. 2020.  Re-evaluating evaluation in text summarization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9347–9359.
  * Bordt and von Luxburg (2023) Sebastian Bordt and Ulrike von Luxburg. 2023.  ChatGPT participates in a computer science exam.  _arXiv preprint arXiv:2303.09461_.
  * Borji (2023) Ali Borji. 2023.  A categorical archive of ChatGPT failures.  _arXiv preprint arXiv:2302.03494_.
  * Bubeck et al. (2023) Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023.  Sparks of artificial general intelligence: Early experiments with GPT-4.  _arXiv preprint arXiv:2303.12712_.
  * Cabrera and Neubig (2023) Alex Cabrera and Graham Neubig. 2023.  Zeno chatbot report.  https://github.com/zeno-ml/zeno-build/tree/main/tasks/chatbot/report.  Accessed: 18/05/2023.
  * Cambria et al. (2022) Erik Cambria, Qian Liu, Sergio Decherchi, Frank Xing, and Kenneth Kwok. 2022.  SenticNet 7: A commonsense-based neurosymbolic AI framework for explainable sentiment analysis.  In _Proceedings of the Thirteenth Language Resources and Evaluation Conference_ , pages 3829–3839, Marseille, France. European Language Resources Association.
  * Chen et al. (2021) Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021.  Evaluating large language models trained on code.  _arXiv preprint arXiv:2107.03374_.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023.  Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality.  Accessed: 16/05/2023.
  * Choi et al. (2023) Jonathan H Choi, Kristin E Hickman, Amy Monahan, and Daniel Schwarcz. 2023.  ChatGPT goes to law school.  _Journal of Legal Education (Forthcoming)_.
  * Chu and Liu (2023) Haoran Chu and Sixiao Liu. 2023.  Can AI tell good stories? narrative transportation and persuasion with ChatGPT.  _PsyArXiv_.
  * Clark (2023) Ted M Clark. 2023.  Investigating the use of an artificial intelligence chatbot with general chemistry exam questions.  _Journal of Chemical Education_.
  * Dahlkemper et al. (2023) Merten Nikolay Dahlkemper, Simon Zacharias Lahme, and Pascal Klein. 2023.  How do physics students evaluate artificial intelligence responses on comprehension questions? A study on the perceived scientific accuracy and linguistic quality.  _arXiv preprint arXiv:2304.05906_.
  * Davis (2023) Ernest Davis. 2023.  Benchmarks for automated commonsense reasoning: A survey.  _arXiv preprint arXiv:2302.04752_.
  * de Wynter et al. (2023) Adrian de Wynter, Xun Wang, Alex Sokolov, Qilong Gu, and Si-Qing Chen. 2023.  An evaluation on large language model outputs: Discourse and memorization.  _arXiv preprint arXiv:2304.08637_.
  * Derner and Batistič (2023) Erik Derner and Kristina Batistič. 2023.  Beyond the safeguards: Exploring the security risks of ChatGPT.  _arXiv preprint arXiv:2305.08005_.
  * Deroy et al. (2023) Aniket Deroy, Kripabandhu Ghosh, and Saptarshi Ghosh. 2023.  How ready are pre-trained abstractive models and LLMs for legal case judgement summarization?  _arXiv preprint arXiv:2306.01248_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional Transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 4171–4186.
  * Fabbri et al. (2021) Alexander R Fabbri, Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. 2021.  SummEval: Re-evaluating summarization evaluation.  _Transactions of the Association for Computational Linguistics_ , 9:391–409.
  * Frieder et al. (2023) Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier, and Julius Berner. 2023\.  Mathematical capabilities of ChatGPT.  _arXiv preprint arXiv:2301.13867_.
  * Gao et al. (2023) Jinglong Gao, Xiao Ding, Bing Qin, and Ting Liu. 2023.  Is ChatGPT a good causal reasoner? a comprehensive evaluation.  _arXiv preprint arXiv:2305.07375_.
  * Geerling et al. (2023) Wayne Geerling, G Dirk Mateer, Jadrian Wooten, and Nikhil Damodaran. 2023.  ChatGPT has aced the test of understanding in college economics: Now what?  _The American Economist_ , page 05694345231169654.
  * Gilardi et al. (2023) Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023.  ChatGPT outperforms crowd-workers for text-annotation tasks.  _arXiv preprint arXiv:2303.15056_.
  * Gilson et al. (2023) Aidan Gilson, Conrad W Safranek, Thomas Huang, Vimig Socrates, Ling Chi, Richard Andrew Taylor, David Chartash, et al. 2023.  How does ChatGPT perform on the United States Medical Licensing Examination? The implications of large language models for medical education and knowledge assessment.  _JMIR Medical Education_ , 9(1):e45312.
  * Grusky et al. (2018) Max Grusky, Mor Naaman, and Yoav Artzi. 2018.  NEWSROOM: A dataset of 1.3 million summaries with diverse extractive strategies.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 708–719.
  * Guan et al. (2021) Jian Guan, Zhexin Zhang, Zhuoer Feng, Zitao Liu, Wenbiao Ding, Xiaoxi Mao, Changjie Fan, and Minlie Huang. 2021.  OpenMEVA: A benchmark for evaluating open-ended story generation metrics.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing_ , pages 6394–6407.
  * Gudibande et al. (2023) Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023.  The false promise of imitating proprietary LLMs.  _arXiv preprint arXiv:2305.15717_.
  * Hatzius et al. (2023) RJan Hatzius, Joseph Briggs, Devesh Kodnani, and Giovanni Pierdomenico. 2023.  The potentially large effects of artificial intelligence on economic growth.  Technical report, Goldman Sachs.
  * Hendy et al. (2023) Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023.  How good are GPT models at machine translation? a comprehensive evaluation.  _arXiv preprint arXiv:2302.09210_.
  * Hirosawa et al. (2023) Takanobu Hirosawa, Yukinori Harada, Masashi Yokose, Tetsu Sakamoto, Ren Kawamura, and Taro Shimizu. 2023.  Diagnostic accuracy of differential-diagnosis lists generated by generative pretrained Transformer 3 chatbot for clinical vignettes with common chief complaints: A pilot study.  _International Journal of Environmental Research and Public Health_ , 20(4):3378.
  * Ho et al. (2015) Mark K Ho, Michael L Littman, Fiery Cushman, and Joseph L Austerweil. 2015.  Teaching with rewards and punishments: Reinforcement or communication?  In _CogSci_.
  * Holterman and van Deemter (2023) Bart Holterman and Kees van Deemter. 2023.  Does ChatGPT have theory of mind?  _arXiv preprint arXiv:2305.14020_.
  * Huang et al. (2023) Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, et al. 2023.  C-EVAL: A multi-level multi-discipline chinese evaluation suite for foundation models.  _arXiv preprint arXiv:2305.08322_.
  * Jentzsch and Kersting (2023) Sophie Jentzsch and Kristian Kersting. 2023.  ChatGPT is fun, but it is not funny! humor is still challenging large language models.  _arXiv preprint arXiv:2306.04563_.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38.
  * Jiao et al. (2023) Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Xing Wang, and Zhaopeng Tu. 2023.  Is ChatGPT a good translator? yes with GPT-4 as the engine.  _arXiv preprint arXiv:2301.08745_.
  * Kahneman (2000) Daniel Kahneman. 2000.  _Thinking Fast and Slow_.  Farrar.
  * Karpinska and Iyyer (2023) Marzena Karpinska and Mohit Iyyer. 2023.  Large language models effectively leverage document-level context for literary translation, but critical errors persist.  _arXiv preprint arXiv:2304.03245_.
  * Kıcıman et al. (2023) Emre Kıcıman, Robert Ness, Amit Sharma, and Chenhao Tan. 2023.  Causal reasoning and large language models: Opening a new frontier for causality.  _arXiv preprint arXiv:2305.00050_.
  * Kocmi and Federmann (2023) Tom Kocmi and Christian Federmann. 2023.  Large language models are state-of-the-art evaluators of translation quality.  _arXiv preprint arXiv:2302.14520_.
  * Kocoń et al. (2023) Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, et al. 2023.  ChatGPT: Jack of all trades, master of none.  _arXiv preprint arXiv:2302.10724_.
  * Kortemeyer (2023) Gerd Kortemeyer. 2023.  Could an artificial-intelligence agent pass an introductory physics course?  _Physical Review Physics Education Research_ , 19(1):010132.
  * Kosinski (2023) Michal Kosinski. 2023.  Theory of mind may have spontaneously emerged in large language models.  _arXiv preprint arXiv:2302.02083_.
  * Kung et al. (2023) Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. 2023.  Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models.  _PLoS Digital Health_ , 2(2):e0000198.
  * Lai et al. (2023) Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023.  ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning.  _arXiv preprint arXiv:2304.05613_.
  * Laskar et al. (2023) Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, and Jimmy Xiangji Huang. 2023.  A systematic study and comprehensive evaluation of chatgpt on benchmark datasets.  _arXiv preprint arXiv:2305.18486_.
  * Leike et al. (2017) Jan Leike, Miljan Martic, and Shane Legg. 2017.  Learning through human feedback.  Technical report, Google DeepMind.
  * Lewkowycz et al. (2022) Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. 2022.  Solving quantitative reasoning problems with language models.  In _Advances in Neural Information Processing Systems_.
  * Li et al. (2023a) Jinyang Li, Binyuan Hui, Ge Qu, Binhua Li, Jiaxi Yang, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin C. C. Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023a.  Can LLM already serve as a database interface? a big bench for large-scale database grounded text-to-SQLs.  _arXiv preprint arXiv:2305.03111_.
  * Li et al. (2023b) Jinyang Li, Binyuan Hui, Ge Qu, Binhua Li, Jiaxi Yang, Bowen Li, Bailin Wang, Bowen Qin, Rongyu Cao, Ruiying Geng, et al. 2023b.  Can LLM already serve as a database interface? a big bench for large-scale database grounded text-to-SQLs.  _arXiv preprint arXiv:2305.03111_.
  * Lin (2004) Chin-Yew Lin. 2004.  ROUGE: A package for automatic evaluation of summaries.  In _Text Summarization Branches Out_ , pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
  * Liu et al. (2023a) Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2023a.  We’re afraid language models aren’t modeling ambiguity.  _arXiv preprint arXiv:2304.14399_.
  * Liu (2023) Gabrielle Kaili-May Liu. 2023.  Perspectives on the social impacts of reinforcement learning with human feedback.  _arXiv preprint arXiv:2303.02891_.
  * Liu et al. (2023b) Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. 2023b.  Evaluating the logical reasoning ability of ChatGPT and GPT-4.  _arXiv preprint arXiv:2304.03439_.
  * Liu et al. (2023c) Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. 2023c.  G-EVAL: NLG evaluation using GPT-4 with better human alignment.  _arXiv preprint arXiv:2303.16634_.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  RoBERTa: A robustly optimized BERT pretraining approach.  _arXiv preprint arXiv:1907.11692_.
  * Mairesse et al. (2010) François Mairesse, Milica Gasic, Filip Jurcicek, Simon Keizer, Blaise Thomson, Kai Yu, and Steve Young. 2010.  Phrase-based statistical language generation using graphical models and active learning.  In _Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics_ , pages 1552–1561.
  * Mao et al. (2022) Rui Mao, Qian Liu, Kai He, Wei Li, and Erik Cambria. 2022.  The biases of pre-trained language models: An empirical study on prompt-based sentiment analysis and emotion detection.  _IEEE Transactions on Affective Computing_ , pages 1–11.
  * Marcus and Davis (2023) Gary Marcus and Ernest Davis. 2023.  How not to test GPT.  Accessed: 29/05/2023.
  * Martínez (2023) Eric Martínez. 2023.  Re-evaluating GPT-4’s bar exam performance.  _Available at SSRN 4441311_.
  * Mehnen et al. (2023) Lars Mehnen, Stefanie Gruarin, Mina Vasileva, and Bernhard Knapp. 2023.  ChatGPT as a medical doctor? a diagnostic accuracy study on common and rare diseases.  _medRxiv_ , pages 2023–04.
  * Mehri and Eskenazi (2020) Shikib Mehri and Maxine Eskenazi. 2020.  USR: An unsupervised and reference free evaluation metric for dialog generation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 681–707.
  * Min et al. (2022) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022.  Rethinking the role of demonstrations: What makes in-context learning work? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 11048–11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Minsky (1991) Marvin L Minsky. 1991.  Logical versus analogical or symbolic versus connectionist or neat versus scruffy.  _AI magazine_ , 12(2):34–34.
  * Moghaddam and Honey (2023) Shima Rahimi Moghaddam and Christopher J. Honey. 2023.  Boosting theory-of-mind performance in large language models via prompting.  _arXiv preprint arXiv:2304.11490_.
  * MosaicML (2023) NLP Team MosaicML. 2023.  Introducing MPT-7B: A new standard for open-source, commercially usable LLMs.  www.mosaicml.com/blog/mpt-7b.  Accessed: 18/05/2023.
  * NASEM, National Academies of Sciences, Engineering, and Medicine and others (2018) NASEM, National Academies of Sciences, Engineering, and Medicine and others. 2018\.  _How people learn II: Learners, contexts, and cultures_.  National Academies Press.
  * OpenAI (2023a) OpenAI. 2023a.  GPT-4 technical report.  _arXiv preprint arXiv:2303.08774_.
  * OpenAI (2023b) OpenAI. 2023b.  Introducing ChatGPT.  openai.com/blog/chatgpt.  Accessed: 11/05/2023.
  * Payne (2020) Brad Payne. 2020.  Privacy protection with AI: Survey of data-anonymization techniques.  Accessed: 03/06/2023.
  * Peng et al. (2023) Xutan Peng, Yipeng Zhang, Jingfeng Yang, and Mark Stevenson. 2023.  On the security vulnerabilities of text-to-sql models.  _arXiv preprint arXiv:2211.15363_.
  * Popović (2015) Maja Popović. 2015.  chrF: character n-gram F-score for automatic MT evaluation.  In _Proceedings of the 10th Workshop on Statistical Machine Translation_.
  * Qin et al. (2023) Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023.  Is ChatGPT a general-purpose natural language processing task solver?  _arXiv preprint arXiv:2302.06476_.
  * Radford et al. (2019) Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019.  Language models are unsupervised multitask learners.  Technical report, OpenAI.
  * Rao et al. (2023) Arya S Rao, Michael Pang, John Kim, Meghana Kamineni, Winston Lie, Anoop K Prasad, Adam Landman, Keith Dryer, and Marc D Succi. 2023.  Assessing the utility of ChatGPT throughout the entire clinical workflow.  _medRxiv_ , pages 2023–02.
  * Russakovsky et al. (2015) Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. 2015\.  ImageNet large scale visual recognition challenge.  _International Journal of Computer Vision_ , 115:211–252.
  * Sang and De Meulder (2003) Erik Tjong Kim Sang and Fien De Meulder. 2003.  Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition.  In _Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003_ , pages 142–147.
  * Seghier (2023) Mohamed L Seghier. 2023.  ChatGPT: Not all languages are equal.  _Nature_ , 615(7951):216–216.
  * Shumailov et al. (2023) Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. 2023.  The curse of recursion: Training on generated data makes models forget.  _arXiv preprint arXiv:2305.17493_.
  * Soni and Wade (2023) Mayank Soni and Vincent Wade. 2023.  Comparing abstractive summaries generated by ChatGPT to real summaries through blinded reviewers and text classification algorithms.  _arXiv preprint arXiv:2303.17650_.
  * Sun et al. (2023) Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. 2023\.  Is ChatGPT good at search? investigating large language models as re-ranking agent.  _arXiv preprint arXiv:2304.09542_.
  * Tang et al. (2021) Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Naman Goyal, Vishrav Chaudhary, Jiatao Gu, and Angela Fan. 2021.  Multilingual translation from denoising pre-training.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 3450–3466.
  * Taori et al. (2023) Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023.  Alpaca: A strong, replicable instruction-following model.  Technical report, Stanford Center for Research on Foundation Models.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023.  LLaMa: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_.
  * van Bergen and Kriegeskorte (2020) Ruben S van Bergen and Nikolaus Kriegeskorte. 2020.  Going in circles is the way forward: The role of recurrence in visual inference.  _Current Opinion in Neurobiology_ , 65:176–193.  Whole-brain interactions between neural circuits.
  * Wang et al. (2023a) Boshi Wang, Xiang Yue, and Huan Sun. 2023a.  Can ChatGPT defend the truth? automatic dialectical evaluation elicits LLMs’ deficiencies in reasoning.  _arXiv preprint arXiv:2305.13160_.
  * Wang et al. (2023b) Jiaan Wang, Yunlong Liang, Fandong Meng, Zhixu Li, Jianfeng Qu, and Jie Zhou. 2023b.  Zero-shot cross-lingual summarization via large language models.  _arXiv preprint arXiv:2302.14229_.
  * Wang et al. (2023c) Jiaan Wang, Yunlong Liang, Fandong Meng, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023c.  Is ChatGPT a good NLG evaluator? A preliminary study.  _arXiv preprint arXiv:2303.04048_.
  * Wang et al. (2023d) Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Wei Ye, Xiubo Geng, Binxin Jiao, Yue Zhang, and Xing Xie. 2023d.  On the robustness of ChatGPT: An adversarial and out-of-distribution perspective.  _arXiv preprint arXiv:2302.12095_.
  * Wei et al. (2023) Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al. 2023.  Zero-shot information extraction via chatting with ChatGPT.  _arXiv preprint arXiv:2302.10205_.
  * Xie et al. (2023) Qianqian Xie, Weiguang Han, Yanzhao Lai, Min Peng, and Jimin Huang. 2023.  The Wall Street neophyte: A zero-shot analysis of ChatGPT over multimodal stock movement prediction challenges.  _arXiv preprint arXiv:2304.05351_.
  * Xu and others from SuperCLUE team (2023) Liang Xu and others from SuperCLUE team. 2023.  SuperCLUE: A benchmark for foundation models in chinese.  https://github.com/CLUEbenchmark/SuperCLUE.  Accessed: 18/05/2023.
  * Ye et al. (2023) Wentao Ye, Mingfeng Ou, Tianyi Li, Yipeng chen, Xuetao Ma, Yifan Yanggong, Sai Wu, Jie Fu, Gang Chen, Haobo Wang, and Junbo Zhao. 2023.  Assessing hidden risks of LLMs: An empirical study on robustness, consistency, and credibility.  _arXiv preprint arXiv:2305.10235_.
  * Yong et al. (2023) Zheng-Xin Yong, Ruochen Zhang, Jessica Zosa Forde, Skyler Wang, Samuel Cahyawijaya, Holy Lovenia, Genta Indra Winata, Lintang Sutawika, Jan Christian Blaise Cruz, Long Phan, Yin Lin Tan, and Alham Fikri Aji. 2023.  Prompting multilingual large language models to generate code-mixed texts: The case of south east asian languages.  _arXiv preprint arXiv:2303.13592_.
  * Yuan et al. (2021) Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021.  BARTScore: Evaluating generated text as text generation.  _Advances in Neural Information Processing Systems_ , 34:27263–27277.
  * Zhang et al. (2023a) Haopeng Zhang, Xiao Liu, and Jiawei Zhang. 2023a.  Extractive summarization via ChatGPT for faithful summary generation.  _arXiv preprint arXiv:2304.04193_.
  * Zhang et al. (2023b) Sarah J. Zhang, Samuel Florin, Ariel N. Lee, Eamon Niknafs, Andrei Marginean, Annie Wang, Keith Tyser, Zad Chin, Yann Hicke, Nikhil Singh, Madeleine Udell, Yoon Kim, Tonio Buonassisi, Armando Solar-Lezama, and Iddo Drori. 2023b.  Exploring the MIT mathematics and EECS curriculum using large language models.  _arXiv preprint arXiv:2306.08997_.
  * Zhang et al. (2019) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019\.  BERTScore: Evaluating text generation with bert.  In _International Conference on Learning Representations_.
  * Zhong et al. (2022) Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, Pengfei Liu, Chenguang Zhu, Heng Ji, and Jiawei Han. 2022.  Towards a unified multi-dimensional evaluator for text generation.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 2023–2038.
  * Zhuo et al. (2023) Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. 2023.  Red teaming ChatGPT via jailbreaking: Bias, robustness, reliability and toxicity.  _arXiv preprint arXiv:2301.12867_.

##  Appendix A ChatGPT and GPT-4 benchmark

We visualize the performance gaps between the surveyed GPT models and
baselines on different tasks in Tables 1-6. The visualization depicts varying
degrees of discrepancy, with darker colors indicating larger gaps and lighter
colors indicating smaller gaps. The degree of discrepancy is measured by
$g/b-1$, where $g$ is the average score of a GPT model on the major metric of
an evaluation task; $b$ is the average score of a baseline on the same major
metric. The color red indicates instances where the GPT models outperformed
the baselines, while blue indicates cases where the GPT models lagged behind.
The teal color represents the gaps between the GPT models and the ground
truth, while gray indicates a lack of comparison. Task abbreviations and
meanings can be viewed in Table 7. The summary of the surveyed assessment
papers can be viewed in Tables 8 and 9.

|  | ChatGPT | GPT-4
---|---|---|---
|  |  Cabrera and Neubig (2023) |  Amin et al. (2023) |  Kocoń et al. (2023) |  Liu et al. (2023b) |  Lai et al. (2023) |  Qin et al. (2023) |  Wei et al. (2023) |  Hendy et al. (2023) |  Bang et al. (2023) |  Gao et al. (2023) |  Kıcıman et al. (2023) |  Wang et al. (2023c) |  Kocmi and Federmann (2023) |  Jiao et al. (2023) |  Karpinska and Iyyer (2023) |  Zhang et al. (2023a) |  Sun et al. (2023) |  Liu et al. (2023a) |  Liu et al. (2023b) |  Gao et al. (2023) |  Kıcıman et al. (2023) |  Liu et al. (2023c) |  Jiao et al. (2023) |  Sun et al. (2023) |  Bubeck et al. (2023) |  Liu et al. (2023a)
Compare to |  LLM |  Supv |  Supv |  Supv |  Supv |  Supv |  Supv |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  Supv |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA |  SOTA
Gene. | Dial.S |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Gene. | Summ. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Gene. | Mach.T |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Gene. | Mach.T* |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Synt. | PoS |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | NER |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | RE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | Ling.A |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | WSD |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | Info.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | Pass.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | QA |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | Stan.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Offe.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Humo.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Sarc.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Spam.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Misi.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Pers.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Sent.A |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Suic.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Emot.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Prag. | Emoj.P |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Caus.D |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | CEG |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Coun.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | T-Sy.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | T-Lo.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | CBQA |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | DLR |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | NLI |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Com.R |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Eval. | TSE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Eval. | SGE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Eval. | D2TE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Eval. | DGE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Eval. | Mach.T |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Table 1: ChatGPT and GPT-4 performance on linguistic and reasoning tasks, compared to baseline models. Supv denotes supervised models. |  | ChatGPT | GPT-4
---|---|---|---
|  |  Hendy et al. (2023) |  Bang et al. (2023) |  Gao et al. (2023) |  Kıcıman et al. (2023) |  Kıcıman et al. (2023) |  Liu et al. (2023b) |  Liu et al. (2023a) |  Gilardi et al. (2023) |  Liu et al. (2023b) |  Gao et al. (2023) |  Kıcıman et al. (2023) |  Kıcıman et al. (2023) |  Liu et al. (2023a)
Compar to |  Human avg |  Ground truth |  Human avg |  Human avg |  Ground true |  Human avg |  Human avg |  Human avg |  Human avg |  Human avg |  Human avg |  Ground true |  Human avg
Gene. | Mach.T |  |  |  |  |  |  |  |  |  |  |  |  |
Sem. | Sent.A |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Logi.R |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Com.R |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Caus.R |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | CEG |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | Coun.R |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | ACI |  |  |  |  |  |  |  |  |  |  |  |  |
Reas. | NLI |  |  |  |  |  |  |  |  |  |  |  |  |
Eval. | T.Acc |  |  |  |  |  |  |  |  |  |  |  |  |
Eval | T.Agre |  |  |  |  |  |  |  |  |  |  |  |  |
Table 2: ChatGPT and GPT-4 performance on linguistic and reasoning tasks, compared to humans or ground truth. |  | ChatGPT | GPT-4 | ChatGPT
---|---|---|---|---
|  |  Xu et al. (2023) |  Huang et al. (2023) |  Lai et al. (2023) |  Wei et al. (2023) |  Wang et al. (2023b) |  Xu et al. (2023) |  Huang et al. (2023) |  Wang et al. (2023b) |  |  |  Lai et al. (2023)
Compare to |  LLM |  LLM |  Supv |  Supv |  SOTA |  LLM |  LLM |  SOTA |  |  |  Supv
en2zh |  Summ. |  |  |  |  |  |  |  |  |  jp |  Com.R |
en2de |  Summ. |  |  |  |  |  |  |  |  |  vi |  PoS |
ru |  PoS |  |  |  |  |  |  |  |  |  vi |  NLI |
ru |  NER |  |  |  |  |  |  |  |  |  vi |  QA |
ru |  RE |  |  |  |  |  |  |  |  |  vi |  Com.R |
ru |  NLI |  |  |  |  |  |  |  |  |  tr |  PoS |
ru |  QA |  |  |  |  |  |  |  |  |  tr |  NER |
ru |  Com.R |  |  |  |  |  |  |  |  |  tr |  NLI |
ru |  Summ. |  |  |  |  |  |  |  |  |  tr |  QA |
de |  PoS |  |  |  |  |  |  |  |  |  tr |  Summ. |
de |  NER |  |  |  |  |  |  |  |  |  sv |  RE |
de |  RE |  |  |  |  |  |  |  |  |  ar |  PoS |
de |  NLI |  |  |  |  |  |  |  |  |  ar |  RE |
de |  QA |  |  |  |  |  |  |  |  |  ar |  NLI |
de |  Com.R |  |  |  |  |  |  |  |  |  ar |  QA |
zh |  PoS |  |  |  |  |  |  |  |  |  ar |  Com.R |
zh |  NER |  |  |  |  |  |  |  |  |  ar |  Summ. |
zh |  RE |  |  |  |  |  |  |  |  |  fa |  NER |
zh |  EE |  |  |  |  |  |  |  |  |  fa |  RE |
zh |  NLI |  |  |  |  |  |  |  |  |  ko |  NER |
zh |  QA |  |  |  |  |  |  |  |  |  ko |  RE |
zh |  Com.R |  |  |  |  |  |  |  |  |  el |  PoS |
zh |  Summ. |  |  |  |  |  |  |  |  |  el |  NLI |
zh |  Ling.A |  |  |  |  |  |  |  |  |  el |  QA |
zh |  Prof.A |  |  |  |  |  |  |  |  |  th |  PoS |
zh |  CFA |  |  |  |  |  |  |  |  |  th |  NLI |
zh |  STEM |  |  |  |  |  |  |  |  |  th |  QA |
zh |  Soci.S |  |  |  |  |  |  |  |  |  uk |  RE |
zh |  Huma. |  |  |  |  |  |  |  |  |  bg |  PoS |
zh |  MVQE |  |  |  |  |  |  |  |  |  bg |  NLI |
fr |  PoS |  |  |  |  |  |  |  |  |  hi |  PoS |
fr |  RE |  |  |  |  |  |  |  |  |  hi |  NLI |
fr |  NLI |  |  |  |  |  |  |  |  |  hi |  QA |
fr |  Com.R |  |  |  |  |  |  |  |  |  hi |  Com.R |
fr |  Summ. |  |  |  |  |  |  |  |  |  hi |  Summ. |
es |  PoS |  |  |  |  |  |  |  |  |  bn |  NER |
es |  NER |  |  |  |  |  |  |  |  |  bn |  Com.R |
es |  RE |  |  |  |  |  |  |  |  |  bn |  Summ. |
es |  NLI |  |  |  |  |  |  |  |  |  ta |  Com.R |
es |  QA |  |  |  |  |  |  |  |  |  ur |  PoS |
es |  Com.R |  |  |  |  |  |  |  |  |  ur |  NLI |
es |  Summ. |  |  |  |  |  |  |  |  |  ur |  Com.R |
it |  PoS |  |  |  |  |  |  |  |  |  ml |  Com.R |
it |  NER |  |  |  |  |  |  |  |  |  mr |  Com.R |
it |  RE |  |  |  |  |  |  |  |  |  te |  Com.R |
it |  Com.R |  |  |  |  |  |  |  |  |  gu |  Com.R |
nl |  PoS |  |  |  |  |  |  |  |  |  my |  Summ. |
nl |  RE |  |  |  |  |  |  |  |  |  kn |  Com.R |
nl |  Com.R |  |  |  |  |  |  |  |  |  sw |  NLI |
pl |  PoS |  |  |  |  |  |  |  |  |  sw |  Com.R |
pl |  RE |  |  |  |  |  |  |  |  |  pa |  Com.R |
pl |  Com.R |  |  |  |  |  |  |  |  |  ky |  Summ. |
pt |  RE |  |  |  |  |  |  |  |  |  or |  Com.R |
pt |  Com.R |  |  |  |  |  |  |  |  |  as |  Com.R |
Table 3: ChatGPT and GPT-4 performance on multi-lingual tasks, compared to baseline models. Languages are chunked by high (en2zh-vi), medium (tr-hi), low (bn-kn), and extremely low (sw-as) language resources, respectively, categorized by Lai et al. (2023). |  | ChatGPT | G4
---|---|---|---
|  |  Xu et al. (2023) |  Bang et al. (2023) |  Xu et al. (2023)
Compare to |  Human |  Ground truth |  Human
zh | Ling.A |  |  |
zh | Prof.A |  |  |
zh | Overa. |  |  |
id | Sent.A |  |  |
id | LID |  |  |
jw | Sent.A |  |  |
jw | LID |  |  |
bug | Sent.A |  |  |
bug | LID |  |  |
Table 4: ChatGPT and GPT-4 performance on multi-lingual tasks, compared to humans or ground truth. |  | ChatGPT | GPT-4
---|---|---|---
|  |  Gilson et al. (2023) |  Liu et al. (2023b) |  Qin et al. (2023) |  Li et al. (2023b) |  Xie et al. (2023) |  Kung et al. (2023) |  Bubeck et al. (2023) |  Liu et al. (2023b) |  Bubeck et al. (2023)
Compare to |  Old GPT |  Supv |  Supv |  SOTA |  SOTA |  Expert LLM |  SOTA |  Supv |  SOTA
Form.S | T2SQL |  |  |  |  |  |  |  |  |
Form.S | Coding |  |  |  |  |  |  |  |  |
Form.S | LeetCo. |  |  |  |  |  |  |  |  |
Form.S | Math |  |  |  |  |  |  |  |  |
Soci.S | CCSE |  |  |  |  |  |  |  |  |
Soci.S | LSAT |  |  |  |  |  |  |  |  |
Soci.S | Finance |  |  |  |  |  |  |  |  |
Medi.S | Med.Kno. |  |  |  |  |  |  |  |  |
Table 5: ChatGPT and GPT-4 performance on scientific knowledge, compared to baselines. |  | ChatGPT | GPT-4
---|---|---|---
|  |  Li et al. (2023b) |  Bordt and von Luxburg (2023) |  Clark (2023) |  Liu et al. (2023b) |  Antaki et al. (2023) |  Hirosawa et al. (2023) |  Mehnen et al. (2023) |  Bubeck et al. (2023) |  Dahlkemper et al. (2023) |  Frieder et al. (2023) |  Kortemeyer (2023) |  Choi et al. (2023) |  Geerling et al. (2023) |  Xie et al. (2023) |  Gilson et al. (2023) |  Kung et al. (2023) |  Antaki et al. (2023) |  Bordt and von Luxburg (2023) |  Liu et al. (2023b) |  Mehnen et al. (2023) |  Bubeck et al. (2023)
Compare to |  Human avg |  Human avg |  Human avg |  Human avg |  Human avg |  Human avg |  Human avg |  Human avg |  Expert |  Ground truth |  Ground truth |  Ground truth |  Ground truth |  Ground truth |  Ground truth |  Ground truth |  Ground truth |  Human avg |  Human avg |  Human avg |  Human avg
Form.S | T2SQL |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Form.S | LeetCo. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Form.S | Math |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Form.S | CS Exam |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Natu.S | Phys.Exam |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Natu.S | Chem.Exam |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Medi.S | USMLE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Medi.S | OKEP |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Medi.S | Diagno. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | Law Exam |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | LSAT |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | CCSE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | TUCE |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | Finance |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | Edu.Sci. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Soci.S | Edu.Ling. |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
Table 6: ChatGPT and GPT-4 performance on scientific knowledge, compared to humans or ground truth. Abbr. | Meaning | Abbr. | Meaning
---|---|---|---
ACI | Actual Causality Inferring | MVQE | Medical and Vocational Qualification Exams
Caus.D | Cause Discovery | Natu.S | Natural Science
Caus.R | Causal Reasoning | NER | Named Entity Recognition
CBQA | Complex Boolean Question Answering | NLI | Natural Language Inference
CCSE | Chinese Civil Service Examination | Offe.D | Offensiveness Detection
CEG | Causal Explanation Generation | OKEP | Ophthalmic Knowledge Assessment Program
CFA | Chinese-Featured Ability | Overa. | Overall
Chem.Exam | Chemistry Exam | Pass.R | Qassage Re-ranking
Com.R | Commonsense Reasoning | Pers.D | Personality Detection
Coun.R | Counterfactual Reasoning | Phys.Exam | Physics Exam
CS Exam | Computer Science Exam | PoS | Part-of-Speech Tagging
D2TE | Data-to-Text Generation Evaluation | Prag. | Pragmatic Processing Tasks
DGE | Dialogue Generation Evaluation | Prof.A | Professional Ability
Diagno. | Diagnosis | QA | Question Answering
Dial.S | Dialogue Systems | RE | Relation Extraction
DLR | Dialogue Logic Reasoning | Reas. | Reasoning
Edu.Ling. | Linguistic Quality in Education | Sarc.D | Sarcasm Detection
Edu.Sci. | Scientific Accuracy in Education | Sem. | Semantic Processing Tasks
EE | Event Extraction | Sent.A | Sentiment Analysis
Emoj.P | Emoji Prediction | SGE | Story Generation Evaluation
Emot.R | Emotion Recognition | Soci.S | Social Science
Eval. | Evaluation | SpamD | Spam Detection
Form.S | Formal Science | Stan.D | Stance Detection
Gene. | Generation Tasks | STEM | Science, Tech., Engineering, and Mathematics
Huma. | Humanity | Suic.D | Suicide Detection
Humo.R | Humour recognition | Summ. | Summarisation
Info.R | Information Retrieval | Synt. | Syntactic Processing Tasks
LeetCo. | LeetCode | T-Lo.R | Task-Oriented Logical Reasoning
LID | Language Identification | T-Sy.R | Task-oriented Symbolic Reasoning
Ling.A | Linguistic Acceptability | T.Acc | Tweet Annotation Accuracy
Logi.R | Logical Reasoning | T.Agre | Tweet Annotation Agreement
LSAT | Law School Admission Test | T2SQL | Text-to-SQL
Mach.T | Machine Translation | TSE | Text Summarisation Evaluation
Med.Kno. | Medical Knowledge | TUCE | Test of Understanding of College Economics
Medi.S | Medical Science | USMLE | United States Medical Licensing Examination
Misi.D | Misinformation Detection | WSD | Word Sense Disambiguation
Table 7: Task abbreviations and meanings.

Reference | Evaluation task | Version | Date | Evaluation method | Evaluation method novelty
---|---|---|---|---|---
Cabrera and Neubig (2023) | Dialogue fluency | CG, G4 | 18/05 | Compare to zero-shot LLMs on benchmark data | Unified interface for open-source LLMs; Online browsing and analysis
Bang et al. (2023) | Summ., Mach.T, Sent.A, QA, Misi.D | CG | 28/02 | Compare to SOTA on benchmark data | Multitask, multilingual, multimodal evaluation
Hendy et al. (2023) | Machine translation | CG | 18/02 | Compare to SOTA and other LLMs on benchmark data | Evaluating by prompting strategies, languages, document-level and domain robustness
Jiao et al. (2023) | Machine translation | CG, G4 | 19/03 | Compare to commercial MT systems on benchmark data | Testing robustness of MT systems on domain biases and noise
Karpinska and Iyyer (2023) | Machine translation | CG, G4 | 22/05 | Compare to commercial MT systems on literacy texts | Conducting large scale human evaluation by experts and error analysis
Qin et al. (2023) | NLP and reasoning tasks | CG | 15/02 | Compare to other LLMs and supervised models on benchmark data | A wide range of task-oriented reasoning tasks
Zhang et al. (2023a) | Summarisation | CG | 09/04 | Compare to SOTA on benchmark data | Exploring several ways to enhance the performance
Soni and Wade (2023) | Summarisation | CG | 30/04 | Compare to humans on benchmark data | Humans and a fine-tuned classifier try to distinguish GPT generated summary from references
Jentzsch and Kersting (2023) | NLG | CG | 07/06 | Analyzed by case study and statistics | Evaluated ChatGPT in joke generation, detection and explanation
Amin et al. (2023) | Pers.D, Sent.A, Suic.D | CG | 01/03 | Compare to naive supervised models on benchmark data | Using transparent models; Novel prompts
Kocoń et al. (2023) | 25 sem. and prag. tasks | CG | 21/02 | Compare to supervised models on benchmark data | Analyzing ChatGPT by task difficulties, contexts, and explainability
Wei et al. (2023) | RE, NER, EE | CG | 20/02 | Compare to supervised models on benchmark data | Comparing to few-shot learning baselines with diverse shot values
Sun et al. (2023) | Document ranking | CG, G4 | 19/04 | Compare to BM25 and SOTA on benchmark data | Testing three ranking strategies, namely query-, relevance- and permutation-generation
Wang et al. (2023c) | NLG evaluation | CG | 25/04 | Compare to SOTA on benchmark data | A wide range of NLG tasks
Liu et al. (2023c) | NLG evaluation | G4 | 06/04 | Compare to SOTA on benchmark data | A wide range of NLG tasks
Kocmi and Federmann (2023) | Machine translation evaluation | CG | 28/02 | Compare to SOTA and other LLMs on benchmark data | Testing four distinctive prompts for machine translation evaluation
Gilardi et al. (2023) | Text classification annotation | CG | 27/03 | Compare to humans on annotation tasks | Comparing ChatGPT with crowd-workers in annotation accuracy and agreements
Xu et al. (2023) | Multi-level language abilities | CG, G4 | 18/05 | Compare to other LLMs on benchmark data | Evaluating language abilities from different aspects
Huang et al. (2023) | Multi-discipline knowledge in Chinese | CG, G4 | 17/05 | Compare to other LLMs on benchmark data | Scientific knowledge benchmarking at multi-levels
Lai et al. (2023) | Multilingual NLP | CG | 12/04 | Compare to supervised models on benchmark data | A wide range of multilingual benchmarking tasks with 13 languages
Wang et al. (2023b) | Multilingual summarization | CG, G4 | 06/04 | Compare to fine-tuned mBART-50 and other LLMs on benchmark data | Comparing different prompting methods
Borji (2023) | Reasoning, linguistics, and perception. | CG | 03/04 | Analyzed by case study | Collecting ChatGPT failures cases from different tasks
Davis (2023) | Commonsense | CG | 22/02 | Analyzed by case study | Reviewing commonsense benchmarks in multiple modalities
Gao et al. (2023) | Causal reasoning | CG, G4 | 25/04 | Compare to SOTA, other LLMs and humans on benchmark data | Fine-grained causal reasoning benchmarking tasks
Kıcıman et al. (2023) | Causal reasoning | CG, G4 | 08/05 | Compare to SOTA and other LLMs on benchmark data | Fine-grained causal reasoning benchmarking tasks
Kosinski (2023) | Theory of Mind | CG, G4 | 14/03 | Compare to ground truth on self-developed questions | Using a number of manmade false and belief tasks
Moghaddam and Honey (2023) | Theory of Mind | CG, G4 | 26/04 | Compare to ground truth on self-developed questions and humans | Using a number of manmade false and belief tasks
Holterman and van Deemter (2023) | Theory of Mind | CG, G4 | 23/05 | Compare to ground truth on self-developed questions | Considering 4 different types of Theory of Mind tasks
Wang et al. (2023a) | Dialectical evaluation | CG | 22/05 | Measure the failure rates on benchmark data | A novel ChatGPT’s deficiency evaluation task with diverse reasoning setups
Liu et al. (2023b) | Reading comprehension and NLI | CG, G4 | 05/05 | Compare to fine-tuned RoBERTa base and humans on benchmark data | Developing a toolkit for prompt-based LLM testing
Liu et al. (2023a) | NLI | CG, G4 | 27/04 | Compare to SOTA on benchmark data | A novel NLI task to evaluate the abilities of ChatGPT and GPT-4 in detecting ambiguous language
Bubeck et al. (2023) | Coding, math, perception abilities | G4 | 13/04 | Compare to SOTA and humans on benchmark data and analyze by cases | Thoroughly evaluating GPT-4 abilities in diverse scenarios
Bordt and von Luxburg (2023) | Math | CG | 22/05 | Compare to ground truth on an exam | Blind evaluation together with 200 participating students.
Li et al. (2023b) | Text-to-SQL | CG | 04/05 | Compare to SOTA models and humans on benchmark data | Developing a large and difficult text-to-SQL benchmarking dataset
Frieder et al. (2023) | Math | CG | 31/01 | Compare to ground truth on benchmark data | Providing a large natural-language mathematics testing-oriented dataset
Kortemeyer (2023) | Physics | CG | 11/05 | Compare to ground truth on multiple testing modules | Evaluating ChatGPT by homework, clicker, programming exercises, and exams
Clark (2023) | Chemistry | CG | 24/04 | Compare to humans on multiple testing modules | Evaluating ChatGPT by closed- and open-response questions
Dahlkemper et al. (2023) | Education | CG | 12/05 | Compare to experts on 3 physics questions | Evaluating student perceptions about ChatGPT and expert answers by scientific and language accuracy
Choi et al. (2023) | Law | CG | 23/01 | Compare to ground truth on an exam | Blind evaluation based on four law class exams
Geerling et al. (2023) | Economics | CG | 28/03 | Compare to ground truth on an exam | Blind evaluation based on TUCE
Xie et al. (2023) | Finance | CG | 28/04 | Compare to SOTA on benchmark data | Evaluating ChatGPT by single- and multi-module setups in stock prediction
Gilson et al. (2023) | Medicine | CG | 08/02 | Compare to ground truth on an exam | Analyzing by question difficulty and response quality.
Kung et al. (2023) | Medicine | CG | 09/02 | Compare to ground truth on an exam | Analyzing by concordance and density of insight
Antaki et al. (2023) | Medicine | CG | 11/06 | Compare to ground truth on an exam | Analyzing by examination sections, cognitive level, and question difficulty
Hirosawa et al. (2023) | Medicine | CG | 15/02 | Compare to humans on 30 clinical vignettes | Comparing ChatGPT with physicians by top 5 and top 1 suggestions
Rao et al. (2023) | Medicine | CG | 26/02 | Compare to ground truth on 36 clinical vignettes | Evaluating ChatGPT throughout the entire clinical workflow
Mehnen et al. (2023) | Medicine | CG | 27/04 | Compare to ground truth on 50 clinical vignettes | Evaluating ChatGPT by common and rare cases
Chu and Liu (2023) | Generation | CG | 05/04 | Analyzed by questionnaires | A psychological study of stories generated by ChatGPT
Table 8: The summary of our surveyed works (Part I). The date denotes the
available online date in 2023. CG denotes ChatGPT. G4 denotes GPT-4.

Reference | Evaluation task | Version | Interesting findings
---|---|---|---
Cabrera and Neubig (2023) | Dialogue fluency | CG, G4 | A chat-tuned model should be leveraged; As the context increases, the utility of prompt engineering became less prominent; Hallucinations and repetitive content
Bang et al. (2023) | Summ., Mach.T, Sent.A, QA, Misi.D | CG | ChatGPT had basic multimodality ability in drawing, albeit with remarkable improvements when provided with a self-generated textual description
Hendy et al. (2023) | Machine translation | CG | ChatGPT had competitive performance for high-resource languages but had limited capabilities for low-resource languages
Jiao et al. (2023) | Machine translation | CG, G4 | ChatGPT performed badly on low resource or distant languages and had lower robustness. GPT-4 could narrow the gap
Karpinska and Iyyer (2023) | Machine translation | CG, G4 | ChatGPT overwhelmingly outperformed Google Translation when doing paragraph-level translation if evaluated by expert translators
Qin et al. (2023) | NLP and reasoning tasks | CG | ChatGPT was good at NLP tasks with intensive reasoning, while weak in sequence tagging tasks; ChatGPT was weaker than GPT-3.5 in commonsense, symbolic, and logical reasoning tasks
Zhang et al. (2023a) | Summarisation | CG | An extract-then-generate pipeline with ChatGPT yielded significant performance improvements over abstractive baselines in terms of summary faithfulness
Soni and Wade (2023) | Summarisation | CG | While text classification algorithms could distinguish between real and generated summaries, humans were unable to distinguish between real summaries and those produced by ChatGPT
Jentzsch and Kersting (2023) | NLG | CG | ChatGPT likely generated repetitive jokes. Out of all the 1008 generated jokes, 90.1% of them matched one of the top 25 jokes
Amin et al. (2023) | Pers.D, Sent.A, Suic.D | CG | ChatGPT’s performance was comparable to BoW and lacked significant improvement
Kocoń et al. (2023) | 25 sem. and prag. tasks | CG | More user context annotations led to better ChatGPT performance
Wei et al. (2023) | RE, NER, EE | CG | Multi-turn QA improved ChatGPT information extraction accuracy
Sun et al. (2023) | Document ranking | CG, G4 | Permutation generation strategy was more supportive for ChatGPT and GPT-4 on ranking tasks
Wang et al. (2023c) | NLG evaluation | CG | The ChatGPT evaluator exhibited sensitivity towards prompts; The presence of inherent biases of evaluation datasets could potentially weaken ChatGPT evaluator
Liu et al. (2023c) | NLG evaluation | G4 | The GPT-4 evaluator exhibited sensitivity towards prompts; LLM-based metrics preferred LLM-generated text to human-generated text
Kocmi and Federmann (2023) | Machine translation evaluation | CG | ChatGPT frequently yielded evaluation scores with explanations
Gilardi et al. (2023) | Text classification annotation | CG | ChatGPT yielded higher agreement rates than crowd-workers
Xu et al. (2023) | Multi-level language abilities | CG, G4 | GPT-4 and ChatGPT achieved human-level accuracy on five basic language ability tests
Huang et al. (2023) | Multi-discipline knowledge in Chinese | CG, G4 | CoT did not consistently enhance outcomes across various subjects, particularly in cases where the subjects did not heavily rely on reasoning abilities
Lai et al. (2023) | Multilingual NLP | CG | Despite processing tasks in other languages, English prompts helped ChatGPT achieve better results
Wang et al. (2023b) | Multilingual summarization | CG, G4 | Certain multi-lingual and bilingual LLMs exhibited constrained zero-shot cross-lingual summarization capabilities
Borji (2023) | Reasoning, linguistics, and perception. | CG | It is necessary to use a standardized set of questions to continuously evaluate the progress of ChatGPT over time, thereby moving away from subjective opinions
Davis (2023) | Commonsense | CG | ChatGPT’s commonsense understanding showed more flaws after “pressing on its weakness"
Gao et al. (2023) | Causal reasoning | CG, G4 | Suffer causal hallucinations; Strong in causal interpretation, but weak in causal reasoning
Kıcıman et al. (2023) | Causal reasoning | CG, G4 | ChatGPT and GPT-4 generally surpassed SOTA causal algorithms in graph discovery and counterfactual inference
Kosinski (2023) | Theory of Mind | CG, G4 | GPT-3.5 solved 90% of false-belief tasks, at the level of seven-year-olds. GPT-4 solved nearly all the tasks
Moghaddam and Honey (2023) | Theory of Mind | CG, G4 | Appropriate prompting enhanced LLM ToM reasoning, and they underscored the context-dependent nature of LLM cognitive capacities
Holterman and van Deemter (2023) | Theory of Mind | CG, G4 | ChatGPT-4 outperformed ChatGPT-3 overall on six different ToM principles
Wang et al. (2023a) | Dialectical evaluation | CG | The belief (and disbelief) of ChatGPT lacked robustness and was susceptible to being perturbated by users
Liu et al. (2023b) | Reading comprehension and NLI | CG, G4 | RoBERTa largely outperformed ChatGPT and GPT-4 on the MNLI dataset; GPT-4 exceeded the average level of human LSAT
Liu et al. (2023a) | NLI | CG, G4 | Language ambiguity detection was a very difficult task for LLMs
Bubeck et al. (2023) | Coding, math, perception abilities | G4 | GPT-4 achieved human-level accuracy, and it consistently outperformed previous models across various evaluation metrics
Bordt and von Luxburg (2023) | Math | CG | ChatGPT achieved a minimum passing score on the exam, while GPT-4 slightly surpassed the performance of average students
Li et al. (2023b) | Text-to-SQL | CG | Execution accuracy was less than half that of humans; Wrong schema linking and misunderstanding database content
Frieder et al. (2023) | Math | CG | ChatGPT could not demonstrate consistent capability in providing reliable proofs or calculations in mathematics. It varied by the complexity of the mathematical questions
Kortemeyer (2023) | Physics | CG | ChatGPT just passed the physics exam with a weakness in mathematics
Clark (2023) | Chemistry | CG | ChatGPT’s chemistry knowledge was lower than the average on closed-response questions and ranked last in open-response questions
Dahlkemper et al. (2023) | Education | CG | ChatGPT’s scientific accuracy was rated similarly to that of the expert solution in a hard question, despite the presence of misleading answers
Choi et al. (2023) | Law | CG | ChatGPT passed all four law classes, yet its performance ranked among the lowest within each class
Geerling et al. (2023) | Economics | CG | ChatGPT ranked top 9% and top 1% among thousands of college students in microeconomics and macroeconomics, respectively
Xie et al. (2023) | Finance | CG | ChatGPT’s performance in multi-modal stock price prediction tasks was constrained, exhibiting lower effectiveness compared to conventional methods in single-modal tests
Gilson et al. (2023) | Medicine | CG | ChatGPT exhibited a performance that aligned with a third-year medical student
Kung et al. (2023) | Medicine | CG | ChatGPT achieved a minimum passing score on the exam
Antaki et al. (2023) | Medicine | CG | ChatGPT did not grasp specialized knowledge well in ophthalmology
Hirosawa et al. (2023) | Medicine | CG | ChatGPT’s top 10 differential-diagnosis lists were acceptable, while the accuracy of its top 1 result significantly lagged behind that of physicians
Rao et al. (2023) | Medicine | CG | ChatGPT achieved higher accuracy on the final diagnoses, compared to the initial diagnoses
Mehnen et al. (2023) | Medicine | CG | GPT-4 needed to propose more than 8 suggestions to diagnose 90% rare cases
Chu and Liu (2023) | Generation | CG | ChatGPT wrote more engaging and persuasive short stories than their human counterparts. However, the conclusion was the opposite if the aim was long stories
Table 9: The summary of our surveyed works (Part II).