# From Data to Commonsense Reasoning: The Use of Large Language Models for
Explainable AI

Stefanie Krause<EMAIL_ADDRESS>Frieder Stolzenburg  fstolzenburg@hs-
harz.de

###### Abstract

Commonsense reasoning is a difficult task for a computer, but a critical skill
for an artificial intelligence (AI). It can enhance the explainability of AI
models by enabling them to provide intuitive and human-like explanations for
their decisions. This is necessary in many areas especially in question
answering (QA), which is one of the most important tasks of natural language
processing (NLP). Over time, a multitude of methods have emerged for solving
commonsense reasoning problems such as knowledge-based approaches using formal
logic or linguistic analysis.

In this paper, we investigate the effectiveness of large language models
(LLMs) on different QA tasks with a focus on their abilities in reasoning and
explainability. We study three LLMs: GPT-3.5, Gemma and Llama 3. We further
evaluate the LLM results by means of a questionnaire. We demonstrate the
ability of LLMs to reason with commonsense as the models outperform humans on
different datasets. While GPT-3.5’s accuracy ranges from 56% to 93% on various
QA benchmarks, Llama 3 achieved a mean accuracy of 90% on all eleven datasets.
Thereby Llama 3 is outperforming humans on all datasets with an average 21%
higher accuracy over ten datasets.

Furthermore, we can appraise that, in the sense of explainable artificial
intelligence (XAI), GPT-3.5 provides good explanations for its decisions. Our
questionnaire revealed that 66% of participants rated GPT-3.5’s explanations
as either ”good” or ”excellent”. Taken together, these findings enrich our
understanding of current LLMs and pave the way for future investigations of
reasoning and explainability.

###### keywords:

large language models , explainable AI , commonsense reasoning , question
answering

††journal: Engineering Applications of Artificial Intelligence

organization=Faculty of Automation and Computer Science, Harz University of
Applied Sciences, addressline=Friedrichstr. 57-59, city=38855 Wernigerode,
country=Germany

##  1 Introduction

Commonsense knowledge encompasses everyday assumptions about the world [53].
It is typically acquired through personal experience but can also be inferred
by generalizing from common knowledge [44]. Humans employ a diverse range of
knowledge and reasoning to comprehend the meanings of language. [45].
Commonsense reasoning enables us to link various pieces of knowledge to draw
new conclusions [45]. For instance, consider the sentence from the commonsense
reasoning dataset CODAH [5], ”Sally thought the test was a piece of cake.” It
is easy for us to infer that Sally found the test easy. While this kind of
knowledge and reasoning is instinctive for human readers, it remains
challenging for machines [45].

LLMs play a crucial role in developing adaptable, general language systems
[4]. Lately, a media hype was triggered by the LLM
ChatGPT.111https://chat.openai.com/ This model from OpenAI provides a user-
friendly interface and excels across various tasks, e.g., businesses can
rapidly and accurately address customer inquiries, thereby freeing up
resources and delivering a more personalized customer experience [10]. In the
education sector, students use this new technology for assignment writing and
exam preparation among other tasks [23]. In healthcare, LLMs can be used,
e.g., for effective medical documentation [18], in legal departments as tax
attorney [34] or in combination with robotics [50, 54]. To facilitate the
application of AI across diverse domains, the need for explainability in these
systems is becoming more crucial. According to [3], it is essential to
consider the target audience when seeking explainability. They define XAI as
follows: “Given an audience, an XAI is one that produces details or reasons to
make its functioning clear or easy to understand.” Explainability refers to
presenting a model’s outcome in a human-readable format, such as through
explanatory text. In our work, we focus on explainability of AI models in the
above sense with the goal of XAI to provide human-readable explanations to
make users understand the automated decision-making of large language models a
posteriori.

There is a strong link between XAI and commonsense reasoning, as both focus on
enhancing the explainability of AI models. Commonsense reasoning can enhance
the explainability of AI models by enabling them to provide intuitive and
human-like explanations for their decisions. According to [9], starting with a
better understanding of human cognition is a solid foundation. Humans use
cognitive reasoning to draw meaningful conclusions despite incomplete and
inconsistent knowledge [15]. For us, cognitive reasoning is particularly
useful when we encounter new situations that are not covered by formal rules
or guidelines. In these situations, we rely on our commonsense to make
judgments and decisions that are appropriate and effective. Furthermore,
commonsense reasoning is essential in interpersonal interactions and
communication, as it allows us to understand the perspectives of others and to
navigate social situations effectively. Commonsense reasoning can help AI
models to be more robust in the context of novel situations. A model that can
reason based on commonsense principles is better equipped to handle situations
that it has not explicitly encountered before, as it can draw on its general
understanding of the world to make informed decisions. So far, commonsense
reasoning is intuitive for humans but has been a long-term challenge for AI
models.

We assume that an LLM can reason similar to humans without the need of logical
formulas or explicit ontology knowledge. Recent advances in LLMs (e.g. [28])
have pushed machines closer to human-like understanding capabilities. We
believe that language comprehension and commonsense reasoning do not require
formal structures, although they eventually may provide a better understanding
afterwards for humans. Instead, we assume that LLMs are the appropriate way
towards human-like ability to reason as well as explain AI decisions. To
tackle the growing demand for explainable AI systems we aim to show that LLM
generated explanations are helpful for users to understand AI decisions. There
is no specific structure of learning necessary: LLMs can generate human-like
explanations a posteriori. For this reason, we formulate the following
hypotheses:

  1. 1.

LLMs can handle commonsense reasoning in question answering tasks with near-
human-level performance.

  2. 2.

LLMs are able to generate good, human-understandable explanations for their
decisions.

We continue our paper by giving an overview of important research directions
in Section 2. Then, we evaluate the performance of the recent LLMs GPT-3.5,
Gemma and Llama 3 on commonsense reasoning tasks in Section 3. Since measuring
explainability is still a problem we address this by first testing the LLMs on
eleven QA datasets where commonsense capabilities are required. Using a random
sample of each benchmark dataset, we subsequently evaluate the quality of
GPT-3.5’s responses with a questionnaire (Section 4).

The main contributions of this paper are described in Section 5 and can be
summarized as following:

  * 1.

evaluation of GPT-3.5’s, Gemmas’s and Llama 3’s ability to perform commonsense
reasoning

  * 2.

quality measurement of GPT-3.5’s explanations by a questionnaire

##  2 Related Work

###  2.1 Commonsense Reasoning Approaches

Commonsense reasoning is a difficult task for a computer to handle [41]. To
address this problem, various approaches have been followed in the past.
McCarthy [30] was the first who outlined the basic approach of representing
commonsense knowledge with predicate logic. Symbolic logic approaches were the
main representation type, see e.g. [13, 25]. While still in use today [7] for
this extremely complex task to work well it requires a large amount of
additional logical scaffolding to precisely define the terms used in the
statement and their interrelationships [27].

There is a big gap between the logical approach with deductive reasoning and
human reasoning, which is largely inductive, associative, and empirical, i.e.,
based on former experience. Human reasoning, in contrast to formal logical
reasoning, does not strictly follow the rules of classical logic. There have
been efforts to utilize an approach which uses an automatic theorem prover
(that allows to derive new knowledge in an explainable way), large existing
ontologies with background knowledge, and recurrent networks with long short-
term memory (LSTM) [20] but still did not stand out much from the baseline
[41].

Recent efforts to acquire and represent commonsense knowledge resulted in
large knowledge graphs, acquired through extractive methods [43] or
crowdsourcing [39]. Some approaches use supervised training on a particular
knowledge base, e.g., ConceptNet for commonsense knowledge. ConceptNet is a
crowd-sourced database that represents commonsense knowledge as a graph of
concepts connected by relations [43].

Interestingly, LLMs (cf. Section 2.2) do not contain any explicit semantic
knowledge or grammatical let alone logical rules that would allow an explicit
reasoning process, not even the large ontologies from the logical knowledge
representation like Cyc [26] or Adimen-SUMO [1]. A way out might be to have
neural networks learn reasoning explicitly, possibly by focusing on certain
sentence forms as in syllogistic reasoning maybe implemented with neural-
symbolic cognitive reasoning by specifically structured neural networks [16,
17, 55]. In contrast to simple deep learning, information from different
places and/or documents must be merged here in any case. It does not suffice
to investigate any local text properties, e.g., determining the text form.

There are different types of commonsense reasoning, e.g., causal, temporal,
physical, social etc. (see Section 3.1), each with its unique characteristics
and applications. Understanding how these different reasoning tasks can be
solved with LLMs is essential.

###  2.2 LLMs

In the past, most deep learning methods used supervised learning and therefore
required substantial amounts of manually labelled data. Recent research has
shown that learning good representations in an unsupervised fashion can
provide a significant performance boost. The capacity of LLMs is essential to
the success of zero-shot task transfer [37]. An example of a premier LLM that
can handle a wide range of natural language processing tasks is OpenAI’s GPT-3
[4]. GPT-3 (Generative Pre-trained Transformer) is a third-generation,
autoregressive language model that uses unsupervised learning to produce
human-like text. In our further investigation, we will focus on version
GPT-3.5 with 175B parameters, as this chatbot based model is regarded as one
of the most groundbreaking LLMs.

We further utilise the open-source model Meta-Llama-3-70B-Instruct by Meta.
This is a instruction-tuned version with 70 billion parameters [48, 32].
Nevertheless, we use the very lightweight model with only 7 billion parameters
”Gemma-1.1-7b-it” by Google. This model is open-source and built for
responsible AI development based on the same research and technology used to
create Gemini models [47]. We carefully chose these 3 LLMs, as they are state-
of-the-art, differ in size (see Table 1) and are mostly open-source.

Table 1: LLM size comparison with respect to number of parameters/neurons in unit B = billion. Model | billion parameters
---|---
GPT-3.5 | 175B
Llama 3 | 70B
Gemma | 7B

###  2.3 Combining LLMs and Reasoning

Reasoning is one of the most actively discussed and debated capabilities of
LLMs. However, it is important to note that proficiency in language does not
necessarily equate to strong reasoning abilities in models [29]. Experts
continue to debate the ability of LLMs to reason effectively in zero-shot
scenarios [12]. While some researchers argue that LLMs demonstrate
satisfactory zero-shot reasoning capabilities [22], others contend that the
models struggle with planning and reasoning tasks [49].

ChatGPT, for example, is often seen as an unreliable reasoner due to issues
such as hallucinations, a problem common to many LLMs [2]. Specifically,
ChatGPT-3.5 faces significant challenges in certain areas, such as mathematics
[12]. The mathematical performance of both GPT-3.5 and GPT-4 falls well below
the level of a graduate student [14]. Attempts to combine elementary
mathematics with commonsense reasoning have shown that no existing AI systems
can reliably solve these problems [8]. Thus, we investigate the reasoning
ability of different current LLMs in a fine-grained manner, which includes
causal, temporal, comparative, physical, social, numerical reasoning, etc.,
via question-answering tasks.

##  3 Evaluating LLMs on QA Tasks

We assess the LLMs twofold: First, we evaluate the accuracy of GPT-3.5, Gemma
and Llama 3 on QA benchmarks with multiple-choice questions. Please note that
the analysis of GPT-3.5 has been taken from our previous work in 2023 [24].
The evaluation of Gemma and Llama 3 was conducted in May 2024. In the
commonsense reasoning benchmarks we considered, the correct answer is
indicated, although it is not always clear whether this answer really is
objectively the best one. Second, we use part of the questions from the QA
benchmarks for a questionnaire to evaluate the quality of all three LLM
responses compared to human accuracy. Furthermore, we evaluate the explanation
quality of GPT-3.5 with the help of our questionnaire.

###  3.1 Benchmark Datasets

We utilize 11 publicly available benchmark datasets carefully designed to be
difficult to solve without commonsense knowledge. From each dataset, we select
30 random examples, covering different QA tasks like text completion,
understanding of cause and effect or temporal relationships. In addition,
different fields like medicine, physics, and everyday life situations are
covered. We evaluate the performance of the LLMs with the QA benchmarks
presented in Table 2.

Table 2: Overview of 11 datasets for commonsense reasoning. For each dataset we report the year the dataset was published, the number of choices each QA task in the respective benchmark has, and the knowledge domain or reasoning scenario. dataset | year | choices | knowledge domain or reasoning scenario
---|---|---|---
Story Cloze Test [33] | 2017 | 2 | causal, temporal
CREAK [35] | 2021 | 2 | physical, social
CODAH [5] | 2019 | 4 | situations observed in videos
COM2SENSE [42] | 2021 | 2 | causal, comparative, temporal, social, numerical
CosmosQA [21] | 2019 | 4 | causal
e-CARE [11] | 2022 | 2 | causal
ARC [6] | 2018 | 4 | natural, grade-school science
Social IQa [40] | 2019 | 3 | social, comparative
COPA [38] | 2011 | 2 | causal
MedMCQA [36] | 2022 | 4 | medical subjects
CommonsenseQA [46] | 2018 | 5 | semantic relations

###  3.2 LLM Results on Different Datasets

Using 30 randomly selected examples from each dataset we tested the respective
QA tasks with GPT-3.5, Gemma and Llama 3. We analysed the accuracy and further
differentiate between incorrect and invalid answers. Invalid means that the
LLM does not respond which answer option is correct and instead asks for
further context information, see Fig. 1 for an example.

Figure 1: Example for a invalid response from GPT-3.5 due to insufficient
context information (COPA example 612).

However, invalid answers have only been an issue with the COPA dataset and the
models GPT-3.5 and Llama 3. GPT-3.5 (33.33% invalid responses) has a lot more
troubles when short context is provided than Llama 3 (3.33% invalid
responses).

Table 3 shows the accuracy of the three LLMs over all datasets. GPT-3.5 and
Gemma performs overall on a similar level with around 72% overall accuracy,
outperformed by Llama 3 with nearly 90% accuracy.

Table 3: Results for 11 datasets of commonsense reasoning. For each dataset we report the accuracy of GPT-3.5, Gemma and Llama 3 on 30 randomly selected examples per dataset. dataset | GPT-3.5 | Gemma | Llama 3
---|---|---|---
Story Cloze Test [33] | 93.33% | 83.33% | 100.00%
CREAK [35] | 86.67% | 86.67% | 93.33%
CODAH [5] | 80.00% | 66.67% | 97.67%
COM2SENSE [42] | 76.67% | 76.67% | 93.33%
CosmosQA [21] | 76.67% | 66.67% | 100.00%
e-CARE [11] | 76.67% | 66.67% | 80.00%
ARC [6] | 70.00% | 70.00% | 93.33%
Social IQa [40] | 66.67% | 73.33% | 83.33%
COPA [38] | 63.33% | 83.33% | 93.33%
MedMCQA [36] | 60.00% | 60.00% | 93.33%
CommonsenseQA [46] | 56.67% | 46.67% | 60.00%
overall | 73.33% | 70.91% | 89.70%

We found that all models have the lowest performance on CommonsenseQA dataset
with less then 60% accuracy. The CommonsenseQA dataset has with 5 the most
answer options and uses semantic relations. The medical dataset MedMCQA was
very difficult for GPT-3.5 and Gemma (both 60% accuracy), however, Llama 3 did
perform quite impressive on this dataset with 93% accuracy. Similar results
for the ARC dataset, where GPT-3.5 and Gemma both achieved 70% accuracy, while
Llama 3 outperforms the models with 93% accurate answers. The best performance
was achieved by Llama 3 with 100% accuracy on the CosmosQA and the Story Cloze
Test. For the Story Cloze Test dataset GPT-3.5 achieved its best results with
93% accuracy and Gemma reaching 83% accuracy. Before we start with the
analysis of the results, we should acknowledge that we did not take into
account the stochastic elements of LLMs, wherein repeated queries might result
in varying answers.

###  3.3 Analysis of LLM Results

In our error analysis, we identified seven types of challenges where some LLMs
still face problems:

  1. 1.

missing context: When GPT-3.5 lacks sufficient context, it may occasionally be
unable to provide an answer to the QA task. This has happened 10 times in
total and solely with examples of the COPA dataset. This could be due to the
very short premise texts in the COPA dataset, see Fig. 1. In this dataset, the
premise texts consist of only five to nine words (on average six words) in the
cases where GPT-3.5 complained about not having enough information to answer
the question. In some of those cases, GPT-3.5 reveals which context
information is missing: “The actual outcome would depend on a variety of
factors, such as the political climate, the credibility of the politician, and
the specific details of the argument in question. Without this information it
is impossible to determine which alternative is more likely.” (COPA example
619).

  2. 2.

comparative reasoning: Both GPT-3.5 and Gemma have problems when more than one
answer option is plausible. This is the case in comparative scenarios in the
COM2SENSE and Social IQa dataset. In such cases, the commonsense reasoner must
explicitly investigate the likelihood of different answer candidates. For the
Social IQa example 26823 “Sasha was throwing a party in her new condo which
they bought a month ago. What does Sasha need to do before this?” GPT-3.5
answers “Turn music on” which is likely but the correct and even more likely
answer is “needed to buy food for the party”.

  3. 3.

subjective reasoning: Some answers depend on the personality of the reasoner,
e.g. Social IQa example 18571: “Alex’s powers were not as strong since he was
just starting out. Alex used Bailey’s powers since hers were stronger. How
would Bailey feel as a result?” the correct answer according to the benchmark
is “good” but instead GPT-3.5 answers “upset” and explains “Bailey may feel
that her powers are being taken advantage of …” which we think is more a
personalized subjective inference instead of a commonsense answer.

  4. 4.

slang, unofficial abbreviations, and youth language: GPT-3.5 and Gemma have
difficulties to understand slang, unofficial abbreviations and youth language
like “subs” for “subscribers” or “yrs” for “years”. This could be observed,
e.g., in Cosmos QA examples 6599 and 5748.

  5. 5.

social situations: We identified a particular difficulty in correctly
understanding social situations in the Social IQa dataset. For instance, when
asked the question “Kai was visiting from out of state and brought gifts for
Quinn’s family. What will Kai want to do next?” GPT-3.5 choose the answer
“needed to leave his hometown” instead of the correct answer option “watch the
opening of gifts” (Social IQa example 6863).

  6. 6.

medical and science domain: The analysis of MedMCQA showed that GPT-3.5 and
Gemma is lacking a deep domain knowledge in the medical field. The answers of
GPT-3.5 were always plausible and explained with a lot of details (on average
43 words per explanation) but 40% were incorrect. This is assumed to be
because of many medical technical terms that are not common knowledge, e.g.,
“Styloglossus muscle” or “Genioglossus muscle” that are different muscles in
the tongue (MedMCQA example 23b363d6-8210-4657-b293-54c9e28bdf31). For non-
medical professionals, these questions are difficult to answer, too.
Furthermore, the natural, grade-school science questions in the ARC dataset
achieved a rather low accuracy of 70% with both GPT-3.5 and Gemma.

  7. 7.

semantic relations: All LLMs had their worst performance on the CommonsenseQA
dataset with five answer options and usage of multiple target concepts that
have the same semantic relation to a single source concept. This indicates
that semantic relations are difficult to understand for the LLMs.

Please be aware that for certain questions to be answered correctly, one must
possess in-depth knowledge rather than commonsense reasoning ability, e.g.,
you have to know that “Prison Break” is a television show, not a movie in a
theatre to tell that “The couple went to the movie theatre to watch Prison
Break” is a wrong statement (CREAK example 98).

##  4 Questionnaire

###  4.1 Design of the Questionnaire

To further evaluate the quality of LLM responses on different benchmark
datasets and to make a comparison to human performance, we created a
questionnaire in our previous work [24]. Since the number of participants was
quite low, we conducted the questionnaire again intending to reach more
participants. For a detailed description of the questionnaire we refer to our
previous work [24]. We used two randomly selected examples for each of the
above mentioned datasets – except for MedMCQA, as we believe these questions
are too challenging for individuals without a medical background to answer.

We developed a publicly open online questionnaire. Participation was
voluntary; participants could not be identified from the material presented
and no plausible harm to participating individuals could arise from the study.
Survey content validity was reviewed in a pretest by one professor, one
academic staff and one non-academic volunteer (business consultant) who did
not participate in developing the survey. The questionnaire was structured in
three parts, the first containing demographic and personal information
(gender, age, nationality, English level). The main part then consists of the
QA tasks of the different datasets as well an evaluation of GPT-3.5
explanations. Note that the survey participants did not know that _all_
explanations have been generated by GPT-3.5. An example of the main
questionnaire section is provided in Fig. 2.

Figure 2: Example of one QA task with the common structure (ARC example). Only
after participants answer the first two questions the next two questions with
the possible response and explanation are shown.

We used this structure for 10 datasets and randomly selected two examples from
every dataset. Therefore we considered $2\cdot 10=20$ QA tasks in our
questionnaire.

###  4.2 Questionnaire Participants

In total, 152 people participated in the questionnaire, but because of missing
data, we only used the responses of 81 participants. The time to fully answer
the whole questionnaire was about 25 minutes, which is probably why several
participants did not complete the questionnaire until the last question. The
participant’s English level was mainly advanced or excellent so there was no
language barrier in understanding the QA tasks. Among the completed
questionnaires, the average age was rather young with 26 years, with a minimum
of 19 years and maximum of 51 years. Most of the participants were from India
or Germany and the rest was from Bangladesh, Pakistan, Finland, Russia,
Turkey, Canada, Iran, Lebanon and Switzerland.

###  4.3 Questionnaire Responses

We found that the participants answered with a mean accuracy of 69.00% on a
subset of QA tasks compared to GPT-3.5 74.33%, Gemma 72.00% and Llama 3 89.00%
(mean accuracy over all datasets, except MedMCQA). Note that 20 in detail
analyzed QA tasks of the questionnaire are not as representative as the over
300 QA tasks from Section 3.2, even though we selected the 20 QA tasks
randomly.

Figure 3: Comparison of accuracy of human questionnaire participants (organge)
and three different LLMs: GPT-3.5, Gemma and Llama 3 (blue) on ten different
datasets.

A comparison of the performance of three different LLMs to the questionnaire
participants on the different datasets is presented in Fig. 3. One can
recognize very quickly that Llama 3 outperforms human accuracy on every
dataset. We found that Llama 3 is outperforming humans impressively by 21%
more accuracy on average over ten datasets. GPT-3.5 and Gemma outperformed
humans on at least six datasets. On the CommonsenseQA dataset the results have
been very similar. The dataset the LLMs perform worst is also the same dataset
humans have the most problems (CommonsenseQA). The greatest difference between
human and LLM performance is on the CosmosQA dataset, where Llama 3
outperforms the survey participants by 45% more accuracy. When comparing human
and GPT-3.5 performance the greatest difference is on the Story Cloze Test and
Cosmos QA datasets. In these two datasets, GPT-3.5 outperforms humans with
over 20% difference in accuracy. It is quite interesting that all LLMs perform
better on Cosmos QA than the survey participants as contextual commonsense
reasoning is needed for this dataset. It focuses on reading between the lines
over a diverse collection of people’s everyday narratives. In contrast, humans
perform a lot better than GPT-3.5 and Gemma on ARC where an understanding of
science questions is needed.

We want to make sure that we recognize any correlations regarding the tasks
comprehensibility influencing the accuracy of answers or the explanation
quality. We found that 82% of the questions used in our questionnaire were
rated ”excellent”, ”good” or ”fair” comprehensible. The SocialIAQ examples
have been rated worst by our human participants. The human accuracy on this QA
task is only 55%, however GPT-3.5 reached 67% accuracy. We aimed to explore
the relationship between the comprehensibility of tasks and GPT-3.5’s
explanations. It is worth noting that most questions of the different QA tasks
are comprehensible according to the participants. We observed that there is a
mean linear positive correlation of 0.64 between the comprehensibility of the
QA tasks and that of GPT-3.5’s explanations. This means that the way the users
comprehend the QA tasks has an impact on the estimated quality of the
explanation from GPT-3.5.

Figure 4: Participants’ rating of GPT-3.5’s explanation quality on a 5 Likert
scale from “very poor” to “excellent”.

The quality of GPT-3,5’s explanations is visualised in Fig. 4. In general,
explanations were mostly rated “good” or “excellent” with 66% and only 42
times very poor. Explanations were rated “fair” or better with 86%.
Furthermore, we found that the explanations for COPA example 610 were often
rated “poor” or “very poor”, and for this example GPT-3.5’s answer was invalid
as it could not decide for one option saying: “It is not specified in the
given information which alternative is the more likely cause.” The average
length of GPT-3.5’s explanations is 38 words for both correct and incorrect
responses. From the optional free text field, we received mostly the same
possible improvement for GPT-3.5’s explanation: GPT-3.5 should explain why the
other answering options are false or less likely and not only focus on
explaining why one option is correct. This is in particular important in
comparative reasoning tasks. Further, the explanation should provide more
information than stated in the question, however, the sentences should not be
too long.

Figure 5: Participants believe how many explanations from zero to twenty are
generated by an AI. Actually all 20 explanations were generated by GPT-3.5.

Fig. 5 shows the survey participants’ guess of how many responses are created
by an AI. In the chart, one can see that the mode is 15 explanations. While
all respondents thinks that at least five explanations are generated by an AI,
the mean amount of AI explanations is 13. Thus all participants believe that
at least 25% of the explanations were AI generated and most believe that more
than half of the explanations are AI generated. In fact, all explanations are
generated with GPT-3.5.

Figure 6: Concerns regarding the use of AI tools mentioned by questionnaire
participants have been categorized into six distinct groups.

We found that most study participants are not very concerned about using AI
(chatbot) tools, the largest number of participants are ”somewhat concerned”
with 41.98% and even 17.28% are ”not concerned at all”, while only 5
participants are ”very concerned”. However, 37 participants expressed multiple
concerns in a free-text field, which we subsequently organized into six
clusters. The results are visualized in Fig. 6.

Most often the negative impact on human skills like critical thinking,
creativity, cheating in education, loss of confidence in own abilities or
dependence on AI tools is mentioned. The second important category of concern
is incorrect answers or explanations. Many participants stated that LLMs can
generate false or misleading information that influence decision making. There
is no fact checking mechanism implemented in the models, which worries the
participants. Worries about data privacy and protection as well a possible
misuse or biased outputs occurred in 14% of the concern responses. Some
participants are worried about the impact on their jobs, especially the
possibility of losing their jobs due to AI replacement. Some fear the rapid
development of the technology, as both users and the government may not be
able to respond adequately.

##  5 Discussion and Future Directions

Over the past, research often focused on logical approaches and large
knowledge graphs to deal with commonsense reasoning. Given that we are
currently in the era of LLMs which have shown substantial performance
improvements across various tasks, we hypothesized that LLMs are capable of
handling commonsense reasoning in QA tasks with almost human-level performance
(Hypothesis 1). As LLMs are trained on a large number of data and produces
human-like text, we assume that it can perform commonsense reasoning without
explicit semantic knowledge or logical rules. To show that we evaluated three
LLMs on eleven different QA benchmark datasets which are difficult to solve
without commonsense reasoning.

Moreover, we evaluated explanations generated with GPT-3.5 by means on an
online questionnaire to investigate how sufficient explanations are to users.
Our Hypothesis 2 is stating that an LLM like GPT-3.5 is able to provide good
explanations to users without the need of explicit formalized knowledge
representation. Most participants are content with GPT-3.5’s explanations.
Thereby apparently the problem of explainability of AI decisions can be
overcome easily.

###  5.1 Main Findings

This study shows that different LLMs reached an overall accuracy of 70.91% -
89.70% on eleven challenging QA datasets that are difficult to handle without
commonsense reasoning. The new Llama 3 model performed a lot better than Gemma
und GPT-3.5 on all eleven commonsense reasoning datasets. While there are
still challenges such as too little context information, semantic relations or
the sciences and medical domain (cf. Section 3.3), GPT-3.5 still outperforms
our questionnaire participants in six out of ten datasets (not considering the
medical dataset MedMCQA). The results of our questionnaire show that
participants answered 69% of the 20 QA tasks correctly and are outperformed by
Llama 3 on every single dataset. We found that Llama 3 is outperforming humans
impressively by an average 21% higher accuracy over ten datasets. Although we
only compared the performance of humans versus LLMs on a small amount of
examples, we believe that the outcome indicates that our Hypothesis 1 is true
and LLMs can handle commonsense reasoning in QA tasks with near-human-level
performance, even outperforming human accuracy.

This research focused as well on assessing the explainability of LLMs,
recognizing the significant importance of addressing the black-box problem.
This is particularly relevant as users need to understand AI decisions. By
means of a web-based questionnaire, we evaluated GPT-3.5’s explanations for 20
QA tasks. We found a mean linear positive correlation of 0.64 between the
comprehensibility of the QA tasks and that of GPT-3.5’s explanations. This
observation is relevant for the way GPT-3.5’s users describe their tasks as it
has an impact on the quality of the explanation they receive. In our
questionnaire, GPT-3.5’s explanations were mostly rated “good” or “excellent”
with 66% and fair or better with 86%. Our Hypothesis 2 that LLMs can generate
good explanations could be confirmed. However, to improve explanations, it is
recommended to not only focus on explaining why one option is correct but also
why other answering options are false or less likely.

###  5.2 Impact on the Field

The development of XAI is facing both scientific and social demands [52], and
scientists aim to achieve this without sacrificing performance. So far, this
grand challenge is mainly dealt with by explicit knowledge, such as knowledge
graphs. However, we found that implicit knowledge in the form of probabilistic
models can generate good explanations. LLMs made significant advancements in
NLP tasks in recent years. Due to the chat function of GPT-3.5, users can
easily ask for explanations to understand the response of the AI system. This
can tackle the lack of explainability and is a quite simple and yet effective
way. Using a questionnaire, we could measure the quality of GPT-3.5’s
explanations.

Moreover, commonsense reasoning is very important for various NLP tasks. It
assesses the relative plausibility of different scenarios and recognizes
causality. Until now, research has focused on mathematical logic and the
formalization of commonsense reasoning knowledge. However, some philosophers,
e.g., Wittgenstein, already claimed that commonsense reasoning knowledge is
unformalizable or mathematical logic is inappropriate [31]. As seen in our
evaluation, LLMs have a good zero-shot performance on different QA tasks that
require commonsense reasoning. Nevertheless, we detected seven problems (cf.
Section 3.3) where some LLMs still have difficulties and further research is
necessary, e.g., little context information, comparative reasoning, knowledge
in the medical and science domain or semantic relations.

Evaluation of LLMs brings AI closer to making a practical impact in the area
of XAI and commonsense reasoning. There are still rich opportunities for novel
AI research to further measure the quality of explanations as well as
opportunities in tackling difficult commonsense reasoning tasks like
CommonsenseQA. In future research, one can also investigate certain domains in
more detail e.g. focus on the medical domain to foster the way for medical LLM
based applications. We believe LLMs have great potential for various
applications in combination with robotics, customer service and even legal.
However, we have to make sure to keep the concerns like incorrect answers and
possible negative impacts on human skills in mind. Furthermore, while the
zero-shot performance of the LLMs was already quite good, the accuracy and
explanation quality might improve using few-shot learning and chain-of-thought
prompting [51, 19].

###  5.3 Limitations

Our study has limitations that need to be acknowledged. The number of survey
participants we included was rather small, which limits generalization of our
results. The average age was 26 years and primarily the participants were
university students. In general, more participants with diverse ages and
nationalities would help to strengthen the results. Furthermore the key
challenge for explainability is to determine what constitutes a ”good”
explanation since this is subjective and depends on context. We evaluated
explanations using a five-level Likert scale from ”very poor” to ”excellent”.
However, we only analyzed 20 explanations of GPT-3.5 and argue that our
Hypothesis 2 (that LLMs can generate good explanations) can be confirmed.
Nevertheless, explainability is very important in the medical field, but we
did not consider the MedMCQA dataset in our questionnaire due to a supposed
lack of participant’s knowledge in medicine.

##  6 Conclusion

The field of AI has made considerable progress towards large-scale models,
especially for NLP tasks. Although the field requires more testing, we argue
that LLMs can be used for commonsense reasoning tasks and as well generate
helpful explanations for users to understand AI decisions. The use of LLMs is
a promising area of research that offers many opportunities to enhance
explainability. However, to unleash their full potential for XAI, it is
crucial to approach the use of these models with caution and to critically
evaluate users concerns. The explainability of LLMs is crucial for many
applications and was an important part of this research. We have shown
important future directions research involving XAI and commonsense reasoning.
Several state-of-the-art LLMs have proven capable of outperforming human
performance on a large number of different QA datasets which require
commonsense reasoning.

Despite the potential of the field of LLMs, important questions remain for a
comprehensive evaluation of LLM explanations. As these key issues are
systematically addressed, the potential real world applications will grow. In
particular, the stochastic aspects of LLMs, where repeated queries may lead to
different answers, should be considered in future work. This would also allow
for a better assessment of the error in the LLM performance estimates.

###  6.1 Acknowledgments

We would like to thank all questionnaire participants as well as Osama
Siddiqui, Muhammad Hassan Salam and Syed Muhammad Aun Raza Zaidi for their
help testing the LLMs on different datasets.

## References

  * [1] Álvez, J., Lucio, P., Rigau, G.: Adimen-SUMO: Reengineering an ontology for first-order reasoning. International Journal on Semantic Web and Information Systems (IJSWIS) 8(4), 80–116 (2012), https://ideas.repec.org/a/igg/jswis0/v8y2012i4p80-116.html
  * [2] Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023
  * [3] Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., Chatila, R., Herrera, F.: Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Information Fusion 58, 82–115 (2020). https://doi.org/10.1016/j.inffus.2019.12.012
  * [4] Brown, T., Mann, B., Ryder, N., Subbiah, M., et al.: Language models are few-shot learners. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems 33 (NeurIPS 2020). pp. 1877–1901 (2020), http://proceedings.neurips.cc/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
  * [5] Chen, M., D’arcy, M., Liu, A., Fernandez, J., Downey, D.: CODAH: An adversarially-authored question answering dataset for common sense. In: Proceedings of the 3rd Workshop on Evaluating Vector Space Representations for NLP. pp. 63–69 (2019), https://www.jaredfern.com/publication/codah/
  * [6] Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., Tafjord, O.: Think you have solved question answering? Try ARC, the AI2 reasoning challenge. CoRR – Computing Research Repository abs/1803.05457, Cornell University Library (2018), https://arxiv.org/abs/1803.05457
  * [7] Davis, E.: Logical formalizations of commonsense reasoning: A survey. Journal of Artificial Intelligence Research 59, 651–723 (2017). https://doi.org/10.1613/jair.5339
  * [8] Davis, E.: Mathematics, word problems, common sense, and artificial intelligence. Bulletin of the American Mathematical Society 61(2), 287–303 (2024)
  * [9] Davis, E., Marcus, G.: Commonsense reasoning and commonsense knowledge in artificial intelligence. Communications of the ACM 58(9), 92–103 (2015). https://doi.org/10.1145/2701413
  * [10] Deng, J., Lin, Y.: The benefits and challenges of ChatGPT: An overview. Frontiers in Computing and Intelligent Systems 2(2), 81–83 (2023). https://doi.org/10.54097/fcis.v2i2.4465
  * [11] Du, L., Ding, X., Xiong, K., Liu, T., Qin, B.: e-CARE: a new dataset for exploring explainable causal reasoning. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 432–446. Association for Computational Linguistics (2022), https://aclanthology.org/2022.acl-long.33/
  * [12] Espejel, J.L., Ettifouri, E.H., Alassan, M., Chouham, E.M., Dahhane, W.: Gpt-3.5 vs gpt-4: Evaluating chatgpt’s reasoning performance in zero-shot learning. arXiv preprint arXiv:2305.12477
  * [13] Forbus, K.D.: Qualitative process theory. Artificial Intelligence 24(1-3), 85–168 (1984). https://doi.org/10.1016/0004-3702(84)90038-9
  * [14] Frieder, S., Pinchetti, L., Griffiths, R.R., Salvatori, T., Lukasiewicz, T., Petersen, P., Berner, J.: Mathematical capabilities of chatgpt. Advances in neural information processing systems 36 (2024)
  * [15] Furbach, U., Hölldobler, S., Ragni, M., Schon, C., Stolzenburg, F.: Cognitive reasoning: A personal view. KI 33(3), 209–217 (2019), https://link.springer.com/article/10.1007/s13218-019-00603-3
  * [16] d’Avila Garcez, A.S., Broda, K., Gabbay, D.M.: Symbolic knowledge extraction from trained neural networks: A sound approach. Artificial Intelligence 125(1-2), 155–207 (2001), https//dx.doi.org/10.1016/S0004-3702(00)00077-1
  * [17] d’Avila Garcez, A., Lamb, L., Gabbay, D.: Neural-Symbolic Cognitive Reasoning. Springer Berlin, Heidelberg (2009)
  * [18] Goyal, S., Rastogi, E., Rajagopal, S.P., Yuan, D., Zhao, F., Chintagunta, J., Naik, G., Ward, J.: Healai: A healthcare llm for effective medical documentation. In: Proceedings of the 17th ACM International Conference on Web Search and Data Mining. pp. 1167–1168 (2024)
  * [19] Gramopadhye, O., Nachane, S.S., Chanda, P., Ramakrishnan, G., Jadhav, K.S., Nandwani, Y., Raghu, D., Joshi, S.: Few shot chain-of-thought driven reasoning to prompt llms for open ended medical question answering. arXiv preprint arXiv:2403.04890
  * [20] Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Computation 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
  * [21] Huang, L., Le Bras, R., Bhagavatula, C., Choi, Y.: Cosmos QA: Machine reading comprehension with contextual commonsense reasoning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 2391–2401. Association for Computational Linguistics (2019), https://aclanthology.org/D19-1243/
  * [22] Kojima, T., Gu, S.S., Reid, M., Matsuo, Y., Iwasawa, Y.: Large language models are zero-shot reasoners. Advances in neural information processing systems 35, 22199–22213 (2022)
  * [23] Krause, S., Panchal, B.H., Ubhe, N.: The evolution of learning: Assessing the transformative impact of generative ai on higher education. arXiv preprint arXiv:2404.10551
  * [24] Krause, S., Stolzenburg, F.: Commonsense reasoning and explainable artificial intelligence using large language models. In: European Conference on Artificial Intelligence. pp. 302–319. Springer (2023)
  * [25] Lenat, D.B.: Cyc. Communications of the ACM 38(11), 33–38 (1995). https://doi.org/10.1145/219717.219745
  * [26] Lenat, D.B.: CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM 38(11), 33–38 (1995)
  * [27] Liu, H., Singh, P.: Commonsense reasoning in and over natural language. In: Knowledge-Based Intelligent Information and Engineering Systems – 8th International Conference, KES 2004. pp. 293–306. LNAI 3215, Springer, Berlin, Heidelberg (2004). https://doi.org/10.1007/978-3-540-30134-9_40
  * [28] Liu, Y., Ott, M., Goyal, N., Du Jingfei, Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: RoBERTa: A robustly optimized BERT pretraining approach. https://arxiv.org/pdf/1907.11692
  * [29] Mahowald, K., Ivanova, A.A., Blank, I.A., Kanwisher, N., Tenenbaum, J.B., Fedorenko, E.: Dissociating language and thought in large language models. Trends in Cognitive Sciences (2024)
  * [30] McCarthy, J.: Programs with common sense (1959), https://www.cs.rit.edu/~rlaz/is2014/files/McCarthyProgramsWithCommonSense.pdf
  * [31] McCarthy, J.: Artificial intelligence, logic and formalizing common sense. In: Thomason, R.H. (ed.) Philosophical Logic and Artificial Intelligence, pp. 161–190. Springer Netherlands, Dordrecht (1989), https://doi.org/10.1007/978-94-009-2448-2_6
  * [32] Meta: Introducing meta llama 3: The most capable openly available llm to date (2024), https://ai.meta.com/blog/meta-llama-3/
  * [33] Mostafazadeh, N., Roth, M., Louis, A., Chambers, N., Allen, J.: LSDSem 2017 shared task: The Story Cloze Test. In: Proceedings of the 2nd Workshop on Linking Models of Lexical, Sentential and Discourse-level Semantics. pp. 46–51 (2017), https://aclanthology.org/W17-0906.pdf
  * [34] Nay, J.J., Karamardian, D., Lawsky, S.B., Tao, W., Bhat, M., Jain, R., Lee, A.T., Choi, J.H., Kasai, J.: Large language models as tax attorneys: a case study in legal capabilities emergence. Philosophical Transactions of the Royal Society A 382(2270), 20230159 (2024)
  * [35] Onoe, Y., Zhang, M.J.Q., Choi, E., Durrett, G.: CREAK: A dataset for commonsense reasoning over entity knowledge, https://arxiv.org/pdf/2109.01653
  * [36] Pal, A., Umapathi, L.K., Sankarasubbu, M.: MedMCQA: A large-scale multi-subject multi-choice dataset for medical domain question answering. ACM Conference on Health (2022), https://arxiv.org/pdf/2203.14371
  * [37] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners. Tech. rep., OpenAI (2019), https://paperswithcode.com/paper/language-models-are-unsupervised-multitask
  * [38] Roemmele, M., Bejan, C.A., Gordon, A.S.: Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In: AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning. pp. 90–95 (2011), https://lmy.de/bGzkH
  * [39] Sap, M., Le Bras, R., Allaway, E., Bhagavatula, C., Lourie, N., Rashkin, H., Roof, B., Smith, N.A., Choi, Y.: Atomic: An atlas of machine commonsense for if-then reasoning. Proceedings of the AAAI Conference on Artificial Intelligence 33(01), 3027–3035 (2019). https://doi.org/10.1609/aaai.v33i01.33013027
  * [40] Sap, M., Rashkin, H., Chen, D., LeBras, R., Choi, Y.: Social IQa: Commonsense reasoning about social interactions. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 4463–4473. Association for Computational Linguistics (2019), https://aclanthology.org/D19-1454/
  * [41] Siebert, S., Schon, C., Stolzenburg, F.: Commonsense reasoning using theorem proving and machine learning. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) Machine Learning and Knowledge Extraction – 3rd IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2019\. pp. 395–413. LNCS 11713, Springer Nature Switzerland, Canterbury, UK (2019), https://doi.org/10.1007/978-3-030-29726-8_25
  * [42] Singh, S., Wen, N., Hou, Y., Alipoormolabashi, P., Wu, T.L., Ma, X., Peng, N.: COM2SENSE: A commonsense reasoning benchmark with complementary sentences. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021\. pp. 883–898. Association for Computational Linguistics (2021), https://aclanthology.org/2021.findings-acl.78
  * [43] Speer, R., Chin, J., Havasi, C.: ConceptNet 5.5: An open multilingual graph of general knowledge. Proceedings of the AAAI Conference on Artificial Intelligence 31(1) (2017). https://doi.org/10.1609/aaai.v31i1.11164
  * [44] Speer, R., Havasi, C., Lieberman, H.: Analogyspace: Reducing the dimensionality of common sense knowledge. In: Aaai. vol. 8, pp. 548–553 (2008)
  * [45] Storks, S., Gao, Q., Chai, J.Y.: Recent advances in natural language inference: A survey of benchmarks, resources, and approaches. arXiv preprint arXiv:1904.01172
  * [46] Talmor, A., Herzig, J., Lourie, N., Berant, J.: CommonsenseQA: A question answering challenge targeting commonsense knowledge, https://arxiv.org/pdf/1811.00937
  * [47] Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M.S., Love, J., et al.: Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295
  * [48] Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al.: Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971
  * [49] Valmeekam, K., Olmo, A., Sreedharan, S., Kambhampati, S.: Large language models still can’t plan (a benchmark for llms on planning and reasoning about change). arXiv preprint arXiv:2206.10498
  * [50] Wang, J., Wu, Z., Li, Y., Jiang, H., Shu, P., Shi, E., Hu, H., Ma, C., Liu, Y., Wang, X., et al.: Large language models for robotics: Opportunities, challenges, and perspectives. arXiv preprint arXiv:2401.04334
  * [51] Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35, 24824–24837 (2022)
  * [52] Xu, F., Uszkoreit, H., Du, Y., Fan, W., Zhao, D., Zhu, J.: Explainable AI: A brief survey on history, research areas, approaches and challenges. In: Tang, J., Kan, M.Y., Zhao, D., Li, S., Zan, H. (eds.) Natural Language Processing and Chinese Computing. pp. 563–574. Springer International Publishing, Cham (2019), https://link.springer.com/chapter/10.1007/978-3-030-32236-6_51
  * [53] Zang, L.J., Cao, C., Cao, Y.N., Wu, Y.M., Cao, C.G.: A survey of commonsense knowledge acquisition. Journal of Computer Science and Technology 28(4), 689–719 (2013)
  * [54] Zeng, F., Gan, W., Wang, Y., Liu, N., Yu, P.S.: Large language models for robotics: A survey. arXiv preprint arXiv:2311.07226
  * [55] Zimmer, M., Feng, X., Glanois, C., Jiang, Z., Zhang, J., Weng, P., Li, D., Hao, J., Liu, W.: Differentiable logic machines. CoRR – Computing Research Repository abs/2102.11529, Cornell University Library (2021), https://arxiv.org/abs/2102.11529, latest revision 2023