# A Survey of Language Model Confidence Estimation and Calibration

${\textbf{Jiahui Geng}}^{1}$, ${\textbf{Fengyu Cai}}^{2}$, ${\textbf{Yuxia
Wang}}^{1}$,
${\textbf{Heinz Koeppl}}^{2}$, ${\textbf{Preslav Nakov}}^{1}$, ${\textbf{Iryna
Gurevych}}^{1}$
1 Mohamed bin Zayed University of Artificial Intelligence
2 Technical University of Darmstadt
{jiahui.geng<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
###### Abstract

Language models (LMs) have demonstrated remarkable capabilities across a wide
range of tasks in various domains. Despite their impressive performance, the
reliability of their output is concerning and questionable regarding the
demand for AI safety. Assessing the confidence of LM predictions and
calibrating them across different tasks with the aim to align LM confidence
with accuracy can help mitigate risks and enable LMs to make better decisions.
There have been various works in this respect, but there has been no
comprehensive overview of this important research area. The present survey
aims to bridge this gap. In particular, we discuss methods and techniques for
LM confidence estimation and calibration, encompassing different LMs and
various tasks. We further outline the challenges of estimating the confidence
for large language models and we suggest some promising directions for future
work.

A Survey of Language Model Confidence Estimation and Calibration

${\textbf{Jiahui Geng}}^{1}$, ${\textbf{Fengyu Cai}}^{2}$, ${\textbf{Yuxia
Wang}}^{1}$, ${\textbf{Heinz Koeppl}}^{2}$, ${\textbf{Preslav Nakov}}^{1}$,
${\textbf{Iryna Gurevych}}^{1}$ 1 Mohamed bin Zayed University of Artificial
Intelligence 2 Technical University of Darmstadt {jiahui.geng,
<EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

LLMs have demonstrated a wide range of capabilities, such as world knowledge
storage, sophisticated language-based reasoning, and in-context learning
Petroni et al. (2019); Wei et al. (2022); Brown et al. (2020), thus paving the
way for large language models (LLMs) exemplified by ChatGPT. However, LLMs do
not consistently achieve good performance. Their predictions still contain
errors and biases Zhao et al. (2021); Wang et al. (2023b), and the generated
content includes hallucinations that do not align with reality Zhang et al.
(2023). To enhance the trustworthiness and transparency of language models,
chain-of-thought (CoT) has been proposed, which allows users to follow the
decision-making process of LMs Wei et al. (2022). Retrieval-augmented
generation (RAG) system Lewis et al. (2020b) combines the generative
capabilities of LLMs with the ability to pull in information from external
databases or the internet. Post-processing methods, including fact-checking
Peng et al. (2023); Min et al. (2023) and revision Gao et al. (2023), also
improve the quality of the generated content.

Another line of work is to assess the model’s confidence or uncertainty in its
decisions, many efforts have been made in this direction Jiang et al. (2021);
Lin et al. (2022, 2023); Manakul et al. (2023). However, existing surveys
either discussed the sources and the applications of uncertainty in natural
language processing Hu et al. (2023); Baan et al. (2023), focusing on specific
tasks Ji et al. (2023); Li et al. (2022), or mainly addressed hallucinations
Zhang et al. (2023). To the best of our knowledge, there is no comprehensive
survey that systematically discusses the technical advancements in confidence
estimation and calibration across various LMs. This survey aims to bridge this
gap.

This paper summarizes research related to confidence estimation and model
calibration in discriminative and generative language models. We analyze the
distinct characteristics and challenges of both types of models, as well as
their similarities and differences in approaches and applications across
various tasks. We start by establishing the concepts of confidence,
uncertainty, and calibration for neural models, and elaborating on the unique
challenges for LMs (Section 2). We then discuss confidence estimation and
calibration techniques for discriminative LMs in Section 3 and for generative
LMs in Section 4. Finally, we discuss key future directions (Section 5).

##  2 Preliminaries and Background

###  2.1 Confidence and Uncertainty

Confidence or uncertainty pertains to the level of certainty or assurance
accompanying a prediction or a decision made by a machine learning model. The
terms are often used interchangeably Xiao et al. (2022); Chen and Mueller
(2023). Kendall and Gal (2017) proposed to categorize uncertainty in machine
learning into _aleatoric_ and _epistemic_ uncertainty, the former captures the
uncertainty from data, and the latter accounts for uncertainty in the model;
they jointly affect the of the model’s confidence. For classification tasks
with input $x$, ground truth $y$, and prediction $\hat{y}$, the model’s
predictive confidence is denoted as $\texttt{conf}(x,\hat{y})$. In current
research about quantifying model’s confidence Kamath et al. (2020); Vazhentsev
et al. (2023a), _Relative confidence scores_ place greater emphasis on the
ability to rank samples, distinguishing correct predictions from incorrect
ones. Ideally, for every pair of $(x_{i},y_{i})$ and $(x_{j},y_{j})$ and their
corresponding predictions $\hat{y}_{i}$ and $\hat{y}_{j}$:

$\begin{split}\texttt{conf}(\mathbf{x}_{i},\hat{y}_{i})&\leq\texttt{conf}(\mathbf{x}_{j},\hat{y}_{j})\\\
\Longleftrightarrow P(\hat{y}_{i}=y_{i}|\mathbf{x}_{i})&\leq
P(\hat{y}_{j}=y_{j}|\mathbf{x}_{j})\end{split}$ (1)

_Absolute confidence scores_ assume that confidence should accurately reflect
the true accuracy of the model in real scenarios, i.e., $q\in[0,1]$:

$P(\hat{y}=y\mid\texttt{conf}(x,\hat{y})=q)=q$ (2)

###  2.2 Calibration

Calibration techniques Guo et al. (2017) were originally inspired by the
definition of absolute confidence scores, aiming to ensure that a model’s
predicted probabilities align with the actual probabilities of the events it
predicts. There exist various calibration measures which indicate the
alignment between a model’s predicted confidence and its actual accuracy:

Calibration errors measure the difference between the predicted and the true
probabilities. It partitions the predicted probabilities $\\{\hat{q}_{i}\\}$
into $M$ equally spaced buckets $\\{B_{m}\\}_{m=1}^{M}$, with $B_{m}$
containing samples with prediction confidence falling within the interval
$(\frac{m-1}{M},\frac{m}{M}]$. The widely used _expected calibration error_
(ECE) is defined as follows:

$ECE=\sum_{m=1}^{M}\frac{\lvert B_{m}\rvert}{n}\lvert
acc(B_{m})-conf(B_{m})\rvert)$ (3)

where the averaged accuracy and confidence of $B_{m}$ are defined as follows:

$\displaystyle\texttt{acc}(B_{m})$ $\displaystyle=\frac{1}{\lvert
B_{m}\rvert}\sum_{x_{i}\in B_{m}}\mathbb{I}(\hat{y}_{i}=y_{i})$ (4)
$\displaystyle\texttt{conf}(B_{m})$ $\displaystyle=\frac{1}{\lvert
B_{m}\rvert}\sum_{x_{i}\in B_{m}}\hat{q}_{i}$ (5)

Various factors, such as the bucket width and the variance of samples within
these buckets, can compromise the reliability of certain measures. To address
this, schemes that account for class conditionality and bucket equilibrium
have been developed, such as static calibration error (SCE), adaptive
calibration error (ACE; Nixon et al. 2019), and classwise ECE Kull et al.
(2019).

Brier score measures the mean squared difference between the predicted
probabilities and the actual outcomes:

$BS=\frac{1}{n}\sum_{i=1}^{n}(\mathbb{I}(\hat{y}_{i}=y_{i})-\texttt{conf}(x_{i},\hat{y}_{i}))^{2}$
(6)

where the indicator $\mathbb{I}(\cdot)$ equals 1 when the prediction is
correct, and otherwise it is 0.

Reliability diagram, also known as _calibration curve_ , graphically assesses
the calibration quality of probabilistic predictions. The average predicted
probabilities (confidence) are plotted on the x-axis and the observed
frequencies or proportions on the y-axis. Points or lines above the diagonal
indicate overconfidence, while those below indicate under-confidence. Besides,
metrics like area under receiver operating characteristic curve (AUROC,
Bradley 1997) and area under accuracy-rejection curve (AUARC, Lin et al.
2023), are able to indicate whether the confidence score can appropriately
differentiate correct and incorrect answers.

##  3 Confidence in Discriminative LMs

###  3.1 Confidence Estimation Methods

The methods for confidence estimation have been extensively studied and can
generally be categorized into the following groups:

##### Logit-based estimation

Given the model input $\mathbf{x}$, the logits $\mathbf{z}$, along with the
prediction $\hat{y}$ (i.e., the class with the highest probability emitted by
softmax activation $\sigma$), the model confidence is estimated directly using
the probability value:

$\texttt{conf}_{sp}(\mathbf{x},\hat{y})=P(\hat{y}|\mathbf{x})=\sigma(\mathbf{z})_{\hat{y}}$
(7)

There are methods for estimating confidence based on transformations of the
logit probabilities, such as examining the gap between the two highest
probabilities or utilizing entropy, which indicates the uncertainty with a
larger value.

##### Ensemble-based & Bayesian methods

_DeepEnsemble methods_ Lakshminarayanan et al. (2017) train multiple neural
networks independently, and estimate the uncertainty by computing the variance
of the outputs from these models. _Monte Carlo dropout_ (MCDropout, Gal and
Ghahramani 2016) methods extend the dropout techniques to estimating
uncertainty. As in the training phase, dropout is also applied during
inference, and multiple forward passes are performed to obtain predictions.
While the final prediction is obtained through averaging predictions, with the
variability of the predictions reflecting the model uncertainty.

Methods such as deep-ensemble and MCDropout introduce a heavy computational
overhead, especially when applied to LLMs Malinin and Gales (2021a); Shelmanov
et al. (2021); Vazhentsev et al. (2022), and there is a need to optimize the
computation. Taking MCDropout as an example, the determinantal point process
Kulesza and Taskar (2012) can be applied to facilitate MCDropout by sampling
diverse neurons in the dropout layer Shelmanov et al. (2021).

##### Density-based estimation

Density-based approaches Lee et al. (2018); Yoo et al. (2022) are based on the
assumption that regions of the input space where training data is dense are
regions where the model is likely to be more confident in its predictions.
Conversely, regions with sparse training data are areas of higher uncertainty.
Lee et al. (2018) first proposed a Mahalanobis distance-based confidence
score, which calculates the distance between one test point and a Gaussian
distribution fitting test data. Confidence can be further estimated by
exponentiating the negative value of the distance.

##### Confidence learning

employs a specific branch to learn the confidence of model predictions.
DeVries and Taylor (2018) leveraged a confidence estimation branch to forecast
a scalar confidence and the original probability is modified by interpolating
the ground truth according to the confidence, to provide LMs with “hints”
during the training process. Besides, it discourages the network from always
asking for hints by applying a small penalty. Corbière et al. (2019)
empirically demonstrated that the confidence based on the true class
probability (TCP) is a better measure to distinguish between correct and
incorrect predictions. Given the ground truth $y$, TCP can be represented as
$P(y|\textbf{x})$. However, $y$ is not available when estimating the
confidence of the predictions. Hence, Corbière et al. (2019) used a confidence
learning network to learn the TCP confidence during the training process.

###  3.2 Model Calibration

Calibration methods can be categorized based on their execution time as _in-
training_ and _post-hoc_ methods.

####  3.2.1 In-Training Calibration

##### Novel loss functions

Many studies considered the cross-entropy (CE) loss to be one of the causes
leading to model miscalibration Mukhoti et al. (2020); Kim et al. (2023).
Mukhoti et al. (2020) demonstrated that _focal loss_ Lin et al. (2017),
designed to give more importance to hard-to-classify examples and to down-
weight the easy-to-classify examples, can improve the calibration of neural
networks. The _correctness ranking loss_ (CRL; Moon et al. 2020) calibrated
models by penalizing incorrect rankings within the same batch and by using the
difference in proportions as the margin to differentiate sample confidence.
Besides, _entropy regularization loss_ (ERL; Pereyra et al. 2017) and _label
smoothing_ (LS; Szegedy et al. 2016) were introduced to discourage overly
confident output distributions.

##### Data augmentation

involves creating new training examples by applying various transformations or
perturbations to the original data. It has been widely used for calibration of
discriminative LMs by alleviating the issue of over-confidence, such as MixUp
Zhang et al. (2018), EDA Wei and Zou (2019), Manifold-MixUp Verma et al.
(2019), MIMO Havasi et al. (2021) and AUM-guided MixUp Park and Caragea
(2022).

##### Ensemble and Bayesian methods

were initially introduced to quantify model uncertainty. However, both can
also be valuable for model calibration, as they can enhance accuracy, mitigate
overfitting, and reduce overconfidence Kong et al. (2020); Kim et al. (2023).

####  3.2.2 Post-Hoc Calibration

##### Scaling methods

are exemplified by _matrix scaling_ , _vector scaling_ and _temperature
scaling_ Guo et al. (2017). They fine-tune the predicted probabilities to
better align with the true outcomes using a validation set, leveraging the
_negative log-likelihood_ (NLL) loss. Among them, temperature scaling (TS) is
popular due to its low complexity and efficiency. It involves re-weighting the
logits before the softmax function by a learned scalar $\tau$, known as the
_temperature_.

##### Feature-based calibrator

leverages both input features and model predictions to refine the predicted
probabilities. To use such a calibrator, one first applies a trained model to
a calibration dataset. Subsequently, both the original input features and the
model’s predictions from this dataset are passed to a binary classifier
Jagannatha and Yu (2020); Jiang et al. (2021); Si et al. (2022).

###  3.3 Summary

##### Confidence estimation

Logit-based methods stand out as the most straightforward to implement and
interpret. However, they have issues with overconfidence and contain implicit
biases Pearce et al. (2021). Reducing computational cost and improving the
sampling efficiency pose challenges to ensemble-based and Bayesian methods.
Density-based estimation can be used to identify which data points are
associated with different types of uncertainties. However, it requires
assumptions of data distribution Baan et al. (2023) and can also be
computationally when dealing with large datasets Sun et al. (2022). Confidence
learning can acquire task-relevant confidence; however, it requires modifying
the neural network and performing specific training.

##### Model calibration

Post-hoc methods are generally model-independent and can calibrate
probabilities without impacting the model’s performance Guo et al. (2017).
Desai and Durrett (2020) empirically found that temperature scaling
effectively reduces calibration error in-domain, whereas label smoothing is
more beneficial in out-of-domain settings. Using different loss functions to
penalize overconfidence involves tuning hyperparameters to balance accuracy
and calibration Kim et al. (2023). Kim et al. (2023) found that augmentation
can enhance both classification accuracy and calibration performance. However,
ensemble methods may sometimes degrade model calibration if individual members
produce similar predictions due to overfitting. It has been found that various
calibration methods are effective for extractive question-answering tasks
based on discriminative LMs Zhang et al. (2021); Si et al. (2022). The upper
part of the Table 1 represents significant work in the calibration of
discriminative LMs. We have comprehensively listed the models, tasks, and
calibration methods they employed.

##  4 Confidence in Generative LMs

In this section, we first analyze the characteristics of generative LMs and
the challenges they introduce in Section 4.1. Subsequently, we summarize the
research advancements in generation and classification tasks separately in
Section 4.2 and Section 4.3, noting their significant differences in
confidence assessment and calibration.

###  4.1 Characteristics and Challenges

In contrast to discriminative models that easily yield probability scores for
specific categories, generative LMs face an exponential expansion of their
output space with increasing sentence length. This expansion, coupled with
unbounded content, makes it impractical to evaluate all potential predictions,
complicating the derivation of confidence or certainty metrics Wang et al.
(2023a).

Semantic nuances and their inherent uncertainties are significant. Unlike
fixed category labels, the outputs of generative language models (LLMs)
capture the diversity of natural language, where the same words or sentences
can have varied meanings across different contexts, and distinct phrases may
convey identical meanings. Additionally, semantics are heavily influenced by
and conditional upon the input Kuhn et al. (2023).

Large language models transcend traditional task boundaries, requiring varied
confidence granularity depending on the task. For example, machine translation
demands token-level uncertainty assessment, whereas question answering
benefits from sentence-level confidence evaluation Duan et al. (2023).
Research indicates that changes in prompt structure, training examples, and
their sequence can affect model output and precision. The chosen methods for
measuring confidence and interpreting generated responses are also crucial in
determining model assessment Zhao et al. (2021); Min et al. (2022b); Zheng et
al. (2023b).

###  4.2 Generative LMs for Generation Tasks

####  4.2.1 Confidence Estimation

Generally, methods for estimating uncertainty in generative LMs are divided
into white-box and black-box approaches. This categorization is also driven by
the increasing prevalence of closed-source language models.

##### Methods for white-box LMs

The logit-based method evaluates sentence uncertainty using token-level
probabilities or entropy. Assuming that $\mathbf{y}=y_{1},\cdots,y_{T}$
denotes the sequence of generated tokens (target sentence), and that
$\mathbf{x}=x_{1},\cdots,x_{S}$ denotes the sequence of input tokens (source
sentence), the sentence confidence can be represented by the factorized
probability: $\prod_{i=1}^{T}P(y_{i}|\mathbf{x},\mathbf{y}_{<i})$. To ensure
an evaluation consistent across sentences of different lengths, the length-
normalized likelihood probability is widely utilized Murray and Chiang (2018).
Moreover, alternatives such as the minimum or average token probabilities, as
well as the average entropy, are also considered Vazhentsev et al. (2023b).
Duan et al. (2023) introduced the concept of _token-level relevance_ , which
evaluated the relevance of the token by comparing semantic change before and
after moving the token with a semantic similarity metric like Cross-Encoder
Reimers and Gurevych (2019). Building upon this idea, sentence uncertainty can
be effectively adjusted based on the token’s relevance.

Logit-based techniques readily adapt to scenarios involving multiple sampling
Vazhentsev et al. (2023b) or ensemble models Malinin and Gales (2021b). To
incorporate semantics, Kuhn et al. (2023) proposed clustering similar meaning
responses to explore the semantic uncertainty of language models. Duan et al.
(2023) suggested a similar idea, introducing _sentence relevance_ of other
answers when assessing a model’s confidence in a specific answer.

Besides, Ren et al. (2022) explored the _density-based method_ and
demonstrated its effectiveness on abstractive summarization and translation.
Kadavath et al. (2022) utilized a binary classification head atop a language
model to learn the probability of providing a correct answer to a question,
referred to as the P(IK) classifier. They sampled multiple answers for each
question at a consistent temperature, labeled the correctness of each answer,
and then used the question-correctness pair as the training data.

In general, the methods for white-box LMs share similarities with the
techniques used in discriminative models. Their primary research focus lies in
addressing variable sentence lengths and incorporating sentence semantics.

##### Methods for black-box LMs

are primarily applicable for evaluating whether hallucinations against facts
are produced in open-ended generation tasks. There are roughly two groups of
methods. _Linguistic confidence_ or _verbalized methods_ refer to language
models expressing uncertainty in the form of tokens. This involves discerning
different levels of uncertainty from the model’s responses, such as "I don’t
know," "most probably," or "Obviously" Mielke et al. (2022) or prompting the
model to output various verbalized words (e.g. "lowest", "low" "medium",
"high", "highest") or numbers (e.g. $"85\%"$). Assuming LMs possess good self-
awareness and self-reflection abilities, they could use linguistic confidence
as a supplement to confidence scores, such as logit probability. However, in
practice, it has been observed that there is a mismatch between them Lin et
al. (2022); Xiong et al. (2023); Liu et al. (2023b).

_Consistency-based_ estimation premises on the assumption that a model’s lack
of confidence correlates with a wide variety of responses, often leading to
hallucinatory outputs. SelfCheckGPT Manakul et al. (2023) proposed a simple
sampling-based approach, which uses the consistency among generations to find
potential hallucination, five variants are utilized to measure the
consistency: BERTScore Zhang et al. (2020b), question-answering, n-gram, NLI,
and LLM prompting. Lin et al. (2023) proposed to calculate the similarity
matrix between generations, and then to estimate the uncertainty based on the
analysis of the similarity matrix, such as the sum of the eigenvalues of the
graph Laplacian, the degree matrix, and the eccentricity. Xiong et al. (2023)
proposed a hybrid method by combining verbalized numbers and consistency-based
scores for benchmarking.

####  4.2.2 Calibration Methods

##### Probabilistic Calibration

Kumar and Sarawagi (2019) showed that most modern NMT models are surprisingly
miscalibrated even when conditioned on the true previous tokens, leading to
severe overconfidence in their predictions. Kumar and Sarawagi (2019)
introduced a modified temperature scaling approach where the temperature value
adjusts according to various factors including the entropy of attention, token
logit, token identity, and input coverage. Wang et al. (2020) noted a
pronounced prevalence of over-estimated tokens compared to under-estimated
ones. They introduced _graduated label smoothing_ , applying heightened
smoothing penalties to confident predictions, resulting in a substantial
reduction of ECE. Xiao and Wang (2021) proposed to use uncertainty-aware beam
search, which calibrated the token’s predictive probability with uncertainty
estimated from deep ensembles. van der Poel et al. (2022) introduced a
decoding strategy that prioritizes optimizing the pointwise mutual information
between the source and the target tokens. Zablotskaia et al. (2023) adapted a
bag of ensemble-based and Bayesian methods to neural summarization to improve
model calibration.

In addition to token-level calibration, Zhao et al. (2022) suggested that MLE
training can result in poorly calibrated sentence-level confidence, i.e., a
skewed sequence likelihood. To rectify this, they proposed the sequence
likelihood calibration (SLiC) technique. It first generates $m$ multiple
sequences $\\{\hat{\mathbf{y}}\\}_{m}$ from the initial model $\theta_{0}$,
then it calibrates the model’s confidence with:

$\sum_{\\{\mathbf{x},\bar{\mathbf{y}}\\}}\mathcal{L}^{cal}(\theta,\mathbf{x},\bar{\mathbf{y}},\\{\hat{\mathbf{y}}\\}_{m})+\lambda\mathcal{L}^{reg}(\theta,\theta_{0},\mathbf{x},\bar{\mathbf{y}})$
(8)

where the calibration loss $\mathcal{L}^{cal}$ aims to align models’ decoded
candidates’ sequence likelihood according to their similarity with the
reference $\bar{\mathbf{y}}$, and regularization loss $\mathcal{L}^{reg}$
prevents models from much deviation. They further introduced SLiC-HF Zhao et
al. (2023), which was designed to adeptly learn from human preferences.

##### Linguistic Calibration

Mielke et al. (2022) proposed a calibrator-controlled method for chatbots,
which involves trained a calibrator to return the model confidence score and
fine-tuned the generative model to enable control over linguistic confidence.
Lin et al. (2022) fine-tuned GPT-3 with human labeled dataset containing
verbalized words and numbers to naturally express uncertainty. Zhou et al.
(2023) empirically found that when injecting expressions of uncertainty into
prompts, the accuracy of GPT-3’s answers would significantly increase.
Moreover, model calibration suffers when teaching the model to emit certainty
rather than uncertainty. Xiong et al. (2023) showcased that prompting
strategies like CoT Wei et al. (2022) can improve the linguistic confidence.
Besides, different datasets Amayuelas et al. (2023); Yin et al. (2023); Wang
et al. (2023c) have been presented on questions that language models cannot
answer, or for which there is no clear answer.

###  4.3 Generative LMs for Classification Tasks

Despite being designed for generation, generative LMs are extensively employed
in classification tasks due to their ability to generalize and adhere to
instructions Brown et al. (2020); Zhao et al. (2021). Moreover, these tasks
are widely used to evaluate LMs’ capabilities as well as whether they are
well-calibrated thanks to standardized evaluation and reduced output ambiguity
Kadavath et al. (2022).

####  4.3.1 In-Context Learning

In-context learning (ICL) is a phenomenon observed in LLMs, where the model
learns to perform a task based on a few examples and the context in which the
task is presented. Assuming that $k$ selected input-label pairs
$(\mathbf{x}_{1},y_{1}),\cdots,(\mathbf{x}_{k},y_{k})$ are given as
demonstrations, ICL makes the prediction as follows:

$\hat{y}=\operatorname*{arg\,max}_{y}P(y|\mathbf{x}_{1},y_{1},\cdots,\mathbf{x}_{k},y_{k},\mathbf{x})$
(9)

When there are no demonstrations, the model performs zero-shot learning. The
channel method Min et al. (2022a) swaps the positions of labels and inputs,
leveraging conditional sequence probability to formulate predictions:

$\hat{y}=\operatorname*{arg\,max}_{y}P(\mathbf{x}|y_{1},\mathbf{x}_{1},\cdots,y_{k},\mathbf{x}_{k},y)$
(10)

##### Calibration methods

ICL is highly sensitive to the prompts including the choice of templates and
demonstrations Min et al. (2022b). We refer to the input-label pairs as
$\mathbf{C}$ for context, and the original predictive probability is denoted
as $P(y|\mathbf{C},\mathbf{x})$. Zhao et al. (2021) introduced a method called
_contextual calibration_. It gauges the model’s bias with context-free prompts
such as "[N/A]", "[MASK]" and empty string, then the context-free scores is
estimated by $\hat{\mathbf{P}}_{\texttt{cf}}=P(y|\mathbf{C},\texttt{[N/A]})$.
Subsequently, it transforms the uncalibrated scores with
$\mathbf{W}=diag(\hat{\mathbf{p}}_{\texttt{cf}})^{-1}$ to offset the
miscalibration. Fei et al. (2023) proposed _domain-context calibration_ ,
which estimates the prior bias for each class with $n$ times model average
with random text of an average sentence length:
$\bar{\mathbf{P}}_{rd}(y|\mathbf{C})=\frac{1}{n}\sum_{i=1}^{n}P(y|\mathbf{C},\texttt{[RANDOM
TEXT]})$. The prediction is obtained with:

$\hat{y}=\operatorname*{arg\,max}_{y}\frac{P(y|\mathbf{C},\mathbf{x})}{\bar{\mathbf{P}}_{rd}(y|\mathbf{C})}$
(11)

Some methods aim to improve few-shot learning performance by combining classic
machine learning techniques. For instance, Han et al. (2022) introduced
_prototypical calibration_ , which employs Gaussian mixture models (GMM) to
learn decision boundaries. However, it involves significant computational
overhead. On the other hand, Nie et al. (2022) enhanced predictions by
integrating a $k$-nearest-neighbor classifier with a datastore containing
cached few-shot instance representations.

####  4.3.2 Multiple-Choice Question Answering

Multiple-choice question answering (MCQA) involves evaluating an LM’s ability
to answer questions with predefined multiple choices. In MCQA, given the
question $\mathbf{q}$, and the set of answer options
$\mathcal{I}(\mathbf{q})=\\{\mathbf{y}_{1},,\cdots,\mathbf{y}_{K}\\}$, with
each prefaced by identifiers such as _(A)_ or _(B)_ , etc, the input (prompt)
$\mathbf{x}$ consists of the question combined with the options, and, if
available, with a demonstration serving as an instruction to aid in answering
the question.

Orthogonal to different LMs, the implementation of the evaluation protocols
can significantly impact the ranking of models. For instance, the original
evaluation of the MMLU Hendrycks et al. (2021) ranks the probabilities of the
four option identifiers. When the highest probability corresponds to the
correct option, the answer is considered correct. In contrast, the HELM
implementation Liang et al. (2022) considers the complete vocabulary, while
the HARNESS implementation111https://github.com/EleutherAI/lm-evaluation-
harness/tree/v0.3.0 prefers length-normalized probabilities of the entire
answer sequence. These protocols, while having minor differences, have a
significant impact on model evaluation.

##### Calibration methods

Jiang et al. (2021) proposed various fine-tuning loss functions and
temperature scaling for calibrating the performance on MQCA datasets.
Additionally, they proposed LM-specific techniques such as candidate output
paraphrasing and input augmentation. Holtzman et al. (2021) claimed that
surface form competition occurs when different valid surface forms compete for
probability, making it problematic to simply choose the highest probability
answer. Thus, they introduced _domain conditional pointwise mutual
information_ (PMIDC), which reweighs each option according to a term that is
proportional to its a priori likelihood within the context of the specific
zero-shot task. Kumar (2022) demonstrated that prevalent datasets tend to
benefit models that depend on context-independent simple indicators, and
introduced answer-level calibration (ALC), a universal method that enhances
the unsupervised performance of LMs by addressing bias in MCQA. Zheng et al.
(2023a) identified that LLMs exhibit an inherent selection bias, which refers
to their preferences to select options located at specific positions. They
proposed PriDe, which first decomposes the observed model prediction
distribution into an intrinsic prediction over option contents and a prior
distribution over option IDs and then estimates the prior by permutating
option contents on a small number of test samples.

Generally, in classification tasks, whether it is ICL or MCQA, the related
work primarily employs logit-based methods to assess model confidence, with a
focus on offsetting bias to enhance the model’s accuracy and robustness.

Logits Internal stateSemantics123pt4 Figure 1: Venn diagram of the taxonomy of
confidence estimation for generative LMs. According to their probing position
during LMs’ inference, those methods can be taxonomized to three families,
including predictive logits, LMs’ internal state, or semantics in generation,
and their intersections (Zone 1 \- 4).

###  4.4 Summary

In this section, we exhaustively review the methods of confidence estimation
and calibration on generative LMs. The bottom half of the Table 1 summarizes
the related studies, and we offer details regarding the models, tasks, and
calibration methods.

Besides, we find one key for confidence estimation, not bound by the task,
i.e., classification or generation, or the model, i.e., white-box or black-
box, that is the _probing_ position. Figure 1 illustrates the taxonomy of
confidence estimation based on the probing position. Information from various
positions can be leveraged to assess the model’s confidence, i.e., internal
state inside LMs Ren et al. (2022); Kadavath et al. (2022), predictive logits
Malinin and Gales (2021b); Vazhentsev et al. (2023b) and semantics in
generation Mielke et al. (2022); Manakul et al. (2023).

The current advanced methods usually combine different dimensions during
confidence estimation. Recent works Kuhn et al. (2023); Duan et al. (2023)
achieves SOTA performance on uncertainty estimation for open-domain QA by
effectively integrating logit-based methods Malinin and Gales (2021b) and
generated semantics, corresponding to Zone 2 in Figure 1. In our review,
however, there is limited literature working on the combination among other
types of information. Encouraged by the achieved success, we expect that
collaborative exploitation of different types of information will benefit
confidence estimation in LLMs.

##  5 Future Directions

Based on our summary of the trends in research on uncertainty in existing
language models, we cautiously propose several research directions in this
field:

##### Hallucination detection and mitigation

Confidence-related methods have showcased their potential for detecting
intrinsic hallucinations, particularly in areas such as machine translation
Kumar and Sarawagi (2019); Lu et al. (2022) and data-to-text generation Xiao
and Wang (2021). Their efficacy in open-ended generation tasks remains a point
of discussion. The current hallucination detection methods tend to be coarse-
grained Manakul et al. (2023); Lin et al. (2023), i.e. whether the sentence is
truthful or not, emphasizing the importance of advancing towards fine-grained
truthfulness assessment.

##### Learning to explain and clarify

Guiding LLMs to explicitly explain uncertainty and pose clarifying questions
during interactions will enhance the model’s trustworthiness. Chain-of-thought
Wei et al. (2022) allows LLMs to execute intricate reasoning tasks while
enhancing the interpretability of their decisions by generating a sequence of
intermediate reasoning steps. Ren et al. (2023) proposed to utilize conformal
prediction to offer statistical guarantees on task completion while minimizing
human help in multi-step planning tasks.

##### Multi-modal large language models

By employing additional pre-training with image-text pairings, or by fine-
tuning on specialized visual-instruction datasets, large language models can
transition into the multimodal domain Dai et al. (2023); Liu et al. (2023a);
Zhu et al. (2023). However, it remains unclear whether these confidence
estimation methods are effective for multilingual large language models
(MLLMs) and whether these models are well-calibrated. Ji et al. (2022)
introduced uncertainty into the pre-training of the vision-language model by
using a probabilistic distribution encoder (PDE) to obtain representations of
different modalities and empirically improved the performance on different
tasks.

##### Calibration on human variation

Plank (2022) clarified the prevalent existence of human variation, i.e.,
humans have different opinions when labeling the same data. Human disagreement
Jiang and Marneffe (2022) can be attributed to task ambiguity Tamkin et al.
(2022), annotator’s subjectivity Sap et al. (2022), and input ambiguity
Meissner et al. (2021). Recent work Baan et al. (2022); Lee et al. (2023)
demonstrated the misalignment between LM calibration measures and human
disagreement in various learning paradigms. Expressing the concern regarding
different types of ambiguity Xiong et al. (2023), abstaining from answering
ambiguous questions Yoshikawa and Okazaki (2023), and further resolving
ambiguity Varshney and Baral (2023) are demanded for trustworthy and reliable
LLMs.

##  6 Conclusion

In this article, we delved into the pressing question of how to improve our
trust in language models. Initially, we explored the very definition of
confidence in LMs and the challenges of confidence estimation. We then shifted
our focus to current methodologies that gauge the confidence of these models.
A significant facet of our study centers on calibration, given its direct
correlation with both LM confidence and potential biases during model
inference. We outline specific measures, techniques, and their respective
applications. Our study demonstrates the landscape of enhancing LLM
trustworthiness through confidence estimation and calibration and suggests
directions for future work.

## Acknowledgement

Fengyu Cai is funded by the German Federal Ministry of Education and Research
and the Hessian Ministry of Higher Education, Research, Science, and the Arts
within their joint support of the National Research Center for Applied
Cybersecurity ATHENE.

## References

  * Amayuelas et al. (2023) Alfonso Amayuelas, Liangming Pan, Wenhu Chen, and William Wang. 2023.  Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models.  _ArXiv preprint_ , abs/2305.13712.
  * Anil et al. (2023) Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vladimir Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, and et al. 2023.  Palm 2 technical report.  _ArXiv preprint_ , abs/2305.10403.
  * Baan et al. (2022) Joris Baan, Wilker Aziz, Barbara Plank, and Raquel Fernandez. 2022.  Stop measuring calibration when humans disagree.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 1892–1915, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Baan et al. (2023) Joris Baan, Nico Daheim, Evgenia Ilia, Dennis Ulmer, Haau-Sing Li, Raquel Fernández, Barbara Plank, Rico Sennrich, Chrysoula Zerva, and Wilker Aziz. 2023.  Uncertainty in natural language generation: From theory to applications.  _ArXiv preprint_ , abs/2307.15703.
  * Bradley (1997) Andrew P. Bradley. 1997.  The use of the area under the roc curve in the evaluation of machine learning algorithms.  _Pattern Recognition_ , 30(7):1145–1159.
  * Brown et al. (2020) Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020.  Language models are few-shot learners.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Chen and Mueller (2023) Jiuhai Chen and Jonas Mueller. 2023.  Quantifying uncertainty in answers from any language model via intrinsic and extrinsic confidence assessment.  _ArXiv preprint_ , abs/2308.16175.
  * Chiang et al. (2023) Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023.  Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality.  _See https://vicuna. lmsys. org (accessed 14 April 2023)_.
  * Corbière et al. (2019) Charles Corbière, Nicolas Thome, Avner Bar-Hen, Matthieu Cord, and Patrick Pérez. 2019.  Addressing failure prediction by learning model confidence.  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pages 2898–2909.
  * Dai et al. (2023) Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. 2023.  Instructblip: Towards general-purpose vision-language models with instruction tuning.  _ArXiv preprint_ , abs/2305.06500.
  * Desai and Durrett (2020) Shrey Desai and Greg Durrett. 2020.  Calibration of pre-trained transformers.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 295–302, Online. Association for Computational Linguistics.
  * DeVries and Taylor (2018) Terrance DeVries and Graham W Taylor. 2018.  Learning confidence for out-of-distribution detection in neural networks.  _ArXiv preprint_ , abs/1802.04865.
  * Duan et al. (2023) Jinhao Duan, Hao Cheng, Shiqi Wang, Chenan Wang, Alex Zavalny, Renjing Xu, Bhavya Kailkhura, and Kaidi Xu. 2023.  Shifting attention to relevance: Towards the uncertainty estimation of large language models.  _ArXiv preprint_ , abs/2307.01379.
  * Fei et al. (2023) Yu Fei, Yifan Hou, Zeming Chen, and Antoine Bosselut. 2023.  Mitigating label biases for in-context learning.  _ArXiv preprint_ , abs/2305.19148.
  * Gal and Ghahramani (2016) Yarin Gal and Zoubin Ghahramani. 2016.  Dropout as a bayesian approximation: Representing model uncertainty in deep learning.  In _Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016_ , volume 48 of _JMLR Workshop and Conference Proceedings_ , pages 1050–1059. JMLR.org.
  * Gao et al. (2023) Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu. 2023.  RARR: researching and revising what language models say, using language models.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023_ , pages 16477–16508. Association for Computational Linguistics.
  * Guo et al. (2017) Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017.  On calibration of modern neural networks.  In _Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017_ , volume 70 of _Proceedings of Machine Learning Research_ , pages 1321–1330. PMLR.
  * Han et al. (2022) Zhixiong Han, Yaru Hao, Li Dong, Yutao Sun, and Furu Wei. 2022.  Prototypical calibration for few-shot learning of language models.  _ArXiv preprint_ , abs/2205.10183.
  * Havasi et al. (2021) Marton Havasi, Rodolphe Jenatton, Stanislav Fort, Jeremiah Zhe Liu, Jasper Snoek, Balaji Lakshminarayanan, Andrew Mingbo Dai, and Dustin Tran. 2021.  Training independent subnetworks for robust prediction.  In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Hendrycks et al. (2021) Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021.  Measuring massive multitask language understanding.  In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Holtzman et al. (2021) Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. 2021.  Surface form competition: Why the highest probability answer isn’t always right.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 7038–7051, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Hu et al. (2023) Mengting Hu, Zhen Zhang, Shiwan Zhao, Minlie Huang, and Bingzhe Wu. 2023.  Uncertainty in natural language processing: Sources, quantification, and applications.  _ArXiv preprint_ , abs/2306.04459.
  * Jagannatha and Yu (2020) Abhyuday Jagannatha and Hong Yu. 2020.  Calibrating structured output predictors for natural language processing.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 2078–2092, Online. Association for Computational Linguistics.
  * Ji et al. (2022) Yatai Ji, Junjie Wang, Yuan Gong, Lin Zhang, Yanru Zhu, Hongfa Wang, Jiaxing Zhang, Tetsuya Sakai, and Yujiu Yang. 2022.  Map: modality-agnostic uncertainty-aware vision-language pre-training model.  _ArXiv preprint_ , abs/2210.05335.
  * Ji et al. (2023) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023.  Survey of hallucination in natural language generation.  _ACM Computing Surveys_ , 55(12):1–38.
  * Jiang and Marneffe (2022) Nan-Jiang Jiang and Marie-Catherine de Marneffe. 2022.  Investigating reasons for disagreement in natural language inference.  _Transactions of the Association for Computational Linguistics_ , 10:1357–1374.
  * Jiang et al. (2021) Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021.  How can we know when language models know? on the calibration of language models for question answering.  _Transactions of the Association for Computational Linguistics_ , 9:962–977.
  * Kadavath et al. (2022) Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022.  Language models (mostly) know what they know.  _ArXiv preprint_ , abs/2207.05221.
  * Kamath et al. (2020) Amita Kamath, Robin Jia, and Percy Liang. 2020.  Selective question answering under domain shift.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 5684–5696, Online. Association for Computational Linguistics.
  * Kendall and Gal (2017) Alex Kendall and Yarin Gal. 2017.  What uncertainties do we need in bayesian deep learning for computer vision? In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 5574–5584.
  * Kim et al. (2023) Jaeyoung Kim, Dongbin Na, Sungchul Choi, and Sungbin Lim. 2023.  Bag of tricks for in-distribution calibration of pretrained transformers.  In _Findings of the Association for Computational Linguistics: EACL 2023_ , pages 551–563, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Kong et al. (2020) Lingkai Kong, Haoming Jiang, Yuchen Zhuang, Jie Lyu, Tuo Zhao, and Chao Zhang. 2020.  Calibrated language model fine-tuning for in- and out-of-distribution data.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1326–1340, Online. Association for Computational Linguistics.
  * Kuhn et al. (2023) Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023.  Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation.  _ArXiv preprint_ , abs/2302.09664.
  * Kulesza and Taskar (2012) Alex Kulesza and Ben Taskar. 2012.  Determinantal point processes for machine learning.  _Foundations and Trends® in Machine Learning_ , 5(2–3):123–286.
  * Kull et al. (2019) Meelis Kull, Miquel Perelló-Nieto, Markus Kängsepp, Telmo de Menezes e Silva Filho, Hao Song, and Peter A. Flach. 2019.  Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration.  In _Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada_ , pages 12295–12305.
  * Kumar and Sarawagi (2019) Aviral Kumar and Sunita Sarawagi. 2019.  Calibration of encoder decoder models for neural machine translation.  _ArXiv preprint_ , abs/1903.00802.
  * Kumar (2022) Sawan Kumar. 2022.  Answer-level calibration for free-form multiple choice question answering.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 665–679, Dublin, Ireland. Association for Computational Linguistics.
  * Lakshminarayanan et al. (2017) Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017.  Simple and scalable predictive uncertainty estimation using deep ensembles.  In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_ , pages 6402–6413.
  * Lee et al. (2018) Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. 2018.  A simple unified framework for detecting out-of-distribution samples and adversarial attacks.  In _Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada_ , pages 7167–7177.
  * Lee et al. (2023) Noah Lee, Na Min An, and James Thorne. 2023.  Can large language models infer and disagree like humans? _ArXiv preprint_ , abs/2305.13788.
  * Lewis et al. (2020a) Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020a.  BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 7871–7880, Online. Association for Computational Linguistics.
  * Lewis et al. (2020b) Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020b.  Retrieval-augmented generation for knowledge-intensive NLP tasks.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_.
  * Li et al. (2022) Wei Li, Wenhao Wu, Moye Chen, Jiachen Liu, Xinyan Xiao, and Hua Wu. 2022.  Faithfulness in natural language generation: A systematic survey of analysis, evaluation and optimization methods.  _ArXiv preprint_ , abs/2203.05227.
  * Liang et al. (2022) Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. 2022.  Holistic evaluation of language models.  _ArXiv preprint_ , abs/2211.09110.
  * Lin et al. (2022) Stephanie Lin, Jacob Hilton, and Owain Evans. 2022.  Teaching models to express their uncertainty in words.  _ArXiv preprint_ , abs/2205.14334.
  * Lin et al. (2017) Tsung-Yi Lin, Priya Goyal, Ross B. Girshick, Kaiming He, and Piotr Dollár. 2017.  Focal loss for dense object detection.  In _IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017_ , pages 2999–3007. IEEE Computer Society.
  * Lin et al. (2023) Zhen Lin, Shubhendu Trivedi, and Jimeng Sun. 2023.  Generating with confidence: Uncertainty quantification for black-box large language models.  _ArXiv preprint_ , abs/2305.19187.
  * Liu et al. (2023a) Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a.  Visual instruction tuning.  _ArXiv preprint_ , abs/2304.08485.
  * Liu et al. (2023b) Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. 2023b.  Trustworthy llms: a survey and guideline for evaluating large language models’ alignment.  _arXiv preprint arXiv:2308.05374_.
  * Lu et al. (2022) Yu Lu, Jiali Zeng, Jiajun Zhang, Shuangzhi Wu, and Mu Li. 2022.  Learning confidence for transformer-based neural machine translation.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 2353–2364, Dublin, Ireland. Association for Computational Linguistics.
  * Malinin and Gales (2021a) Andrey Malinin and Mark J. F. Gales. 2021a.  Uncertainty estimation in autoregressive structured prediction.  In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Malinin and Gales (2021b) Andrey Malinin and Mark J. F. Gales. 2021b.  Uncertainty estimation in autoregressive structured prediction.  In _9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021_. OpenReview.net.
  * Manakul et al. (2023) Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023.  Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models.  _ArXiv preprint_ , abs/2303.08896.
  * Meissner et al. (2021) Johannes Mario Meissner, Napat Thumwanit, Saku Sugawara, and Akiko Aizawa. 2021.  Embracing ambiguity: Shifting the training target of NLI models.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)_ , pages 862–869, Online. Association for Computational Linguistics.
  * Mielke et al. (2022) Sabrina J. Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. 2022.  Reducing conversational agents’ overconfidence through linguistic calibration.  _Transactions of the Association for Computational Linguistics_ , 10:857–872.
  * Min et al. (2023) Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023.  Factscore: Fine-grained atomic evaluation of factual precision in long form text generation.  _ArXiv preprint_ , abs/2305.14251.
  * Min et al. (2022a) Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2022a.  MetaICL: Learning to learn in context.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 2791–2809, Seattle, United States. Association for Computational Linguistics.
  * Min et al. (2022b) Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022b.  Rethinking the role of demonstrations: What makes in-context learning work? In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 11048–11064, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Moon et al. (2020) Jooyoung Moon, Jihyo Kim, Younghak Shin, and Sangheum Hwang. 2020.  Confidence-aware learning for deep neural networks.  In _Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 7034–7044. PMLR.
  * Mukhoti et al. (2020) Jishnu Mukhoti, Viveka Kulharia, Amartya Sanyal, Stuart Golodetz, Philip H. S. Torr, and Puneet K. Dokania. 2020.  Calibrating deep neural networks using focal loss.  In _Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual_ , pages 15288–15299.
  * Murray and Chiang (2018) Kenton Murray and David Chiang. 2018.  Correcting length bias in neural machine translation.  In _Proceedings of the Third Conference on Machine Translation: Research Papers_ , pages 212–223, Brussels, Belgium. Association for Computational Linguistics.
  * Nie et al. (2022) Feng Nie, Meixi Chen, Zhirui Zhang, and Xu Cheng. 2022.  Improving few-shot performance of language models via nearest neighbor calibration.  _ArXiv preprint_ , abs/2212.02216.
  * Nixon et al. (2019) Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. 2019.  Measuring calibration in deep learning.  In _IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16-20, 2019_ , pages 38–41. Computer Vision Foundation / IEEE.
  * Park and Caragea (2022) Seo Yeon Park and Cornelia Caragea. 2022.  On the calibration of pre-trained language models using mixup guided by area under the margin and saliency.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 5364–5374, Dublin, Ireland. Association for Computational Linguistics.
  * Pearce et al. (2021) Tim Pearce, Alexandra Brintrup, and Jun Zhu. 2021.  Understanding softmax confidence and uncertainty.  _arXiv preprint arXiv:2106.04972_.
  * Penedo et al. (2023) Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. 2023.  The refinedweb dataset for falcon LLM: outperforming curated corpora with web data, and web data only.  _ArXiv preprint_ , abs/2306.01116.
  * Peng et al. (2023) Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023.  Check your facts and try again: Improving large language models with external knowledge and automated feedback.  _ArXiv preprint_ , abs/2302.12813.
  * Pereyra et al. (2017) Gabriel Pereyra, George Tucker, Jan Chorowski, Łukasz Kaiser, and Geoffrey Hinton. 2017.  Regularizing neural networks by penalizing confident output distributions.  _ArXiv preprint_ , abs/1701.06548.
  * Petroni et al. (2019) Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019.  Language models as knowledge bases? In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 2463–2473, Hong Kong, China. Association for Computational Linguistics.
  * Plank (2022) Barbara Plank. 2022.  The “problem” of human label variation: On ground truth in data, modeling and evaluation.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021.  Learning transferable visual models from natural language supervision.  In _Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event_ , volume 139 of _Proceedings of Machine Learning Research_ , pages 8748–8763. PMLR.
  * Raffel et al. (2020) Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020.  Exploring the limits of transfer learning with a unified text-to-text transformer.  _The Journal of Machine Learning Research_ , 21(1):5485–5551.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019_ , pages 3980–3990. Association for Computational Linguistics.
  * Ren et al. (2023) Allen Z Ren, Anushri Dixit, Alexandra Bodrova, Sumeet Singh, Stephen Tu, Noah Brown, Peng Xu, Leila Takayama, Fei Xia, Jake Varley, et al. 2023.  Robots that ask for help: Uncertainty alignment for large language model planners.  _ArXiv preprint_ , abs/2307.01928.
  * Ren et al. (2022) Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, and Peter J Liu. 2022.  Out-of-distribution detection and selective generation for conditional language models.  _ArXiv preprint_ , abs/2209.15558.
  * Sap et al. (2022) Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022.  Annotators with attitudes: How annotator beliefs and identities bias toxic language detection.  In _Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 5884–5906, Seattle, United States. Association for Computational Linguistics.
  * Shelmanov et al. (2021) Artem Shelmanov, Evgenii Tsymbalov, Dmitri Puzyrev, Kirill Fedyanin, Alexander Panchenko, and Maxim Panov. 2021.  How certain is your Transformer? In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 1833–1840, Online. Association for Computational Linguistics.
  * Si et al. (2022) Chenglei Si, Chen Zhao, Sewon Min, and Jordan Boyd-Graber. 2022.  Re-examining calibration: The case of question answering.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 2814–2829, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Sun et al. (2022) Hao Sun, Boris van Breugel, Jonathan Crabbe, Nabeel Seedat, and Mihaela van der Schaar. 2022.  Daux: a density-based approach for uncertainty explanations.  _arXiv preprint arXiv:2207.05161_.
  * Szegedy et al. (2016) Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. 2016.  Rethinking the inception architecture for computer vision.  In _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_ , pages 2818–2826. IEEE Computer Society.
  * Tamkin et al. (2022) Alex Tamkin, Kunal Handa, Avash Shrestha, and Noah Goodman. 2022.  Task ambiguity in humans and language models.  _ArXiv preprint_ , abs/2212.10711.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023.  Llama: Open and efficient foundation language models.  _ArXiv preprint_ , abs/2302.13971.
  * van der Poel et al. (2022) Liam van der Poel, Ryan Cotterell, and Clara Meister. 2022.  Mutual information alleviates hallucinations in abstractive summarization.  In _Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing_ , pages 5956–5965, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Varshney and Baral (2023) Neeraj Varshney and Chitta Baral. 2023.  Post-abstention: Towards reliably re-attempting the abstained instances in QA.  In _ACL (1)_ , pages 967–982. Association for Computational Linguistics.
  * Vazhentsev et al. (2022) Artem Vazhentsev, Gleb Kuzmin, Artem Shelmanov, Akim Tsvigun, Evgenii Tsymbalov, Kirill Fedyanin, Maxim Panov, Alexander Panchenko, Gleb Gusev, Mikhail Burtsev, Manvel Avetisian, and Leonid Zhukov. 2022.  Uncertainty estimation of transformer predictions for misclassification detection.  In _Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 8237–8252, Dublin, Ireland. Association for Computational Linguistics.
  * Vazhentsev et al. (2023a) Artem Vazhentsev, Gleb Kuzmin, Akim Tsvigun, Alexander Panchenko, Maxim Panov, Mikhail Burtsev, and Artem Shelmanov. 2023a.  Hybrid uncertainty quantification for selective text classification in ambiguous tasks.  In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 11659–11681.
  * Vazhentsev et al. (2023b) Artem Vazhentsev, Akim Tsvigun, Roman Vashurin, Sergey Petrakov, Daniil Vasilev, Maxim Panov, Alexander Panchenko, and Artem Shelmanov. 2023b.  Efficient out-of-domain detection for sequence to sequence models.  In _Findings of the Association for Computational Linguistics: ACL 2023_ , pages 1430–1454.
  * Verma et al. (2019) Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, and Yoshua Bengio. 2019.  Manifold mixup: Better representations by interpolating hidden states.  In _Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA_ , volume 97 of _Proceedings of Machine Learning Research_ , pages 6438–6447. PMLR.
  * Wang et al. (2023a) Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, Xiangru Tang, Tianhang Zhang, Jiayang Cheng, Yunzhi Yao, Wenyang Gao, Xuming Hu, Zehan Qi, Yidong Wang, Linyi Yang, Jindong Wang, Xing Xie, Zheng Zhang, and Yue Zhang. 2023a.  Survey on factuality in large language models: Knowledge, retrieval and domain-specificity.  _ArXiv preprint_ , abs/2310.07521.
  * Wang et al. (2020) Shuo Wang, Zhaopeng Tu, Shuming Shi, and Yang Liu. 2020.  On the inference calibration of neural machine translation.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 3070–3079, Online. Association for Computational Linguistics.
  * Wang et al. (2023b) Xiaosu Wang, Yun Xiong, Beichen Kang, Yao Zhang, Philip S Yu, and Yangyong Zhu. 2023b.  Reducing negative effects of the biases of language models in zero-shot setting.  In _Proceedings of the Sixteenth ACM International Conference on Web Search and Data Mining_ , pages 904–912.
  * Wang et al. (2023c) Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2023c.  Do-not-answer: A dataset for evaluating safeguards in LLMs.  _ArXiv preprint_ , abs/2308.13387.
  * Wei et al. (2022) Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022.  Chain-of-thought prompting elicits reasoning in large language models.  _Advances in Neural Information Processing Systems_ , 35:24824–24837.
  * Wei and Zou (2019) Jason Wei and Kai Zou. 2019.  EDA: Easy data augmentation techniques for boosting performance on text classification tasks.  In _Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)_ , pages 6382–6388, Hong Kong, China. Association for Computational Linguistics.
  * Xiao and Wang (2021) Yijun Xiao and William Yang Wang. 2021.  On hallucination and predictive uncertainty in conditional language generation.  In _Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume_ , pages 2734–2744, Online. Association for Computational Linguistics.
  * Xiao et al. (2022) Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, and Louis-Philippe Morency. 2022.  Uncertainty quantification with pre-trained language models: A large-scale empirical analysis.  In _Findings of the Association for Computational Linguistics: EMNLP 2022_ , pages 7273–7284, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
  * Xiong et al. (2023) Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2023.  Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms.  _ArXiv preprint_ , abs/2306.13063.
  * Yin et al. (2023) Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023.  Do large language models know what they don’t know? _ArXiv preprint_ , abs/2305.18153.
  * Yoo et al. (2022) KiYoon Yoo, Jangho Kim, Jiho Jang, and Nojun Kwak. 2022.  Detection of word adversarial examples in text classification: Benchmark and baseline via robust density estimation.  _ArXiv preprint_ , abs/2203.01677.
  * Yoshikawa and Okazaki (2023) Hiyori Yoshikawa and Naoaki Okazaki. 2023.  Selective-LAMA: Selective prediction for confidence-aware evaluation of language models.  In _Findings of the Association for Computational Linguistics: EACL 2023_ , pages 2017–2028, Dubrovnik, Croatia. Association for Computational Linguistics.
  * Zablotskaia et al. (2023) Polina Zablotskaia, Du Phan, Joshua Maynez, Shashi Narayan, Jie Ren, and Jeremiah Liu. 2023.  On uncertainty calibration and selective generation in probabilistic neural summarization: A benchmark study.  _ArXiv preprint_ , abs/2304.08653.
  * Zhang et al. (2018) Hongyi Zhang, Moustapha Cissé, Yann N. Dauphin, and David Lopez-Paz. 2018.  mixup: Beyond empirical risk minimization.  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.
  * Zhang et al. (2020a) Jingqing Zhang, Yao Zhao, Mohammad Saleh, and Peter J. Liu. 2020a.  PEGASUS: pre-training with extracted gap-sentences for abstractive summarization.  In _Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event_ , volume 119 of _Proceedings of Machine Learning Research_ , pages 11328–11339. PMLR.
  * Zhang et al. (2021) Shujian Zhang, Chengyue Gong, and Eunsol Choi. 2021.  Knowing more about questions can help: Improving calibration in question answering.  In _Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021_ , pages 1958–1970, Online. Association for Computational Linguistics.
  * Zhang et al. (2020b) Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020b.  Bertscore: Evaluating text generation with BERT.  In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
  * Zhang et al. (2023) Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, et al. 2023.  Siren’s song in the ai ocean: A survey on hallucination in large language models.  _ArXiv preprint_ , abs/2309.01219.
  * Zhao et al. (2023) Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. 2023.  Slic-hf: Sequence likelihood calibration with human feedback.  _ArXiv preprint_ , abs/2305.10425.
  * Zhao et al. (2022) Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. 2022.  Calibrating sequence likelihood improves conditional language generation.  _ArXiv preprint_ , abs/2210.00045.
  * Zhao et al. (2021) Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021.  Calibrate before use: Improving few-shot performance of language models.  In _Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event_ , volume 139 of _Proceedings of Machine Learning Research_ , pages 12697–12706. PMLR.
  * Zheng et al. (2023a) Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2023a.  Large language models are not robust multiple choice selectors.  _ArXiv preprint_ , abs/2309.03882.
  * Zheng et al. (2023b) Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2023b.  On large language models’ selection bias in multi-choice questions.  _ArXiv preprint_ , abs/2309.03882.
  * Zhou et al. (2023) Kaitlyn Zhou, Dan Jurafsky, and Tatsunori Hashimoto. 2023.  Navigating the grey area: Expressions of overconfidence and uncertainty in language models.  _ArXiv preprint_ , abs/2302.13439.
  * Zhu et al. (2023) Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023.  Minigpt-4: Enhancing vision-language understanding with advanced large language models.  _ArXiv preprint_ , abs/2304.10592.

##  Appendix A Appendix

Study |  Model |  Task |  Calibration Methods
---|---|---|---
Studies of calibrating discriminative LMs
Desai and Durrett (2020) |  BERT, RoBERTa |  Nature Language Inference, Paraphrase Detection, Commonsense Reasoning |  TS, LS
Kim et al. (2023) |  RoBERTa |  Text Classification |  BL, ERL, MixUp, DeepEnsemble, MCDropout, MIMO
Park and Caragea (2022) |  BERT, RoBERTa |  Nature Language Inference, Paraphrase Detection, Commonsense Reasoning |  TS, LS, MixUp, Manifold-MixUp, AUM-guided MixUp
Zhang et al. (2021) |  BERT-based Span Extractor |  Extractive Question Answering |  FBC
Si et al. (2022) |  BERT-based Span Extractor |  Extractive Question Answering |  LS, TS, FBC, CONSCAL
Studies of calibrating generative LMs
Kumar and Sarawagi (2019) |  LSTM, Transformer |  Machine Translation |  TS with learnable parameters
Lu et al. (2022) |  Transformer |  Machine Translation |  Learned-Confidence-Based LS
Wang et al. (2020) |  Transformer |  Machine Translation |  LS, Dropout
Xiao and Wang (2021) |  LSTM, Transformer |  Data2Text Generation, Image Captioning |  Uncertainty-Aware Decoding
van der Poel et al. (2022) |  BART Lewis et al. (2020a) |  Text Summarization |  CPMI-Based Decoding
Zablotskaia et al. (2023) |  T5 Raffel et al. (2020) |  Text Summarization |  MC-Dropout, BE, SNGP, DeepEnsemble
Zhao et al. (2022) |  PEGASUS Zhang et al. (2020a) |  Text Summarization, Question Answering |  SLiC
Zhao et al. (2023) |  T5 Raffel et al. (2020) |  Text Summarization |  SLiC-HF
Mielke et al. (2022) |  BlenderBot |  Dialogue Generation |  Linguistic Calibration
Lin et al. (2022) |  GPT-3 |  Math Question Answering |  Fine-Tuning
Zhao et al. (2021) |  GPT-3 |  In-Context Learning |  Contextual Calibration
Fei et al. (2023) |  PALM-2 Anil et al. (2023), CLIP Radford et al. (2021) |  In-Context Learning |  Domain-Context Calibration
Han et al. (2022) |  GPT-2 |  In-Context Learning |  Prototypical Calibration
Kumar (2022) |  GPT-2 |  Multiple Choice Question Answering |  Answer-Level Calibration
Holtzman et al. (2021) |  GPT-2, GPT-3 |  Multiple Choice Question Answering |  PMIDC (Domain Conditional Pointwise Mutual Information)
Zheng et al. (2023a) |  LLaMA Touvron et al. (2023), Vicuna Chiang et al. (2023), Falcon Penedo et al. (2023), GPT-3.5 |  Multiple Choice Question Answering |  PriDE (debiasing with prior estimation)

Table 1: Studies of LM calibration. Top: discriminative LM; Bottom: generative
LM. Task: NLI=nature language inference, PD=paraphrase detection,
CR=commonsense reasoning, EQA=extractive question answering, Calibration
methods: LS=label smoothing, TS=temperature scaling, BL=brier loss,
ERL=entropy regularization loss, BE=Bayesian Ensemble, SNGP: spectral-
normalized Gaussian process, FBC=feature-based calibrator, CPMI=conditional
pointwise mutial information.