Proximity-Informed Calibration for Deep Neural Networks
Confidence calibration is central to providing accurate and interpretable
uncertainty estimates, especially under safety-critical scenarios. However, we
find that existing calibration algorithms often overlook the issue of
*proximity bias*, a phenomenon where models tend to be more overconfident in
low proximity data (i.e., data lying in the sparse region of the data
distribution) compared to high proximity samples, and thus suffer from
inconsistent miscalibration across different proximity samples. We examine the
problem over 504 pretrained ImageNet models and observe that: 1) Proximity bias
exists across a wide variety of model architectures and sizes; 2)
Transformer-based models are relatively more susceptible to proximity bias than
CNN-based models; 3) Proximity bias persists even after performing popular
calibration algorithms like temperature scaling; 4) Models tend to overfit more
heavily on low proximity samples than on high proximity samples. Motivated by
the empirical findings, we propose ProCal, a plug-and-play algorithm with a
theoretical guarantee to adjust sample confidence based on proximity. To
further quantify the effectiveness of calibration algorithms in mitigating
proximity bias, we introduce proximity-informed expected calibration error
(PIECE) with theoretical analysis. We show that ProCal is effective in
addressing proximity bias and improving calibration on balanced, long-tail, and
distribution-shift settings under four metrics over various model
architectures. We believe our findings on proximity bias will guide the
development of *fairer and better-calibrated* models, contributing to the
broader pursuit of trustworthy AI. Our code is available at:
https://github.com/MiaoXiong2320/ProximityBias-Calibration.
Miao Xiong, Ailin Deng, Pang Wei Koh, Jiaying Wu, Shen Li, Jianqing Xu, Bryan Hooi
# Proximity-Informed Calibration for Deep Neural Networks

###### Abstract

Confidence calibration is central to providing accurate and interpretable uncertainty estimates, especially under safety-critical scenarios. However, we find that existing calibration algorithms often overlook the issue of _proximity bias_, a phenomenon where models tend to be more overconfident in low proximity data (i.e., data lying in the sparse region of the data distribution) compared to high proximity samples, and thus suffer from inconsistent miscalibration across different proximity samples. We examine the problem over \(504\) pretrained ImageNet models and observe that: 1) Proximity bias exists across a wide variety of model architectures and sizes; 2) Transformer-based models are more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms like temperature scaling; 4) Models tend to overfit more heavily on low proximity samples than on high proximity samples. Motivated by the empirical findings, we propose ProCal, a plug-and-play algorithm with a theoretical guarantee to adjust sample confidence based on proximity. To further quantify the effectiveness of calibration algorithms in mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis. We show that ProCal is effective in addressing proximity bias and improving calibration on balanced, long-tail, and distribution-shift settings under four metrics over various model architectures. 2

Footnote 2: Our codes are available at: [https://github.com/MiaoXiong2320/ProximityBias-Calibration.git](https://github.com/MiaoXiong2320/ProximityBias-Calibration.git)

## 1 Introduction

Machine learning systems are increasingly deployed in high-stakes and safety-critical applications such as autonomous driving and medical diagnosis [30; 36; 9; 6], where incorrect decisions can have severe human health consequences. To ensure safe and reliable deployment, _confidence calibration_ approaches [10; 23; 28] are employed to produce more accurate uncertainty estimates, which allow models to establish trust by communicating their level of uncertainty, and to defer to human decision-making when the models are uncertain.

In this paper, we present a calibration-related phenomenon termed _proximity bias_, which refers to the tendency of current deep classifiers to exhibit higher levels of overconfidence on samples of low proximity, i.e., samples in sparse areas within the data distribution (see Figure 1bc for an illustrative example). In this study, we quantify the proximity of a sample as the average distance to its \(K\) (e.g. \(K=10\)) nearest neighbor samples in the data distribution, and we observe that proximity bias holds for various choices of \(K\). Importantly, the phenomenon persists even after applying existing popular calibration methods, leading to different levels of miscalibration across proximities.

The proximity bias issue raises safety concerns in real-world applications, particularly for underrepresented populations (i.e. low proximity samples) [30; 35]. A recent skin cancer analysis highlights
this concern by revealing that AI-powered models demonstrate high performance for light-skinned individuals but struggle with dark-skinned individuals due to their underrepresentation [11]. This issue can also manifest in the form of proximity bias: suppose a dark-skinned individual has a high risk of 40% of having the cancer. However, due to their underrepresentation within the data distribution, the model overconfidently assigns them 98% confidence of not having cancer. As a result, these low proximity individuals may be deprived of timely intervention.

To study the ubiquity of this problem, we examine \(504\) ImageNet pretrained models from the timm library [41] and make the following key observations: 1) Proximity bias exists generally across a wide variety of model architectures and sizes; 2) Transformer-based models are more susceptible to proximity bias than CNN-based models; 3) Proximity bias persists even after performing popular calibration algorithms including temperature scaling; 4) Low proximity samples are more prone to model overfitting while high proximity samples are less susceptible to this issue.

Besides, we argue that proximity bias is overlooked by _confidence calibration_. Revisiting its definition, \(\mathbb{P}(Y=\hat{Y}\mid\hat{P}=p)=p\) for all \(p\in[0,1]\), we find that its primary goal is to match confidence with the accuracy of samples sharing the same confidence level. However, Figure 1a reveals that although the model seems well-calibrated within each confidence group, there still exists miscalibration errors among these groups (e.g. low and high proximity samples) due to proximity bias.

Motivated by this, we propose a debiased variant of the expected calibration error (ECE) metric, called proximity-informed expected calibration error (PIECE) to further capture the miscalibration error due to proximity bias. The effectiveness is supported by our theoretical analysis that PIECE is at least as large as ECE. This equality holds when there is no cancellation effect with respect to proximity bias.

To tackle proximity bias and further improve confidence calibration, we propose a plug-and-play method, ProCal. Intuitively, ProCal learns a joint distribution of proximity and confidence to adjust probability estimates. To fully leverage the characteristics of the input information, we develop two separate algorithms tailored for continuous and discrete inputs. We evaluate the algorithms on large-scale datasets: **balanced datasets** including ImageNet [7] and Yahoo-Topics[47], **long-tail datasets**iNaturalist 2021 [3] and ImageNet-LT[27] and **distribution-shift datasets**MultiNLI [42] and ImageNet-C[15]. The results show that our algorithm consistently improves the performance of existing algorithms under four metrics with 90% significance (p-value < 0.1).

Our main contributions can be summarized as follows:

* **Findings**: We discover the proximity bias issue and show its prevalence over large-scale analysis (\(504\) ImageNet pretrained models).
* **Metrics**: To quantify the effectiveness of mitigating proximity bias, we introduce proximity-informed expected calibration error (PIECE) with theoretical analysis.
* **Method Effectiveness**: To address proximity bias and improve calibration, we propose a plug-and-play algorithm ProCal with the theoretical guarantee. We verify its effectiveness on image and text datasets with balanced, distribution-shift, and long-tail settings.

## 2 Related Work

Confidence CalibrationConfidence calibration aims to yield uncertainty estimates via aligning a model's confidence with the accuracy of samples with the same confidence level [10; 25; 28]. To

Figure 1: **Samples with lower (higher) proximity tend to be more overconfident (underconfident)**. These results are obtained using XCiT, an Image Transformer, on the ImageNet validation set (**All Samples**). The proximity of a sample is measured as its average distance to its nearest neighbors (\(K=10\)) in the validation set. We split samples into \(10\) equal-size bins based on proximity and choose the bin with the highest proximity (**High Proximity Samples**) and lowest proximity (**Low Proximity Samples**).
achieve this, **Scaling-based** methods represented by temperature scaling [10] adjust the predicted probabilities by learning a temperature scalar for all samples. Similarly, parameterized temperature scaling [39] offers improved expressiveness via input-dependent temperature parameterization, and Mix-n-Match [46] adopts ensemble and composition strategies to yield data-efficient and accuracy-preserving estimates. **Binning-based** methods divide samples into multiple bins based on confidence and calibrate each bin. Popular methods include classic histogram binning [44], mutual-information-maximization-based binning [32], and isotonic regression [45]. However, existing calibration methods overlook the proximity bias issue, which fundamentally limits the methods' capabilities in delivering reliable and interpretable uncertainty estimates.

MulticalibrationMulticalibration algorithms [13; 21] aim to achieve a certain level of fairness by ensuring that a predictor is well-calibrated for the overall population as well as different computationally-identifiable subgroups. [33] proposes a grouping loss to evaluate subgroup calibration error while we propose a metric to integrate the group cancellation effect into existing calibration loss. [21] focuses on understanding the fundamental trade-offs between group calibration and other fairness criteria, and [13] proposes a conceptual iterative algorithm to learn a multi-calibrated predictor. In this regard, our proposed framework can be considered a specific implementation of the fairness objectives outlined in [13], with a particular focus on proximity-based subgroups. This approach offers easier interpretation and implementation compared to subgroups discussed in [13; 21].

## 3 What is Proximity Bias?

In this section, we study the following questions: What is proximity bias? When and why does proximity bias occur?

BackgroundWe consider a supervised multi-class classification problem, where input \(X\in\mathcal{X}\) and its label \(Y\in\mathcal{Y}=\{1,2,\cdots,C\}\) follows a joint distribution \(\pi(X,Y)\). Let \(f\) be a classifier with \(f(X)=(\hat{Y},\hat{P})\), where \(\hat{Y}\) represents the predicted label, and \(\hat{P}\) is the model's confidence, i.e. the estimate of the probability of correctness [10]. For simplicity, we use \(\hat{P}\) to denote both the model's confidence and the confidence calibrated using existing calibration algorithms.

ProximityWe define proximity as a function of the average distance between a sample and its \(K\) nearest neighbors in the data distribution:

\[D(X)=\exp\left(-\frac{1}{K}\sum_{X_{i}\in\mathcal{N}_{K}(X)}\mathrm{dist}(X,X_ {i})\right), \tag{1}\]

where \(\mathcal{N}_{K}(X)\) represents the set of \(K\) nearest neighbors of sample \(X\), and \(\mathrm{dist}(X,X_{i})\) denotes the distance between sample \(X\) and its \(i\)-th nearest neighbor \(X_{i}\), estimated using the Euclidean distance between the features of \(X\) and \(X_{i}\) from the model's penultimate layer. \(K\) is a hyperparameter (we set \(K=10\) in this paper). We use the validation set as a proxy to estimate the data distribution. Although the training set can also be employed to compute proximity, we utilize the validation set because it is readily accessible during the calibration process.

This definition allows us to capture the local density of a sample and its relationship to its neighborhood. For instance, a sample situated in a sparse region of the training distribution would receive a low proximity value, while a sample located in a dense region would receive a high proximity value. Samples with low proximity values represent **underrepresented samples** in the data distribution that merit attention, such as rare ("long-tail") diseases, minority populations, and samples with distribution shift.

Proximity BiasTo investigate the relationship between proximity and model miscalibration, we define proximity bias as follows:

**Definition 3.1**.: Given any confidence level \(p\), the model suffers from proximity bias if the following condition does not hold:

\[\mathbb{P}\left(\hat{Y}=Y\mid\hat{P}=p,D=d_{1}\right)=\mathbb{P}\left(\hat{Y}= Y\mid\hat{P}=p,D=d_{2}\right)\quad\forall\;d_{1},d_{2}\in(0,1],d_{1}\neq d_{2}.\]

The intuition behind this definition is that, ideally, a sample with a confidence level of \(p\) should have a probability of being correct equal to \(p\), regardless of proximity. However, if low proximity samples
consistently display higher confidence than high proximity samples (as shown in Figure 1), it can lead to unreliable and unjust decision-making, particularly for underrepresented populations.

### Main Empirical Findings

To showcase the ubiquity of this problem, we examine the proximity bias phenomenon on \(504\) ImageNet pretrained models from the timm library [41] and show the results in Figure 2 (see Appendix D for additional figures and analysis).

We use statistical hypothesis testing to investigate the presence of proximity bias. The null hypothesis \(H_{0}\) is that proximity bias does not exist, formally, for any confidence \(p\) and proximities \(d_{1}>d_{2}\):

\[\mathbb{P}\left(\hat{Y}=Y\mid\hat{P}=p,D=d_{1}\right)=\mathbb{P}\left(\hat{Y} =Y\mid\hat{P}=p,D=d_{2}\right). \tag{2}\]

To test the above null hypothesis, i.e., to evaluate whether there is a significant difference in the sample means (i.e. accuracy) of the two selected groups, we use the Wilcoxon rank-sum test [24]. Specifically, we split the samples into 5 equal-sized proximity groups and select the highest and lowest proximity groups. Every time we randomly sample a point from the highest (or lowest) group and search for a point from the lowest (or highest) group _with the same confidence_. To ensure statistical significance, we repeat this process 20,000 times. We call the resulting sampled points the _confidence-matched high and low proximity groups_\(B_{H}\), \(B_{L}\), and apply the Wilcoxon rank-sum test to evaluate whether the mean difference in their accuracy is significantly different from zero.

The hypothesis testing results indicate that over 80% of \(504\) models have a p-value less than \(0.05\) (72% after Bonferroni correction [4]), i.e., the null hypothesis is rejected with a confidence level of at least 95%, indicating that proximity bias plagues most of the models in timm.

Inspired by the hypothesis testing, we define **Bias Index** as the accuracy drop between the confidence-matched high proximity group \(B_{H}\) and low proximity group \(B_{L}\) to reflect the degree of bias:

\[\text{Bias Index}=\frac{\sum_{(X,Y)\in B_{H}}\mathbbm{1}\{\hat{Y}=Y\}}{|B_{H}|} -\frac{\sum_{(X,Y)\in B_{L}}\mathbbm{1}\{\hat{Y}=Y\}}{|B_{L}|}. \tag{3}\]

Figure 2: Proximity bias analysis on \(504\) public models. Each marker represents a model, where marker sizes indicate model parameter numbers and different colors/shapes represent different architectures. The bias index is computed using Equation (3) (0 indicates no proximity bias). **Left**: We observed the following: 1) Models with higher accuracy tend to have a larger bias index. 2) Proximity bias exists across a wide range of model architectures. 3) Transformer variants (e.g. DEiT, XCiT, CaiT, and SwinV2) have a relatively larger bias compared to convolution-based networks (e.g. VGG and ResNet variants). **Right**: Confidence calibrated by temperature scaling (Upper Right) is similar to the original model confidence w.r.t proximity bias. Our ProCal (Bottom Right) is effective in reducing proximity bias. Analysis of other existing calibration algorithms can be found in Appendix D.
Note that \(B_{H},B_{L}\) are obtained from the hypothesis testing process and hence have the same mean confidence.

We show the bias index of \(504\) models in Figure 2 and make the following findings:

**1. Proximity bias exists generally across a wide variety of model architecture and sizes.** Figure 2 shows that most models (85% of the models as supported by hypothesis testing) have a bias index larger than 0, indicating the existence of proximity bias.

**2. Transformer-based methods are more susceptible to proximity bias than CNN-based methods.** In Figure 2, models with lower accuracy (primarily CNN-based models such as VGG, EfficientNet, MobileNet, and ResNet [12] variants) tend to have lower bias index. On the other hand, among models with higher accuracy, Transformer variants (e.g., DEiT, XCiT [1], CaiT [40], and SwinV2) demonstrate relatively higher bias compared to convolution-based networks (e.g., ResNet variants). This is concerning given the increasing popularity of Transformer-based models in recent years and highlights the need for further research to study and address this issue.

**3. Popular calibration methods such as temperature scaling do not noticeably alleviate proximity bias.** Figure 2 (upper right) shows that the proximity bias index remains large even after applying temperature scaling, indicating that this method does not noticeably alleviate the problem. In contrast, Figure 2c demonstrates that our proposed approach successfully shifts the models to a much closer distribution around the line \(y=0\) (indicating no proximity bias). The bias index figures for more existing calibration methods are provided in Appendix D.

**4. Low proximity samples are more prone to model overfitting.** Figure 4 in Appendix D shows that the model's accuracy difference between the training and validation set is more significant on low proximity samples (31.67%) compared to high proximity samples (0.6%). This indicates that the model generalizes well on samples of high proximity but tends to overfit on samples of low proximity. The overconfidence of low proximity samples can be a consequence of the overfitting tendency, as the overfitting gap also reflects the mismatch between the model's confidence and its actual accuracy.

## 4 Proximity-Informed ECE

As depicted in Figure 1, _existing evaluation metrics underestimate the true miscalibration level_, as proximity bias causes certain errors in the model to cancel out. As an example, consider a scenario:

**Example 4.1**.: All samples are only drawn from two proximity groups of equal probability mass, \(d=0.2\) and \(d=0.8\), with true probabilities of \(\mathbb{P}(Y=\hat{Y}|X,f)\) being \(0.5\) and \(0.9\), respectively. The model outputs the confidence score \(p=0.7\) to all samples.

We consider the most commonly used metric, expected calibration error (ECE) that is defined as \(\mathrm{ECE}=\mathbb{E}_{\hat{P}}\left[\left|\mathbb{P}(\hat{Y}=Y\mid\hat{P})- \hat{P}\right|\right]\). In Example 4.1, the ECE is 0, suggesting that the model is perfectly calibrated in terms of confidence calibration. In fact, the model has significant miscalibration issues: it is heavily overconfident in one proximity group while heavily underconfident in the other, highlighting the limitations of existing calibration metrics. The miscalibration errors within the same confidence group are _canceled out_ by samples with both high and low proximity, resulting in a phenomenon we term _cancellation effect_.

To further evaluate the miscalibration canceled out by proximity bias, we propose the proximity-informed expected calibration error (PIECE). PIECE is defined in an analogous fashion as ECE, yet it further examines information about the proximity of the input sample, \(D(X)\), in the calibration evaluation:

\[\mathrm{PIECE}=\mathbb{E}_{\hat{P},D}\left[\left|\mathbb{P}(\hat{Y}=Y\mid\hat {P},D)-\hat{P}\right|\right]. \tag{4}\]

Back to Example 4.1 where \(\mathrm{ECE}=0\), we have PIECE \(=0.2\), revealing its miscalibration level in the subpopulations of different proximities, i.e., the calibration error regarding proximity bias. Additionally, we demonstrate that PIECE is _always at least as large_ as ECE, with the equality holding only when there is no cancellation effect w.r.t proximity. The detailed proof is relegated to Appendix B.

**Theorem 4.2** (PIECE captures cancellation effect.).: _Given any joint distribution \(\pi(X,Y)\) and any classifier \(f\) that outputs model confidence \(\hat{P}\) for sample \(X\), we have the following inequality, where
equality holds only when there is no cancellation effect with respect to proximity:_

\[\underbrace{\mathbb{E}_{\hat{P}}\left[\left|\mathbb{P}(\hat{Y}=Y\mid\hat{P})-\hat{ P}\right|\right]}_{\text{ECE}}\leq\underbrace{\mathbb{E}_{\hat{P},D}\left[\left| \mathbb{P}(\hat{Y}=Y\mid\hat{P},D)-\hat{P}\right|\right]}_{\text{PIECE}}.\]

## 5 How to Mitigate Proximity Bias?

In this section, we propose ProCal to achieve three goals: 1) **mitigate proximity bias**, i.e., ensure samples with the same confidence level have the same miscalibration gap across all proximity levels, 2) **improve confidence calibration** by reducing overconfidence and underconfidence, and 3) provide a **plug-and-play** method that can combine the strengths of existing approaches with our proximity-informed approach.

The high-level intuition is to explicitly incorporate proximity when estimating the underlying probability of the model prediction being correct. To fully leverage the distinct properties of the input information, we develop two separate algorithms tailored for continuous and discrete inputs. This differentiation is based on the observation that continuous outputs (e.g., those produced by scaling-based methods) contain rich distributional information suitable for density estimation. On the other hand, discrete inputs (e.g., those generated by binning-based methods) allow for robust binning-based adjustments. By treating these inputs separately, we can effectively harness the characteristics of each type.

### Continuous Confidence: Density-Ratio Calibration

Our objective is to estimate the likelihood of a model prediction \(\hat{Y}\) being identical to the ground truth label \(Y\) for every sample \(X\). Computing this probability directly with density estimation methods can be computationally demanding, particularly in high-dimensional spaces. To circumvent the curse of dimensionality and address proximity bias, we incorporate the model confidence \(\hat{P}\) and proximity information \(D(X)\) to estimate the posterior probability of correctness, i.e., \(\mathbb{P}(\hat{Y}=Y\mid\hat{P},D)\). This approach is data-efficient since it conducts density estimation in a two-dimensional space only, rather than in the higher dimensional feature or prediction simplex space.

Consider a test sample \(X\) with proximity \(D=D(X)\) and uncalibrated confidence score \(\hat{P}\), which can be the standard Maximum Softmax Probability (MSP), or the output of any calibration method. \(\mathbb{P}(\hat{Y}=Y\mid\hat{P},D)\) can be computed via Bayes' rule:

\[\mathbb{P}\left(\hat{Y}=Y\mid\hat{P},D\right)=\frac{\mathbb{P}\left(\hat{P},D \mid\hat{Y}=Y\right)\mathbb{P}\left(\hat{Y}=Y\right)}{\mathbb{P}(\hat{P},D)},\]

where \(\hat{Y}\) is the model prediction and \(Y\) is the ground truth label. This can be re-expressed as follows by using the law of total probability:

\[\frac{\mathbb{P}(\hat{P},D\mid\hat{Y}=Y)}{\mathbb{P}\left(\hat{P},D\mid\hat{Y }=Y\right)+\mathbb{P}\left(\hat{P},D\mid\hat{Y}\neq Y\right)\cdot\frac{ \mathbb{P}\left(\hat{Y}\neq Y\right)}{\mathbb{P}\left(\hat{Y}=Y\right)}}.\]

To compute this calibrated score, we need to estimate the distributions \(\mathbb{P}\left(\hat{P},D\mid\hat{Y}=Y\right)\) and \(\mathbb{P}\left(\hat{P},D\mid\hat{Y}\neq Y\right)\), and the class ratio \(\frac{\mathbb{P}\left(\hat{Y}\neq Y\right)}{\mathbb{P}\left(\hat{Y}=Y\right)}\).

To estimate the probability density functions \(\mathbb{P}\left(\hat{P},D\mid\hat{Y}=Y\right)\) and \(\mathbb{P}\left(\hat{P},D\mid\hat{Y}\neq Y\right)\), various density estimation methods can be used, such as parametric methods like Gaussian mixture models or non-parametric methods like kernel density estimation (KDE)[31]. We choose KDE because it is flexible and robust, making no assumptions about the underlying distribution. Specifically, we split samples into two groups based on whether they are correctly classified and then use KDE to estimate the two densities. To obtain the class ratio \(\frac{\mathbb{P}\left(\hat{Y}\neq Y\right)}{\mathbb{P}\left(\hat{Y}=Y\right)}\), we simply use the ratio of the number of correctly classified samples and the number of misclassified samples in the validation set. The **pseudocode** for inference and training can be found in Appendix 2 and 1.
### Discrete Confidence: Bin Mean-Shift

The Bin-Mean-Shift approach aims to first use 2-dimensional binning to estimate the joint distribution of proximity \(D\) and input confidence \(\hat{P}\) and then estimate \(\mathbb{P}\left(\hat{Y}=Y\mid\hat{P},D\right)\). Considering test samples \(X\) with proximity \(D=D(X)\) and uncalibrated confidence score \(\hat{P}\), we first group samples into 2-dimensional equal-size bins based on their \(D\) and \(\hat{P}\) (other binning schemes can also be used; we choose quantile for simplicity). Next, for each bin \(B_{mh}\), we calculate its accuracy \(\mathcal{A}(B_{mh})\) and mean confidence \(\mathcal{F}(B_{mh})\). Then, for all the samples in the bin, we adjust their confidence scores as follows:

\[\hat{P}_{ours}=\hat{P}+\lambda\cdot\left(\mathcal{A}(B_{mh})-\mathcal{F}(B_{mh })\right), \tag{5}\]

where the weight hyperparameter \(\lambda\in(0,1]\) is a constant, acting as regularization to stabilize the adjustment.

Note that our approach (i.e. applying a mean-shift in each bin) differs from the typical histogram binning method (replacing the confidence scores with its mean accuracy in each bin). Rather than completely replacing the input confidence scores \(\hat{P}\) (which are often reasonably well-calibrated), our approach better utilizes these scores by only adjusting them by the minimal mean-shift needed to correct for proximity bias in each of the 2-dimensional bins.

### Theoretical Guarantee

Here we present that our method, Bin-Mean-Shift, can consistently achieve a smaller Brier Score given a sufficient amount of data in the context of binary classification. The Brier Score [5] is a strictly proper score function that measures both calibration and accuracy aspects [22], with a smaller value indicating better performance. As illustrated below, our algorithm's Brier Score is asymptotically bounded by the original Brier Score, augmented by a non-negative term.

**Theorem 5.1** (Brier Score after Bin-Mean-Shift is asymptotically bounded by Brier Score before calibration).: _Given a joint data distribution \(\pi(X,Y)\) and a binary classifier \(\hat{f}\), for any calibration algorithm \(h\) that outputs score \(h(\hat{P})\) based on model confidence \(\hat{P}\), we apply Bin-Mean-Shift to derive calibrated score \(\tilde{h}(\hat{P})\) as defined in Equation (5). Let \(h_{c}(\hat{P})=h(\hat{P})\times\mathbb{1}\{\hat{Y}=1\}+(1-h(\hat{P}))\times \mathbb{1}\{\hat{Y}=0\}\) denote the probability assigned to class 1 by \(h(\hat{P})\), and define \(\tilde{h}_{c}(\hat{P})\) similarly. Then, the Brier Score before calibration can be decomposed as follows:_

\[\underbrace{\mathbb{E}_{\pi(X,Y)}\left[\left(h_{c}(\hat{P})-Y\right)^{2} \right]}_{\text{Brier Score before Calibration}}=\underbrace{\mathbb{E}_{\pi(X,Y)} \left[\left(\tilde{h}_{c}(\hat{P})-Y\right)^{2}\right]}_{\text{Brier Score after Calibration}}+\underbrace{\mathbb{E}_{B\sim\mathbb{P}(B)}\left[\left(\hat{\mathcal{A}}(B)- \hat{\mathcal{F}}(B)\right)^{2}\right]}_{\geq 0}+o(1),\]

_where \(\mathbb{P}(B)\) is determined by the binning mechanism used in Bin-Mean-Shift._

Remark.Note that when the calibration algorithm \(h\) is an identity mapping, it demonstrates that Bin-Mean-Shift achieves better model calibration performance than the original model confidence, given a sufficient amount of data. The detailed proof is relegated to Appendix A.

## 6 Experiments

In this section, we aim to answer the following questions:

* **Performance across different datasets and model architectures:** How does ProCal perform on datasets with balanced distribution, long-tail distribution, and distribution shift, as well as on different model architectures?
* **Inference efficiency:** How efficient is our ProCal? (see Appendix E.1)
* **Hyperparameter sensitivity**: How sensitive is ProCal to different hyperparameters, e.g. neighbor size \(K^{2}\) (See Appendix F.2)
* **Ablation study:** What is the difference between Density-Ratio and Bin-Mean-shift on calibration? How should the choice between these techniques be determined? (See Appendix F.1)
### Experiment Setup

Evaluation Metrics.Following [10], we adopt 3 commonly used metrics to evaluate the _confidence calibration_: Expected Calibration Error (ECE), Adaptive Calibration Error (ACE[29]), Maximum Calibration Error (MCE) and our proposed PIECE to evaluate the _bias mitigation_ performance.

Datasets.We evaluate the effectiveness of our approach across large-scale datasets of three types of data characteristics (balanced, long-tail and distribution-shifted) in image and text domains: (1) Dataset with **balanced** class distribution (i.e. each class has an equal size of samples) on vision dataset ImageNet [7] and two text datasets including Yahoo Answers Topics [47] and MultiNLI-match [42]; (2) Datasets with **long-tail** class distribution on two image datasets, including iNaturalist 2021 [3] and ImageNet-LT [27]; (3) Dataset with **distribution-shift** on two datasets, including ImageNet-C [15] and MultiNLI-Mismatch [42].

Comparison methods.We compare our method to existing calibration algorithms: base confidence score (Conf)[16], _scaling-based methods_ such as Temperature Scaling (TS) [10], Ensemble Temperature Scaling (ETS) [46], Parameterized Temperature Scaling (PTS) [39], Parameterized Temperature Scaling with K Nearest Neighbors (PTSK), and _binning based methods_ such as Histogram Binning (HB), Isotonic Regression (IR) and Multi-Isotonic Regression (MIR) [46]. Throughout the experiment section, we apply Density-Ratio Calibration to Conf, TS, ETS, PTS, and PTSK and apply Bin-Mean-Shift to binning-based methods IR, HB, and MIR. HB and IR are removed from the long-tail setting due to its instability when the class sample size is very small.

More details on baseline algorithms, datasets, pretrained models, hyperparameters, and implementation details can be found in Appendix C.

### Effectiveness

Datasets with the balanced class distribution.The results on ImageNet of 504 models from timm[41] are depicted in Figure 3, where our method (red color markers) consistently appears at the bottom, achieving the lowest calibration error across all four evaluation metrics in general. This indicates that our method consistently outperforms other approaches in eliminating proximity bias and improving confidence calibration. Additionally, we summarize the results of four popular models, namely BeiT[2], MLP Mixer[38], ResNet50[12] and ViT[8] in Table 4 of Appendix E. Additionally, Table 2a presents the calibration results for the text classification task on Yahoo

\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{2}{c}{ECE \(\downarrow\)} & \multicolumn{2}{c}{ACE \(\downarrow\)} & \multicolumn{2}{c}{MCE \(\downarrow\)} & PIECE \(\downarrow\) \\ \cline{2-7} Method & base & +ours & base & +ours & base & +ours & base & +ours \\ \hline Conf & 4.85 & **0.78*** & 4.86 & **0.76*** & 0.55 & **0.18*** & 4.91 & **1.51*** \\ TS & 2.03 & **0.70*** & 2.02 & **0.78*** & 0.30 & **0.14*** & 2.34 & **1.43*** \\ ETS & 1.12 & **0.66*** & 1.15 & **0.77*** & 0.18 & **0.13*** & 1.79 & **1.38*** \\ PTS & 4.86 & **0.71*** & 4.90 & **0.86*** & 2.96 & **0.12** & 7.04 & **1.44*** \\ PTSK & 2.97 & **0.65*** & 3.01 & **0.82*** & 0.56 & **0.11*** & 4.66 & **1.41*** \\ MIR & 1.05 & **0.98** & 1.09 & **1.06** & **0.18** & 0.19 & **1.63** & 1.64 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Performance on iNaturalist 2021 long-tail dataset with ResNet50 pretrained on ImageNet. * denotes significant improvement (p-value < 0.1). ‘Base’ refers to existing calibration methods, ‘Ours’ to our method applied to calibration. Note that ‘Conf+Ours’ shows the result of our method applied directly to model confidence. Calibration error is given by \(\times 10^{-2}\).

Figure 3: Calibration errors on ImageNet across 504 timm models. Each point represents the calibration result of applying a calibration method to the model confidence. Marker colors indicate different calibration algorithms used. Among all calibration algorithms, our method consistently appears at the bottom of the plot. See Appendix E Figure 11 for high resolution figures.
Answers Topics where our method consistently improves the calibration of existing methods and model confidence. See Table 5 in Appendix E for results of the text understanding task MultiNLI.

Datasets with the long-tail class distribution.Table 1 shows the results on the image dataset inNaturalist 2021. Our method ('ours') consistently improves upon existing algorithms ('base') regarding reducing confidence calibration errors (ECE, ACE, and MCE) and mitigating proximity bias (PIECE). Note that even when used independently without combining with existing algorithms ('Conf+ours'), our method achieves the best performance across all metrics. This result suggests that our algorithm can make the model more calibrated in the long-tail setting by effectively mitigating the bias towards low proximity samples (i.e. tail classes), highlighting its practicality in real-world scenarios where data is often imbalanced and long-tailed. Besides, ImageNet-TL results in Table 6 of Appendix E.3 show similar improvement.

Datasets with distribution shift.Table 2b shows our method's calibration performance when trained on an in-distribution validation set (MultiNLI Match) and applied to a cross-domain test set (MultiNLI Mismatch). The results show that our method can improve upon most existing methods on ECE, ACE and MCE, and gain consistent improvement on PIECE, indicating its effectiveness in mitigating proximity bias. Besides, compared to Bin-Mean-Shift, Density-Ratio exhibits more stable performance on enhancing the existing baselines. More analysis on their comparison can be found in Appendix F.1. Moreover, empirical results on ImageNet-C on Figure 12 of Appendix E also demonstrate consistent improvements of our method over baselines.

## 7 Conclusions and Discussion

In this paper, we focus on the problem of proximity bias in model calibration, a phenomenon wherein deep models tend to be more overconfident on data of low proximity (i.e. lying in the sparse region of data distribution) and thus suffer from miscalibration. We study this phenomenon on \(504\) public models across a wide variety of model architectures and sizes on ImageNet and find that the bias persists even after applying the existing calibration methods, which drives us to propose ProCal for tackling proximity bias. To further evaluate the miscalibration due to proximity bias, we propose a proximity-informed expected calibration error (PIECE) with theoretical analysis. Extensive empirical studies on balanced, long-tail, and distribution-shifted datasets under four metrics support our findings and showcase the effectiveness of our method.

Potential Impact, Limitations and Future WorkWe uncover the proximity bias phenomenon and show its prevalence through large-scale analysis, highlighting its negative impact on the safe deployment of deep models, e.g. unfair decisions on minority populations and false diagnoses for underrepresented patients. We also provide ProCal as a starting point to mitigate the proximity bias, which we believe has the potential to inspire more subsequent works, serve as a useful guidance in the literature and ultimately lead to _improved and fairer decision-making in real-world applications_, especially for underrepresented populations and safety-critical scenarios. However, our study also has several limitations. First, our ProCal maintains a held-out validation set during inference for computing proximity. While we have shown that the cost can be marginal for large models (see Inference Efficiency in Appendix E.1), it may be challenging if applied to small devices where memory is limited. Future research can investigate the underlying mechanisms of proximity bias and explore various options to replace the existing approach of local density estimation. Additionally, we only focus on the closed-set multi-class classification problem; future work can generalize this to multi-label, open-set or generative settings.

\begin{table}

\end{table}
Table 2: We use RoBERTa models [26] fine-tuned on Yahoo and MultiNLI Match, respectively, as their models. ‘Base’ refers to existing calibration methods and ‘Ours’ refers to our method applied to existing calibration methods. Calibration error is given by \(\times 10^{-2}\).