# Comment on a no-go theorem for $\psi$-ontic models

Laurens Walleghem  Department of Mathematics, University of York, Heslington,
York, YO10 5DD, United Kingdom  Shashaank Khanna  Department of Mathematics,
University of York, Heslington, York, YO10 5DD, United Kingdom  Rutvij Bhavsar
<EMAIL_ADDRESS>Department of Mathematics, University of York,
Heslington, York, YO10 5DD, United Kingdom<EMAIL_ADDRESS>Department of Mathematics, University of York, Heslington, York, YO10 5DD,
United Kingdom<EMAIL_ADDRESS>Department of Mathematics, University of
York, Heslington, York, YO10 5DD, United Kingdom

###### Abstract

In a recent paper [Carcassi, Oldofredi and Aidala, Found Phys 54, 14 (2024)]
it is claimed that the whole Harrigan–Spekkens framework of ontological models
is inconsistent with quantum theory. They show this by showing that all pure
quantum states in $\psi$-ontic models must be orthogonal. In this note, we
identify some crucial mistakes in their argument to the extent that the main
claim is incorrect.

##  1 Introduction

In this short note, we consider the result of [1], where grand claims on the
nature of the quantum state are made. It is claimed that the Harrigan-Spekkens
[2] categorisation of ontic vs epistemic states itself is problematic. The
authors of [1] claim that for $\psi$-ontic models all pure states in quantum
mechanics must be orthogonal, in stark violation of quantum mechanics. Hence,
they conclude that their result shows that even $\psi$-ontic models cannot
reproduce Quantum Mechanics. An immediate contradiction to their result is the
existence of the de Broglie- Bohm theory which is $\psi$-ontic and reproduces
Quantum Mechanics. Together with their result and the PBR theorem [3], which
suggests that the wavefunction in quantum mechanics is ontic, the authors of
[1] conclude that it is the Harrigans-Spekkens framework itself that is wrong.
As we will argue, three points weaken their argument to the point that their
conclusion does not hold together. Namely,

  * •

it is assumed that the entropy of the quantum state equals the entropy of the
underlying ontological distribution, which need not necessarily be the case,

  * •

even if we do assume that the entropy of the quantum state should be the same
as the entropy of the underlying ontological distribution (which as mentioned
is meaningless), their result would still restrict to $\psi$-complete models
and does not capture _all_ $\psi$-ontic models,

  * •

the claim (and its proof) of [1] that ‘no ontological model can reproduce
quantum theory’ is wrong. The research paradigm of generalised
noncontextuality [4, 5, 6, 7, 8, 9, 10, 11, 12] and related establishes that
rather ‘any ontological model for quantum theory needs fine-tuning’.

##  2 Recap of the argument of [1]

We shortly recap the argument of [1] here and state the assumptions upon which
we comment in the next section.

Two non-orthogonal quantum states $|\psi\rangle,|\phi\rangle$ are considered,
and the entropy of their mixture
$\rho=\frac{1}{2}|\psi\rangle\langle\psi|+\frac{1}{2}|\phi\rangle\langle\phi|$
is calculated in two ways, namely (i) using the von Neumann entropy for
quantum states and (ii) using the ontological $\psi$-ontic representation of
those quantum states and assuming convexity-preserving. Using the fact that
$\rho$ can be diagonalised as
$\rho=\frac{1+|\langle\psi\mid\phi\rangle|}{2}|+\rangle\langle+|+\frac{1-|\langle\psi\mid\phi\rangle|}{2}|-\rangle\langle-|$
for some basis $|\pm\rangle$ the von Neumann entropy is given by

$\begin{split}H(\rho)&=-\operatorname{Tr}\big{[}\rho\log\rho\big{]}=H\left(\frac{1+|\langle\psi\mid\phi\rangle|}{2},\frac{1-|\langle\psi\mid\phi\rangle|}{2}\right)\\\
&=-\frac{1+\sqrt{p}}{2}\log\frac{1+\sqrt{p}}{2}-\frac{1-\sqrt{p}}{2}\log\frac{1-\sqrt{p}}{2},\end{split}$
(1)

with $p=|\langle\psi|\phi\rangle|^{2}$. On the other hand, consider an
ontological model for quantum theory, that is $\psi$-ontic such that
$\mu(\lambda|\psi)$ and $\mu(\lambda|\phi)$ have nonoverlapping support
$U_{\psi}$ and $U_{\phi}$. For ease of notation, we define
$\mu_{P_{\sigma}}(\lambda):=\mu(\lambda|P_{\sigma})$ for a density operator
$\sigma$. Furthermore, the following is assumed, as stated in eq. (10) in [1].

######  Assumption 1.

The ontological model preserves convex combinations of preparations, i.e. if
density operator $\rho$ with preparation $P_{\rho}$ is obtained by sampling
between preparing $P_{\psi}$ and $P_{\phi}$ with equal probability, then their
ontological representations satisfy

$\mu_{P_{\rho}}=\frac{1}{2}\mu_{P_{\psi}}+\frac{1}{2}\mu_{P_{\phi}}.$ (2)

For an ontic distribution $\mu_{P_{\psi}}(\lambda)$, the authors then define
the following:

$\displaystyle
H_{\mathrm{\psi}}(\mu_{p_{\psi}}):=\int_{\Lambda}\mu_{P_{\psi}}(\lambda)\log(\mu_{P_{\psi}}(\lambda))$
(3)

where $\log$ are is assumed to be base two. Calculating the entropy
$H_{\mathrm{ontic}}$ of the ontological distribution $\mu_{P_{\rho}}$, one
finds

$\displaystyle H_{\mathrm{ontic}}(\mu_{P_{\rho}})$
$\displaystyle=-\int_{\Lambda}\mu_{P_{\rho}}\log\mu_{P_{\rho}}d\lambda$ (4)
$\displaystyle=-\int_{U_{\phi}}\mu_{P_{\rho}}\log\mu_{P_{\rho}}d\lambda-\int_{U_{\psi}}\mu_{P_{\rho}}\log\mu_{P_{\rho}}d\lambda$
$\displaystyle=-\int_{U_{\psi}}\frac{1}{2}\mu_{P_{\psi}}\log\frac{1}{2}\mu_{P_{\psi}}d\lambda-\int_{U_{\phi}}\frac{1}{2}\mu_{P_{\phi}}\log\frac{1}{2}\mu_{P_{\phi}}d\lambda$
$\displaystyle=-\frac{1}{2}\int_{U_{\psi}}\mu_{P_{\psi}}\log\frac{1}{2}d\lambda-\frac{1}{2}\int_{U_{\psi}}\mu_{P_{\psi}}\log\mu_{P_{\psi}}d\lambda$
$\displaystyle-\frac{1}{2}\int_{U_{\phi}}\mu_{P_{\phi}}\log\frac{1}{2}d\lambda-\frac{1}{2}\int_{U_{\phi}}\mu_{P_{\phi}}\log\mu_{P_{\phi}}d\lambda$
$\displaystyle=-\frac{1}{2}\log\frac{1}{2}-\frac{1}{2}\log\frac{1}{2}+\frac{1}{2}H\left(\mu_{P_{\psi}}\right)+\frac{1}{2}H_{\mathrm{ontic}}\left(\mu_{P_{\phi}}\right)$
$\displaystyle=1+\frac{1}{2}H_{\mathrm{ontic}}\left(\mu_{P_{\psi}}\right)+\frac{1}{2}H_{\mathrm{ontic}}\left(\mu_{P_{\phi}}\right).$

In the first and second line, we used that $\mu_{P_{\rho}}$ can be written as
a convex combination of $\mu_{P_{\psi}}$ and $\mu_{P_{\phi}}$, which have
nonoverlapping ontological support $U_{\psi}$ and $U_{\phi}$. Next, the
authors assume that as pure states have zero entropy that this holds on the
ontological level as well, i.e.

$H(\mu_{P_{\psi}})=0=H(\mu_{P_{\phi}}).$ (5)

######  Assumption 2.

For any pure quantum state $\psi$ we have
$H_{\mathrm{ontic}}(\mu_{P_{\psi}})=0$.

As we argue below in Section 3.1 this implies $\psi$-completeness. We find
using eq. 4 that

$H_{\mathrm{ontic}}(\mu_{P_{\rho}})=1,$ (6)

which does not equal the von Neumann entropy for the quantum state $\rho$,
given in eq. 1 as $|\psi\rangle$ and $|\phi\rangle$ are nonorthogonal, i.e.
$p=|\langle\psi|\phi\rangle|^{2}\neq 0$. The authors argue that this is
contradicting, as they assume the von Neumann entropy of a quantum state
should equal the ontological entropy, namely they assume the following.

######  Assumption 3.

The entropy of a quantum state equals the entropy of its ontological
representation.

We can summarise these findings in the following theorem, the main result of
[1] but now stating the (hidden) assumptions.

######  Theorem 1.

A $\psi$-complete model for quantum theory where for quantum states the
ontological Shannon entropy equals the von Neumann entropy of quantum theory
cannot be ontological for convexity-preserving preparations.

##  3 Discussing the precise assumptions

###  3.1 The result of [1] assumes $\psi$-completeness instead of just
$\psi$-onticness

It is straightforward to argue that assumption 2 directly entails
$\psi$-completeness.

######  Lemma 1.

Assumption 1 and Assumption 2 implies $\psi$ completeness.

###### Proof.

Let $\psi$ be any pure state, and let $p_{\psi}$ denote its ontological
distribution. Assumption 3 implies the following:

$-\int_{\Lambda}d\lambda p(\lambda)\log_{2}(p(\lambda))=0$ (7)

The consequence of Equation (7) is that for $p_{\psi}\in[0,1]$, the expression
$-p_{\psi}\log(p_{\psi})>0$ implies

$\displaystyle\forall\lambda\in\Lambda:\left(\quad
p_{\psi}(\lambda)=0\qquad\text{or}\qquad p_{\psi}(\lambda)=1\right)$ (8)

As $p_{\psi}$ is a probability distribution, this further implies that for
each preparation $P_{\psi}$ there exists an ontic state $\lambda_{P_{\psi}}$
such that we have

$\mu_{P_{\psi}}(\lambda)=\delta(\lambda-\lambda_{P_{\psi}}),$ (9)

but $\psi$-completeness needs

$\mu_{P_{\psi}}(\lambda)=\delta(\lambda-\lambda_{\psi}).$ (10)

If we additionally assume convexity-preserving preparations, i.e.

$\frac{1}{2}\mu_{P_{\psi}}(\lambda)+\frac{1}{2}\mu_{P^{\prime}_{\psi}}(\lambda)=\mu_{P^{\prime\prime}_{\psi}}(\lambda)$
(11)

then we find that all these delta functions must be equal to each other, i.e.
$\lambda_{P_{\psi}}=\lambda_{\psi}$. And if $H(\mu_{P_{\psi}})=0$, we recover
the result of [1]. ∎

Thus, any ontological model satisfying the (implicit) assumptions of [1],
namely 1 and 2, must be $\psi$-complete. This lemma thus shows that the claim
of [1] already contradicts the fact that one can produce $\psi$-incomplete
ontological models for quantum theory.

###  3.2 The entropy of a quantum state and its underlying ontological
distribution need not be equal

Assumption 3 is not a good assumption for several reasons. Firstly, there is
no inherent requirement that the von Neumann entropy of a state must be
derived via the ontological model. From a purely information-theoretic
perspective, most entropies are defined based on specific operational tasks
and do not necessarily represent physical properties of the system. For
instance, the operational meaning of von Neumann entropy is derived from
scenarios like state discrimination [13]. Consider Alice generating an
ensemble of pure states ${\ket{\psi_{x}}}{x}$ with a probability distribution
$p_{X}$ and sending it to Bob. Bob receives the state
$\sigma=\sum_{x}p_{X}(x)\ket{\psi_{x}}\bra{\psi_{x}}$, and his task is to
determine which state Alice has prepared. The von Neumann entropy $H(\sigma)$
quantifies Bob’s uncertainty about the pure state prepared by Alice. However,
this quantity is not inherently tied to the underlying ”elements of reality”
of the pure states themselves. As the task at hand is to distinguish pure
states, it is reasonable to assume that if the ensemble consists of a single
pure state $\ket{\psi}$, then the uncertainty is zero, implying
$H(\ket{\psi}\bra{\psi})=0$. Within quantum theory, too, different entropic
quantities are usually defined depending on the task at hand. The entropic
quantity $H_{\mathrm{ontic}}$ would be the right operational quantity to
consider if Alice prepares (if possible) an ensemble
$\\{p(\lambda),\lambda\\}$ and Bob’s task was to determine which $\lambda$ is
sent.

Moreover, it is not the case that entropies from two different theoretical
descriptions of the same information-theoretic task must lead to the same
results. For instance, consider the state discrimination task for the ensemble
$\\{p_{X}(x),\ket{\psi_{x}}\\}$ as discussed earlier. This task can be
approached via two different theoretical frameworks: one in classical theory
and the other in quantum theory. Using the data-processing inequality, it
becomes apparent that the Shannon entropy $S(X)$ (entropy for the classical
case) cannot be smaller than the von Neumann entropy of the ensemble $H(\rho)$
(see Exercise 11.9.3 from [13]). In fact they differ from each other in most
cases. Similarly, there is no inherent reason why the entropy
$H_{\mathrm{ontic}}(.)$ of the ensemble should reproduce the von Neumann
entropy $H(.)$, as they arise from different theoretical descriptions
altogether. Thus, the quantum entropy $H(.)$ can differ from entropy
$H_{\mathrm{ontic}}(.)$ even if they were to describe the same operational
task.

Finally, regarding the Harrigan-Spekkens model, it is crucial to acknowledge
that their description of quantum states is fundamental only to pure quantum
states, i.e. mixed states need not have a unique ontological representation.
The objective of the framework is to determine whether a quantum state is a
state of complete knowledge. However, a mixed quantum state fails to meet this
criterion, even within quantum theory. Mixed states are effective states
arising themselves from a lack of classical knowledge. Hence, expecting the
framework to establish a one-to-one correspondence with mixed states is an
additional assumption proposed by the authors, which should not inherently
hold. Nonetheless, this does not imply that mixed states are indescribable
within the theory. For every mixed state, there exists a purifying system and
a corresponding pure state. This pure state should be regarded as representing
complete knowledge and can be in one-to-one correspondence with some
probability $\mu$ on the ontic space. Demanding that every mixed state should
also possess a preparation-independent distribution, as discussed further in
section 3.3, is an additional assumption used on top of the Harrigan-Spekkens
framework. As a result, even if all the aforementioned arguments are
discounted, this is the sole reason why a one-to-one correspondence with mixed
states is an added assumption and not an inherent one of the framework. Hence,
it cannot serve as the basis for rejecting the framework.

###  3.3 No fine-tuning no-go theorems rather than the nonexistence of
ontological models for quantum theory

Furthermore, the statement ‘No ontological model can reproduce quantum theory’
([1], p.14) is wrong, and there are examples of such models that reproduce
quantum theory, such as the Beltrametti-Bugajski model [14, 4]. However, a
more correct statement that appears in quantum foundations is the following:

> No ontological model that satisfies no-fine tuning can reproduce quantum
> theory.

Here no fine-tuning means that operational identities are reflected in the
ontological model [15, 16, 4, 17], such as the same density operator having
the same ontological representation, no matter how it is produced, also
referred to as preparation noncontextuality [4]. Proofs of this result fall
amongst others in the paradigm of generalised noncontextuality [4, 5, 6, 7, 8,
11, 9, 18, 19, 12, 20, 21].

In fact, as we have established in Section 3.1, the assumptions of [1] imply
$\psi$-completeness and thus preparation noncontextuality for pure states,
i.e. all pure states have the same ontological delta distribution.
Furthermore, 3 implies that all ontological distributions of a mixed state
preparation have the same entropy, they all have the same ontological shape,
which gets close to assuming all preparations of the same mixed state to have
the same ontological distribution, which amounts to assuming preparation
noncontextuality for which many no-go theorems have already been established
in the paradigm of generalised noncontextuality [4, 5, 6, 7, 8, 11, 9] without
assuming any entropy relations.

## References

  * [1] Gabriele Carcassi, Andrea Oldofredi and Christine A Aidala  “On the reality of the quantum state once again: A no-go theorem for $\psi$-ontic models”  In _arXiv preprint arXiv:2201.11842_ , 2022
  * [2] Nicholas Harrigan and Robert W Spekkens  “Einstein, incompleteness, and the epistemic view of quantum states”  In _Foundations of Physics_ 40 Springer, 2010, pp. 125–157
  * [3] Matthew F Pusey, Jonathan Barrett and Terry Rudolph  “On the reality of the quantum state”  In _Nature Physics_ 8.6 Nature Publishing Group UK London, 2012, pp. 475–478
  * [4] Robert W Spekkens  “Contextuality for preparations, transformations, and unsharp measurements”  In _Physical Review A_ 71.5 APS, 2005, pp. 052108
  * [5] Robert W Spekkens  “The status of determinism in proofs of the impossibility of a noncontextual model of quantum theory”  In _Foundations of Physics_ 44 Springer, 2014, pp. 1125–1155
  * [6] David Schmid, Robert W Spekkens and Elie Wolfe  “All the noncontextuality inequalities for arbitrary prepare-and-measure experiments with respect to any fixed set of operational equivalences”  In _Physical Review A_ 97.6 APS, 2018, pp. 062103
  * [7] David Schmid, John H Selby, Matthew F Pusey and Robert W Spekkens  “A structure theorem for generalized-noncontextual ontological models”  In _arXiv preprint arXiv:2005.07161_ , 2020
  * [8] David Schmid et al.  “Characterization of noncontextuality in the framework of generalized probabilistic theories”  In _PRX Quantum_ 2.1 APS, 2021, pp. 010331
  * [9] Michael D Mazurek et al.  “An experimental test of noncontextuality without unphysical idealizations”  In _Nature communications_ 7.1 Nature Publishing Group UK London, 2016, pp. ncomms11780
  * [10] Ravi Kunjwal  “Beyond the Cabello-Severini-Winter framework: Making sense of contextuality without sharpness of measurements”  In _Quantum_ 3 Verein zur Förderung des Open Access Publizierens in den Quantenwissenschaften, 2019, pp. 184
  * [11] Anubhav Chaturvedi, Máté Farkas and Victoria J Wright  “Characterising and bounding the set of quantum behaviours in contextuality scenarios”  In _Quantum_ 5 Verein zur Förderung des Open Access Publizierens in den Quantenwissenschaften, 2021, pp. 484
  * [12] David Schmid, John H Selby and Robert W Spekkens  “Addressing some common objections to generalized noncontextuality”  In _arXiv preprint arXiv:2302.07282_ , 2023
  * [13] Mark M. Wilde  “Quantum Information Theory”  Cambridge University Press, 2013  DOI: 10.1017/CBO9781139525343
  * [14] EG Beltrametti and S Bugajski  “A classical extension of quantum mechanics”  In _Journal of Physics A: Mathematical and General_ 28.12 IOP Publishing, 1995, pp. 3329
  * [15] Lorenzo Catani and Matthew Leifer  “A mathematical framework for operational fine tunings”  In _Quantum_ 7 Verein zur Förderung des Open Access Publizierens in den Quantenwissenschaften, 2023, pp. 948
  * [16] Christopher J Wood and Robert W Spekkens  “The lesson of causal discovery algorithms for quantum correlations: causal explanations of Bell-inequality violations require fine-tuning”  In _New Journal of Physics_ 17.3 IOP Publishing, 2015, pp. 033002
  * [17] Robert W Spekkens  “The ontological identity of empirical indiscernibles: Leibniz’s methodological principle and its significance in the work of Einstein”  In _arXiv preprint arXiv:1909.04628_ , 2019
  * [18] Anubhav Chaturvedi and Debashis Saha  “Quantum prescriptions are more ontologically distinct than they are operationally distinguishable”  In _Quantum_ 4 Verein zur Förderung des Open Access Publizierens in den Quantenwissenschaften, 2020, pp. 345
  * [19] John H Selby et al.  “Contextuality without incompatibility”  In _Physical Review Letters_ 130.23 APS, 2023, pp. 230201
  * [20] Matthew F Pusey  “Robust preparation noncontextuality inequalities in the simplest scenario”  In _Physical Review A_ 98.2 APS, 2018, pp. 022112
  * [21] Massy Khoshbin, Lorenzo Catani and Matthew Leifer  “Alternative robust ways of witnessing nonclassicality in the simplest scenario”  In _arXiv preprint arXiv:2311.13474_ , 2023


# Identifying and Mitigating Model Failures through Few-shot CLIP-aided
Diffusion Generation

Atoosa Chegini<EMAIL_ADDRESS>
Department of Computer Science
University of Maryland, College Park Soheil Feizi<EMAIL_ADDRESS>
Department of Computer Science
University of Maryland, College Park

###### Abstract

Deep learning models can encounter unexpected failures, especially when
dealing with challenging sub-populations. One common reason for these failures
is the occurrence of objects in backgrounds that are rarely seen during
training. To gain a better understanding of these failure modes, human-
interpretable descriptions are crucial for further analysis and improvement
which is expensive. In this study, we propose an end-to-end framework that
utilizes the capabilities of large language models (ChatGPT) and vision-
language deep models (CLIP) to generate text descriptions of failure modes
associated with spurious correlations (e.g. rarely seen backgrounds) without
human-in-the-loop intervention. These descriptions can be used to generate
synthetic data using generative models, such as diffusion models. The model
can now use this generated data to learn from its weaknesses and enhance its
performance on backgrounds that are uncommon for each class of data. Our
approach serves as a broad solution, promising progress in comprehending model
failure modes and strengthening deep learning models across a wide range of
failure scenarios (e.g. bacckgrounds, colors) automatically in a few-shot
manner. Our experiments have shown remarkable improvements in accuracy
($\sim\textbf{21\%}$) on hard sub-populations (particularly for wrong
background association) across $40$ different models, such as ResNets,
EfficientNets, DenseNets, Vision Transformer (ViT), SwAVs, MoCos, DINOs, and
CLIPs on various datasets such as ImageNet-1000, CIFAR-10, and CIFAR-100.

Figure 1: A summary of our approach: For a model based on the wrongly
predicted debug samples (failure samples), we form two sets - debug seed and
debug heldout. We use the debug seed set to address the model’s failures by
inputting them to CLIP Radford et al. (2021), along with a set of backgrounds
obtained from ChatGPT where objects are less likely to occur. We then obtain a
set of backgrounds and remove redundancies, and generate synthetic data by
inputting the prompt "A photo of {class_name} {background}" to Stable
Diffusion Rombach et al. (2021). With this synthetic data that precisely
captures the model’s failure modes, we can now debug the model’s predictions
on other test data by training a very low-cost network on top of our model,
which assigns different weights to the original data and the generated data.

##  1 Introduction

The quality of training data directly impacts the performance and robustness
of machine learning models. Despite careful curation of training data, models
can still exhibit failure modes where their performance deteriorates in
specific sub-populations of data, leading to misclassifications or inaccurate
predictions Jiang et al. (2018); Arpit et al. (2017). The failure modes of
deep networks can arise from various factors, such as noisy labels Sukhbaatar
et al. (2014); Jiang et al. (2018); Reed et al. (2015), multi-labels Zhang et
al. (2018b), and spurious correlations Zhou et al. (2020), particularly when
it comes to distinguishing between objects and their backgrounds Kattakinda &
Feizi (2021); Xiao et al. (2020). (See the figure 7 in appendix for examples
of these failures.)

Similar to how humans use image backgrounds as cues for object recognition,
studies have shown that machine learning models also rely on backgrounds when
making decisions. In some cases, models may prioritize backgrounds to the
point of overlooking important object features for classification Zhang et al.
(2007); Ribeiro et al. (2016); Sagawa et al. (2019).

Various approaches have been attempted to address failure modes caused by
spurious background associations, but many of them are inadequate in
addressing all aspects of the problem. Some approaches involve human-in-the-
loop interventions Mitchell et al. (2021); Santurkar et al. (2021), which is
labor-intensive and challenging to apply to large-scale operations. Moreover,
many of them only target a limited set of failure modes, neglecting the
comprehensive spectrum of potential failures Barbu et al. (2019); Hendrycks et
al. (2021a); Hendrycks & Dietterich (2019); Hendrycks et al. (2021b);
Kattakinda & Feizi (2021). Additionally, some of the existing work lacks clear
descriptions of model failures in a human-understandable manner, posing
challenges in terms of interpretability and validation.

Alongside research on identifying failure modes, there are several debugging
approaches that seek to utilize these failure modes to enhance the accuracy of
machine learning models. These include creating supplementary datasets with
failure samples to help the model learn robust features Xiao et al. (2020);
Singla et al. (2022), or modifying the model’s parameters to incorporate
information from identified failure modes Rame et al. (2022). These studies
lack failure mode descriptions that are easily understandable for humans,
making it challenging to interpret their validity.

##  2 Our contribution

This research leverages recent generative models, large language models, and
CLIP to introduce an automated framework addressing failure modes (spurious
correlations) in diverse task-specific deep learning models. The framework,
outlined in Figure 1, answers key questions such as identifying and rectifying
spurious associations leading to model failure, utilizing these failure modes
to enhance model performance, exploring patterns in failure modes across a
model group, and using a single set of auxiliary data to simultaneously
improve a subgroup of models.

To summarize, our approach initially identifies all model failures on a
specific subset, denoted as DebugSet, within the validation set. We then
pinpoint spurious correlations, such as background associations, for each
dataset class by querying ChatGPT with "What are the uncommon backgrounds that
a class_name can appear in?" and remove redundacies after obtaining all
uncommon backgrounds. Subsequently, a zero-shot classification using CLIP
identifies the background for each failure among all the uncommon backgrounds.
To enhance model performance, we generate $k$ artificial images with prompts
like "[class_name] in [background_name]" and incorporate this supplementary
data into the original training set. In the second phase, we demonstrate that
models with similar architectures exhibit analogous failures, allowing for
efficient troubleshooting of a model group using a single set of generated
auxiliary data. This approach proves both time and memory efficient. The
results of our experiments, detailed in section 5, underscore the
effectiveness of this straightforward method in achieving interpretability and
debugging goals.

Our paper presents several contributions to the field of machine learning
model failure analysis and debugging. These contributions include, but are not
limited to, the following:

  * •

Generalizability: Introducing a comprehensive end-to-end framework that
interprets and rectifies failures arising from specific spurious associations,
such as incorrect background, color, and size correlations, which can
contribute to any model inaccuracies.

  * •

Failure Inspection: Identification of spurious background association failure
modes of $\sim 40$ multiple models on ImageNet in an interpretable manner
(section 5.2.1), and exploring common patterns in failure modes among
individual models with same architectures (section 5.2.2) automatically and
without any human intervention.

  * •

Failure Mitigation: Improving the performance of individual models on
challenging sub-populations (5.3.1), and boosting the performance of model
subsets by employing a unified set of auxiliary data, leveraging shared
failures to enhance efficiency in both time and memory usage (section 5.3.2).

  * •

Collective Failure Mitigation: Enhancing subsets of models’ performance
through a unified set of auxiliary data owing to their shared failures which
saves time and memory. To the best of our knowledge, this work represents the
first effort to collectively address failures within a subgroup of models
simultaneously. (section 5.3.2).

  * •

Dataset creation for debugging ImageNet, Cifar10, and Cifar100: We created a
dataset that will improve models’ performance on background and color
associated failure modes on these three datasets.

##  3 Related work

###  3.1 Failure mode detection

Numerous studies have been conducted to detect machine learning model failure
modes. As previously mentioned, some involve human-in-the-loop methods, where
failure examples are reviewed to identify common patterns Mitchell et al.
(2021); Santurkar et al. (2021). Others adopt automated approaches by
introducing frameworks that effectively capture model failures Chung et al.
(2019); Singla et al. (2021); Nushi et al. (2018); Singla & Feizi (2021); Wong
et al. (2021); Wu et al. (2019); Zhang et al. (2018a); Jain et al. (2022). For
instance, Chung et al. (2019) employs a technique that slices the validation
data to isolate sections where the model performs poorly. Singla et al. (2021)
identifies visual attributes that lead to inadequate performance when present
or absent. Jain et al. (2022) identifies and represents model failures as
directions in the latent space, and Eyuboglu et al. (2022) that proposes an
evaluation framework to systematically compare (slice discovery method) SDMs
across diverse slice settings by generating captions for hard sub-populations.
Distinguishing itself from existing methodologies, our approach provides
enhanced generality by permitting the explicit selection of the spurious
correlation targeted for mitigation. For instance, although the approach
presented by Kattakinda et al. Kattakinda et al. (2022) effectively tackles
spurious correlations tied to foreground and background features by learning
disentangled representations, it encounters difficulties when confronted with
a wider spectrum of spurious correlations, e.g. color. This is due to the
inherent challenge of learning disentangled representations for many spurious
correlations in isolation from the foreground object.

###  3.2 mitigation of hard subpopulations and interpretability of models

Several methodologies leverage extracted failure modes to debug and enhance
the performance of deep learning models. Singla et al. Singla et al. (2022)
introduce a framework that identifies visually similar images to model
failures and incorporates them as new data for debugging various models.
Kattakinda et al. Kattakinda et al. (2022) focus on learning invariant
features for foreground and background by penalizing the mutual information
between the features and background/foreground labels. This approach
contributes to robust model training, particularly by addressing issues
related to spurious correlations.

In the context of data generation, Bansal and Grover Bansal & Grover (2023)
and Wiles et al. Wiles et al. (2022) use generated data to diversify training
datasets. However, it’s essential to note that their methods do not
specifically target failure modes like spurious correlations. They rely on
class names and general captions for generating auxiliary data, which may not
be tailored to address specific failure modes.

Moreover, Wiles et al. Wiles et al. (2022) propose a bug discovery approach
using off-the-shelf image generation and captioning, contributing to the
interpretability of model predictions. On the other hand, Jain et al. Jain et
al. (2022) leverage Support Vector Machines (SVMs) to distill model failures
as directions in latent space, focusing on latent representations of model
failures.

In comparison to existing methodologies that address failure modes on specific
datasets, our framework introduces two noteworthy contributions. Firstly, it
achieves enhanced model performance with significantly fewer generated
examples, (5 for each failure). Secondly, our experiments extend to collective
debugging, demonstrating the ability to improve a subset of model failures by
generating a single auxiliary artificial dataset based on only one model’s
failures. This is particularly valuable given our observation that models
within the same categories exhibit similar failures, a phenomenon also noted
in Wiles et al. (2022).

Moreover, our approach stands out for its efficiency, eliminating the
necessity for complete model retraining or fine-tuning. We exclusively focus
on retraining the linear head for classification, streamlining the failure
mode mitigation process.

##  4 Main method

###  4.1 Failure-mode detection

A common reason for accuracy drops during inference is the model’s learned
spurious correlations from training. For example, Associating objects with
backgrounds, a spurious correlation, can hinder the model’s ability to learn
objects themselves. This challenge arises when the model encounters objects in
unfamiliar backgrounds during testing, notably in computer vision tasks where
backgrounds define object context. To tackle this, introducing the model to a
range of scenarios that address the particular failure mode (such as color or
background associations) we aim to mitigate, can improve its ability to
identify objects in different contexts, and avoid correlating the objects and
their changable features (e.g. color) or contexts (e.g. background).

In this work we explain how we use this framework for wrong background
associations, however it can easily be applied to all other spurious
correlations that models may mistakenly learn.

To address and rectify failures attributed to backgrounds, we utilize the
feature extractor for each model on the datasets, generating a feature vector
for each data point. The subsequent linear head atop this feature extractor is
responsible for executing the classification task. Instances where the model
makes incorrect predictions form a set termed the "debug set". This debug set
serves as a tool for identifying and resolving failure modes, as it comprises
all examples where the model fails. While these failures may stem from various
factors, our experiments underscore the significance of mitigating incorrect
background associations, as it significantly improves the performance of all
models.

Class name | Uncommon backgrounds
---|---
Sea lion | Desert, Rain forests, Urban Areas, Polar Ice Caps,
| Caves, Grasslands, Volcanic Areas
Siberian husky | Jungle Canopies, In sky, Caves, Underwater,
| Indoor Spaces, Marshlands, Tropical Rainforests
croquet ball | Mountain Peaks, Busy Streets, Frozen Lakes,
| Underneath Building Foundations, Subway Tunnels, in a restaurant
lipstick, lip rouge | Gyms and Fitness Centers, Swimming Pools,
| Medical Facilities, Construction Sites, Sports Events, Military Training
Table 1: Examples of suggested uncommon backgrounds for a class of data by
ChatGPT

###  4.2 Failure-mode textualization

Vision-language models are popular as they can provide more comprehensive
understanding of complex phenomena by combining information from different
modalities like text, images, and audio, enabling them to interpret data in a
more human-readable form Lu et al. (2019); Chen et al. (2018); Mithun et al.
(2020).

Understanding failure modes is critical for validating proposed debugging
methods. By identifying the causes of failure, we can improve our models and
refine our data collection methods. For each class_name in our dataset, we
first prompt ChatGPT "What are the uncommon backgrounds that a class_name can
appear in?" and filter out the redundant suggested backgrounds. Some examples
can be seen in Table 1. Then, we use CLIP Radford et al. (2021) to interpret
failure modes by splitting the failures from the debug set into two sets
called debug seed and debug heldout. We then perform zero-shot classification
by passing debug seed along the set of class_wise uncommon backgrounds
proposed by ChatGPT to a CLIP model, so for each data point, CLIP will opt for
the background that is more likely to be the actual background of the object
shown in the image. For each data class, we then pinpoint the k most
frequently selected backgrounds by CLIP, which the model failed to classify.
This will provide valuable insights into the ways how a model may fail when
confronted with a particular selected spurious association.

Figure 2: Examples of generated data by Stable Diffusion

###  4.3 Generating synthetic data

By leveraging the detected backgrounds of failures by Clip, we can both
interpret those failures, and use them to debug models. For instance, in the
case of the Imagenet class "tench," errors predominantly occur when the fish
is held by a person’s hand, a scenario rarely encountered during training. To
address this, a generative model like Stable Diffusion Ho et al. (2021) can be
used to create images that familiarize the model with diverse object contexts.
For the "tench" class, we generate data by inputting the prompt "tench in a
hand" to the Stable Diffusion. Examples of such generated data are presented
in figure 2

###  4.4 Retraining the linear head

After collecting the additional synthetic data for the failed scenarios, which
we call debug_train, we can use it along with our original trainset to debug
our models. To achieve this, we only need to train a linear head on top of the
model’s feature extractor for the classification purpose and not the whole
model. It is important to note that we assign different weights to the
datapoints from the original_train set and the Debug_train set in our linear
head training loss 1. This parameter is called lambda, and in our experiments,
we report its effect on the overall performance of the model. By incorporating
the additional debug_train data and carefully tuning the lambda parameter, we
can potentially improve the performance of our models.

$L_{cl}=L_{cl}(Original\\_train)+\lambda*L_{cl}(Debug\\_train)$ (1)

Figure 3: Some examples of failure modes of ResNets and DINOs

##  5 Experiments

###  5.1 Setting

We load our datasets and use their training data to train the linear head on
top of the models’ extracted features. For ImageNet, we use $30$ data points
per class and the overall $30000$ training images. We choose $30000$ images
out of $50000$ of Imagenet’s validation set as our debug_set, and the
remaining $20000$ samples will be considered for the testing process. Each
image’s resolution is $224*224$, and the task performed here is the
classification task on Imagenet classes. Other settings include hyper-
parameters that we used can be seen in table 4 in the appendix.

We tested our method on $40$ different models including ResNets He et al.
(2016), EfficientNets Tan & Le (2019), DenseNets Huang et al. (2017), Vision
Transformer (ViT) Dosovitskiy et al. (2021), SwAVs Caron et al. (2020), MoCos
He et al. (2019), DINOs Caron et al. (2021), and Clips Radford et al. (2021).
The exact list of used models can be seen in table 5 in the appendix. For the
sake of space, we only show experiments on DINO and ResNet models, and
experiments for other models can be found in appendix B.

We split the detected failures of debug set in half. The first half, debug
seed will be used for debugging. For obtaining uncommon backgrounds, we use
ChatGPT $3.5$. The CLIP model we use for choosing backgrounds for data points
is ViT-B32 CLIP. For generating synthetic data, we use Stable Diffusion $V1-5$
imported from the diffusers package.

###  5.2 Failure inspection

The initial stage of our framework involves analyzing how various models fail
to classify objects on different datasets. To accomplish this, we use the CLIP
model to identify backgrounds on which models struggle to classify objects.
This results in captions that describe failures related to rare backgrounds.
In the following stage, we examine these identified failures and explore how
the generated captions help us to recover from them. We investigate results
for both individual and Collective failure inspection.

Figure 4: Comparing failures of all models (Intersection/Union).We observe
that models belonging to the same categories tend to exhibit more comparable
failures.

####  5.2.1 Individual Failure Inspection

In figure 3, we show some instances where ResNet and DINO models have failed
and see that these failures are due to wrong background association. In this
figure, the six images on the left (a-f) are examples of Resnets’ failures,
and the five images on the right (g-k) are failure modes’ of DINO models. For
example, image c shows "a wallet in jungle", which can be regarded as an
uncommon background for this object. As a result, the ResNet34 model is unable
to classify it accurately and instead predicts a "mushroom" which is more
likely to be found "in garden", especially under a plant, despite having no
resemblance to the actual object in the image. Similarly, image h illustrates
"a bell or wind chime in sky", which is uncommon since "bell" is more likely
to be seen with other backgrounds such as "a door, a building or a wall".
Therefore, the DINO_reset50 model mispredicts as "traffic light" because
"traffic light" is more common to be seen "in a sky background".

In general, our approach is capable of addressing failure scenarios
originating from uncommon backgrounds of objects. Analyzing the relevant
backgrounds allows us to readily understand the cause of such failure
instances.

####  5.2.2 Collective Failure Inspection

Within this section, we will conduct a comparison of the failure modes for all
models to assess their alignment. Our aim is to determine the extent to which
failures are consistent across models. While numerous studies have focused on
analyzing similarities in the learning process and representations of
different models, such as Raghu et al. (2021) that demonstrated the similarity
between convolutional neural networks (convnets) and other convnets, as well
as the similarity between vision transformers (ViTs) and other ViTs, our focus
is on investigating whether models also fail in similar ways. This will enable
us to gain a deeper understanding of how to address the issue of failures in a
more generalized manner without taking the specific model into consideration.

The failures of different models in various categories are compared in figure
4 by computing the intersection over union of the failures. It can be observed
that models within the same category fail in more similar samples. Typically,
the failures between models from the same category are over $80\%$ similar
(e.g. CLIPs and EfficientNets). Among all $40$ models, the intersection of
failure modes is above $40\%$, indicating that models tend to fail in very
similar ways, even with different architectures. This will raise the question
of "how to utilize this similarity in failures to enhance a group of models’
performance?" which we will explore more in section 5.3.2.

It is pertinent to mention that Wiles et al. (2022) has also recognized
patterns of consistencies in failures among models within the same category.
however, we take a step further and delve into leveraging these consistencies
to systematically mitigate shared failure modes.

###  5.3 Failure Mitigation

####  5.3.1 Individual Failure Mitigation

The outcomes from employing our framework are presented in Table 2. we only
included the results for ResNets and DINOs, but we have results for other
models (EfficientNets, DenseNets, ViTs, SWaVs, MoCo, and CLIPs) in the
appendix. In Table 2, we constructed debug_seed and debug_heldout sets to
yield zero accuracy for the model, as they are composed of model failures.
Post debugging and utilizing debug_seed, we observe substantial improvements
in debug_heldout data that we didn’t use for debugging, ensuring an unbiased
evaluation. This improvement underscores that many failure modes stem from
incorrect associations models make between objects and backgrounds. Some might
argue that this gain results from the additional data. Thus, we present
results for a baseline we term Random_debugging. This baseline similarly uses
debug_seed and debug_heldout, then generates synthetic data using only class
names (prompts are structured as "A photo of [class_name]"). This comparison
illustrates that the improvement of our method arises from considering
background information. In the outcomes of Random_debugging, the improvement
over debug_seed and debug_heldout is roughly equivalent since no information
from the background association of either set was utilized. Random_debugging
solely employs class names to generate data. However, this improvement is not
on par with the gain achieved by incorporating background information when
generating new data.

It’s worth noting that despite incorporating Stable Diffusion-generated data,
which could be seen as out-of-distribution samples, a positive impact on model
performance remains. This is primarily attributed to the parameter $\lambda$
that controls the contribution of the generated images in our training
process. The influence of this parameter is depicted in figure 5.

Another crucial hyper-parameter is the number (#) of generated synthetic data
per class. The effect of this hyper-parameter, denoted as k, is illustrated in
figure 5.

The improvement observed in debug_heldout data surpasses $\sim 21\%$ for all
models, highlighting the tendency of models to fail in associating backgrounds
with objects and utilizing this association to predict objects, neglecting
object-specific features. This can be contrasted with the accuracy gain
achieved by the Random_debugging baseline, which is significantly smaller
compared to our method.

Models | Accuracies
---|---
Model |  | Accuracy | Accuracy of | Accuracy of
model | before | Individual Debugging | Random debugging
category | name | debugging | (ours) |
|  | Test | Test | seed | heldout | Test | seed | heldout
ResNet | resnet18 | $0.6236$ | $0.6413$ | $0.2636$ | 0.2128 | $0.6242$ | $0.1134$ | $0.1129$
resnet26 | $0.6593$ | $0.6671$ | $0.2856$ | 0.228 | $0.6604$ | $0.09539$ | $0.0904$
resnet34 | $0.7017$ | $0.7165$ | $0.3061$ | 0.2531 | $0.7099$ | $0.09856$ | $0.08615$
resnet50 | $0.7631$ | $0.7671$ | $0.3444$ | 0.2717 | $0.7657$ | $0.1102$ | $0.1072$
resnet101 | $0.796$ | $0.8024$ | $0.3574$ | 0.2656 | $0.7974$ | $0.1132$ | $0.1181$
resnet152 | $0.816$ | $0.8238$ | $0.3609$ | 0.2817 | $0.8183$ | $0.08207$ | $0.08804$
DINO | ViTs8 | $0.6977$ | $0.7008$ | $0.3117$ | 0.2494 | $0.6979$ | $0.1134$ | $0.1129$
ViTs16 | $0.649$ | $0.6558$ | $0.2922$ | 0.2379 | $0.6502$ | $0.09539$ | $0.0904$
ViTb8 | $0.7101$ | $0.7136$ | $0.3325$ | 0.2518 | $0.7104$ | $0.09856$ | $0.08615$
ViTb16 | $0.6832$ | $0.6851$ | $0.3067$ | 0.2477 | $0.681$ | $0.1102$ | $0.1072$
Table 2: Accuracy of our method compared to the Random_debugging. Note that
the accuracy of models on debug_seed and debug_heldout was zero before
debugging. After applying our debugging method, we gain above $\sim 21\%$
improvements in accuracies for all models, showcasing that more than $\sim
21\%$ of model errors in the heldout set come from wrong background
associations. Figure 5: a) As the value of $lambda$ increases, the accuracy on
the heldout set improves while the accuracy on the test set decreases.
However, there is a specific point (0.5) on the plot where the accuracy of the
models on both the test and heldout sets stabilizes. b) Increasing the number
of chosen backgrounds per class enhances the accuracy on the heldout set.
Considering the high cost of generating additional data, we opt for k = 3,
where the plot exhibits a significant slope.

####  5.3.2 Collective Failure Mitigation

As discussed in section 5.2.2, since models from the same categories have very
similar failures, we have considered the possibility of using a single set of
generated data, called collective_debug_train, to debug all models within the
same categories. To achieve this, we have devised two different settings: 1)
we get the failure modes of all models in the same category (e.g. ResNets),
and we select k samples from all their failures. Therefore, background
failures that occurred more have a higher probability of being chosen for the
collective_debug_train. We then use this data to debug individual models in
this category. 2) We get the failure modes of only one of the models in a
category and then use this to debug all models. This approach is more
efficient in terms of time and memory, as it requires running only one model
per category. The results for this experiment are shown in table 3. Based on
our observations in section 5.2.2, having the same debug_train data for
debugging (collective_debug_train), improves the accuracies among all models
in the same category. This approach offers greater efficiency as it eliminates
the need to generate debug_train data for each individual model. Consequently,
it saves us both time and memory that would otherwise be required for storing
such data.

In an overview, the collective debugging approach showcases the capability to
resolve above 75% of failures corrected by individual debugging (Debugging
each model based on its failures) 6.

Models | Accuracies
---|---
|  | Accuracy | Accuracy of | Accuracy of
Model | model | before | Collective Debugging-type 1 | Collective Debugging-type 2
category | name | debugging | (ours) | (ours)
|  | Test | Test | seed | heldout | Test | seed | heldout
ResNet | resnet18 | $0.6236$ | $0.6364$ | $0.2291$ | 0.2078 | $0.6413$ | $0.2636$ | 0.2128
resnet26 | $0.6593$ | $0.6655$ | $0.2396$ | 0.2192 | $0.6669$ | $0.2185$ | 0.1853
resnet34 | $0.7017$ | $0.7149$ | $0.2312$ | 0.2188 | $0.7135$ | $0.2391$ | 0.2178
resnet50 | $0.7631$ | $0.7644$ | $0.2419$ | 0.2217 | $0.7641$ | $0.2325$ | 0.2105
resnet101 | $0.796$ | $0.8001$ | $0.2474$ | 0.2226 | $0.7999$ | $0.2244$ | 0.2045
resnet152 | $0.816$ | $0.8182$ | $0.2509$ | 0.2317 | $0.8188$ | $0.2253$ | 0.2081
DINO | ViTs8 | $0.6977$ | $0.70001$ | $0.2729$ | 0.2488 | $0.70008$ | $0.3117$ | 0.2494
ViTs16 | $0.649$ | $0.6522$ | $0.2673$ | 0.2394 | $0.6504$ | $0.2563$ | 0.2267
ViTb8 | $0.7101$ | $0.7125$ | $0.2913$ | 0.2509 | $0.7117$ | $0.2903$ | 0.2540
ViTb16 | $0.6832$ | $0.6840$ | $0.2985$ | 0.2467 | $0.6833$ | $0.2855$ | 0.2488
Table 3: Collective_debug_train method’s results. Figure 6: Comparison of
resolved failures between collective_debugging and individual_debugging as a
percentage. Relative accuracy is the ratio of the combined debugging’s
accuracy over individual debugging’s accuracy on each specific model.

##  6 Conclusion

In our project, we’ve developed a technique to identify failure modes by
focusing on a specific category of spurious correlations. We then leverage
these detected failures to generate additional samples, allowing the model to
learn from and address its shortcomings. We’ve illustrated the resemblance of
failures within a particular model category, highlighting that models with the
same architecture share more similar failures. Exploiting this insight, we’ve
devised a method to alleviate failures across all models in a category using a
single set of generated data based on the failures of just one model in that
category. Our results indicate that this collective debugging approach can
resolve over $75\%$ of failures addressed through individual debugging
efforts. Our framework empowers users to select the spurious correlation to
identify and mitigate, facilitating the simultaneous debugging of a subset of
models with a single (small) auxiliary set of additional data, thereby saving
both time and resources.

## References

  * Arpit et al. (2017) Devansh Arpit, Stanisław Jastrzębski, Nicolas Ballas, David Krueger, Emmanuel Bengio, Maxinder S Kanwal, Tegan Maharaj, Asja Fischer, Aaron Courville, Yoshua Bengio, et al.  A closer look at memorization in deep networks.  In _International conference on machine learning_ , pp. 233–242. PMLR, 2017.
  * Bansal & Grover (2023) Hritik Bansal and Aditya Grover.  Leaving reality to imagination: Robust classification via generated datasets.  _arXiv preprint arXiv:2302.02503_ , 2023.
  * Barbu et al. (2019) Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz.  Objectnet: A large-scale bias-controlled dataset for pushing the limits of object recognition models.  _Advances in neural information processing systems_ , 32, 2019.
  * Caron et al. (2020) Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, and Piotr Bojanowski.  Unsupervised learning of visual features by contrasting cluster assignments.  In _European Conference on Computer Vision_ , pp. 116–132. Springer, 2020.
  * Caron et al. (2021) Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.  Emerging properties in self-supervised vision transformers.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pp. 9650–9660, 2021.
  * Chen et al. (2018) Xiaozhi Chen, Kaustav Kundu, Zhiqiang Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun.  3d object proposals for accurate object class detection.  In _Advances in Neural Information Processing Systems_ , pp. 2146–2156, 2018.
  * Chung et al. (2019) Yeounoh Chung, Tim Kraska, Neoklis Polyzotis, Ki Hyun Tae, and Steven Euijong Whang.  Slice finder: Automated data slicing for model validation.  In _2019 IEEE 35th International Conference on Data Engineering (ICDE)_ , pp. 1550–1553. IEEE, 2019.
  * Dosovitskiy et al. (2021) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 10687–10698, 2021.
  * Eyuboglu et al. (2022) Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, and Christopher Ré.  Domino: Discovering systematic errors with cross-modal embeddings.  _arXiv preprint arXiv:2203.14960_ , 2022.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 770–778, 2016.
  * He et al. (2019) Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick.  Momentum contrast for unsupervised visual representation learning.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 9729–9738, 2019.
  * Hendrycks & Dietterich (2019) Dan Hendrycks and Thomas Dietterich.  Benchmarking neural network robustness to common corruptions and perturbations.  _arXiv preprint arXiv:1903.12261_ , 2019.
  * Hendrycks et al. (2021a) Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al.  The many faces of robustness: A critical analysis of out-of-distribution generalization.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 8340–8349, 2021a.
  * Hendrycks et al. (2021b) Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song.  Natural adversarial examples.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 15262–15271, 2021b.
  * Ho et al. (2021) Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  In _International Conference on Machine Learning_ , 2021.
  * Huang et al. (2017) Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger.  Densely connected convolutional networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pp. 4700–4708, 2017.
  * Jain et al. (2022) Saachi Jain, Hannah Lawrence, Ankur Moitra, and Aleksander Madry.  Distilling model failures as directions in latent space.  _arXiv preprint arXiv:2206.14754_ , 2022.
  * Jiang et al. (2018) Lu Jiang, Dong Meng, Ioannis Mitliagkas, and J. Zico Kolter.  Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels.  In _Advances in Neural Information Processing Systems_ , pp. 10154–10163, 2018.
  * Kattakinda & Feizi (2021) Priyatham Kattakinda and Soheil Feizi.  Focus: Familiar objects in common and uncommon settings.  _arXiv preprint arXiv:2110.03804_ , 2021.
  * Kattakinda et al. (2022) Priyatham Kattakinda, Alexander Levine, and Soheil Feizi.  Invariant learning via diffusion dreamed distribution shifts.  _arXiv preprint arXiv:2211.10370_ , 2022.
  * Lu et al. (2019) Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee.  Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks.  In _Advances in Neural Information Processing Systems_ , pp. 13–23, 2019.
  * Mitchell et al. (2021) Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning.  Fast model editing at scale.  _arXiv preprint arXiv:2110.11309_ , 2021.
  * Mithun et al. (2020) Nilaksh Mithun, Soubhik Biswas, and CV Jawahar.  Neural modular network for visual reasoning.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 13306–13315, 2020.
  * Nushi et al. (2018) Besmira Nushi, Ece Kamar, and Eric Horvitz.  Towards accountable ai: Hybrid human-machine analyses for characterizing system failure.  In _Proceedings of the AAAI Conference on Human Computation and Crowdsourcing_ , volume 6, pp. 126–135, 2018.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pp. 8748–8763. PMLR, 2021.
  * Raghu et al. (2021) Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy.  Do vision transformers see like convolutional neural networks?  _Advances in Neural Information Processing Systems_ , 34:12116–12128, 2021.
  * Rame et al. (2022) Alexandre Rame, Matthieu Kirchmeyer, Thibaud Rahier, Alain Rakotomamonjy, Patrick Gallinari, and Matthieu Cord.  Diverse weight averaging for out-of-distribution generalization.  _arXiv preprint arXiv:2205.09739_ , 2022.
  * Reed et al. (2015) Scott Reed, Honglak Lee, Dragomir Anguelov, Christian Szegedy, Dumitru Erhan, and Andrew Rabinovich.  Training deep neural networks on noisy labels with bootstrapping.  In _International Conference on Learning Representations (ICLR)_ , 2015.
  * Ribeiro et al. (2016) Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin.  " why should i trust you?" explaining the predictions of any classifier.  In _Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining_ , pp. 1135–1144, 2016.
  * Rombach et al. (2021) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models. 2022 ieee.  In _CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pp. 10674–10685, 2021.
  * Sagawa et al. (2019) Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang.  Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization.  _arXiv preprint arXiv:1911.08731_ , 2019.
  * Santurkar et al. (2021) Shibani Santurkar, Dimitris Tsipras, Mahalaxmi Elango, David Bau, Antonio Torralba, and Aleksander Madry.  Editing a classifier by rewriting its prediction rules.  _Advances in Neural Information Processing Systems_ , 34:23359–23373, 2021.
  * Singla & Feizi (2021) Sahil Singla and Soheil Feizi.  Salient imagenet: How to discover spurious features in deep learning?  _arXiv preprint arXiv:2110.04301_ , 2021.
  * Singla et al. (2021) Sahil Singla, Besmira Nushi, Shital Shah, Ece Kamar, and Eric Horvitz.  Understanding failures of deep networks via robust feature extraction.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 12853–12862, 2021.
  * Singla et al. (2022) Sahil Singla, Atoosa Malemir Chegini, Mazda Moayeri, and Soheil Feiz.  Data-centric debugging: mitigating model failures via targeted data collection.  _arXiv preprint arXiv:2211.09859_ , 2022.
  * Sukhbaatar et al. (2014) Sainbayar Sukhbaatar, Joan Bruna, Manohar Paluri, and Rob Fergus.  Training convolutional networks with noisy labels.  In _Advances in Neural Information Processing Systems (NIPS)_ , pp. 468–476, 2014.
  * Tan & Le (2019) Mingxing Tan and Quoc V Le.  Efficientnet: Rethinking model scaling for convolutional neural networks.  _arXiv preprint arXiv:1905.11946_ , 2019.
  * Wiles et al. (2022) Olivia Wiles, Isabela Albuquerque, and Sven Gowal.  Discovering bugs in vision models using off-the-shelf image generation and captioning.  _arXiv preprint arXiv:2208.08831_ , 2022.
  * Wong et al. (2021) Eric Wong, Shibani Santurkar, and Aleksander Madry.  Leveraging sparse linear layers for debuggable deep networks.  In _International Conference on Machine Learning_ , pp. 11205–11216. PMLR, 2021.
  * Wu et al. (2019) Tongshuang Wu, Marco Tulio Ribeiro, Jeffrey Heer, and Daniel S Weld.  Errudite: Scalable, reproducible, and testable error analysis.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pp. 747–763, 2019.
  * Xiao et al. (2020) Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry.  Noise or signal: The role of image backgrounds in object recognition.  _arXiv preprint arXiv:2006.09994_ , 2020.
  * Zhang et al. (2007) Jianguo Zhang, Marcin Marszałek, Svetlana Lazebnik, and Cordelia Schmid.  Local features and kernels for classification of texture and object categories: A comprehensive study.  _International journal of computer vision_ , 73:213–238, 2007.
  * Zhang et al. (2018a) Jiawei Zhang, Yang Wang, Piero Molino, Lezhi Li, and David S Ebert.  Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models.  _IEEE transactions on visualization and computer graphics_ , 25(1):364–373, 2018a.
  * Zhang et al. (2018b) Min-Ling Zhang, Xingquan Wu, and Zhi-Hua Zhou.  Multi-label learning with missing labels: A probabilistic perspective.  _IEEE Transactions on Knowledge and Data Engineering_ , 30(3):504–517, 2018b.
  * Zhou et al. (2020) Bowen Zhou, Yanhua Wu, Bingbing Ni, Xueying Zhou, Jing Wen, and Jie Zou.  Towards mitigating bias in deep learning-based automated skin lesion classification with balanced groups and label correction.  _IEEE Journal of Biomedical and Health Informatics_ , 24(10):2803–2813, 2020.

##  Appendix A Appendix

Figure 7: examples of 3 most common failure modes of deep learning models Parameter | Value
---|---
Leearning rate | 0.2
Epochs | 1000
Momentum | 0.9
Weight decay | 0.0005
# Chosen common BGs | 5
Lambda | 0.5
Table 4: Shared parameters among all dataset. Models
---
Model_category | Model_name
ResNet | resnet18 | resnet26 | resnet34
resnet50 | resnet101 | resnet152
EfficientNet | efficientnet_b0 | efficientnet_b1 | efficientnet_b2
efficientnet_b3 | efficientnet_b4 | efficientnet_b5
efficientnet_b6 | efficientnet_b7 | efficientnet_b8
efficientnet_l2 |  |
DenseNet | densenet121 | densenet161 |
ViT | vit_base_patch16_224 | vit_base_patch32_224 | vit_large_patch16_224
vit_large_patch32_224 | vit_base_resnet26d_224 | vit_base_resnet50d_224
SWaV | resnet50 | resnet50w2 | resnet50w4
resnet50w5 |  |
MoCo | moco_v2_800ep |  |
DINO | dino_resnet50 | dino_vitb16 | dino_vitb8
dino_vits16 | dino_vits8 |
CLIP | ViT-B32 | RN50 | RN101
ViT-L14 | RN50x4 | RN50x16
Table 5: List of models we tested our debugging framework on.

##  Appendix B Results for all models on ImageNet

Models | Accuracies
---|---
Model |  | Accuracy | Accuracy of | Accuracy of
model | before | Individual Debugging | Random debugging
category | name | debugging | (ours) |
|  | Test | Test | seed | heldout | Test | seed | heldout
EfficientNet | b0 | $0.715$ | $0.7198$ | $0.2034$ | 0.1842 | $0.7134$ | $0.0894$ | $0.0883$
b1 | $0.7373$ | $0.74415$ | $0.2154$ | 0.1885 | $0.7399$ | $0.1003$ | $0.1011$
b2 | $0.7525$ | $0.7591$ | $0.2283$ | 0.1909 | $0.7448$ | $0.0957$ | $0.0942$
b3 | $0.7634$ | $0.7730$ | $0.2318$ | 0.1991 | $0.7669$ | $0.1066$ | $0.1010$
b4 | $0.7701$ | $0.7780$ | $0.2398$ | 0.2068 | $0.7719$ | $0.0955$ | $0.0960$
b5 | $0.7821$ | $0.7819$ | $0.2405$ | 0.2055 | $0.7761$ | $0.0893$ | $0.0915$
b6 | $0.7884$ | $0.7886$ | $0.2561$ | 0.2083 | $0.7863$ | $0.0941$ | $0.0914$
b7 | $0.7895$ | $0.7903$ | $0.2600$ | 0.2126 | $0.7898$ | $0.0951$ | $0.0972$
b8 | $0.7928$ | $0.7951$ | $0.2653$ | 0.2147 | $0.7932$ | $0.0972$ | $0.0934$
DenseNet | 121 | $0.6792$ | $0.6869$ | $0.2138$ | 0.1592 | $0.6773$ | $0.0651$ | $0.0664$
161 | $0.7254$ | $0.7332$ | $0.2418$ | 0.1833 | $0.7249$ | $0.0779$ | $0.0771$
ViT | base_patch16_224 | $0.739$ | $0.7477$ | $0.2501$ | 0.2193 | $0.7399$ | $0.1047$ | $0.1044$
base_patch32_224 | $0.7456$ | $0.7493$ | $0.2574$ | 0.2199 | $0.7469$ | $0.1078$ | $0.1072$
large_patch16_224 | $0.7493$ | $0.7539$ | $0.2644$ | 0.2263 | $0.7468$ | $0.0952$ | $0.0957$
large_patch32_224 | $0.7535$ | $0.7545$ | $0.2674$ | 0.2274 | $0.7553$ | $0.1023$ | $0.1041$
SWaV | resnet50 | $0.4254$ | $0.4384$ | $0.1403$ | 0.1274 | $0.4267$ | $0.0662$ | $0.0624$
resnet50w2 | $0.4317$ | $0.4328$ | $0.1583$ | 0.1294 | $0.4319$ | $0.0683$ | $0.0652$
resnet50w4 | $0.4402$ | $0.4477$ | $0.1592$ | 0.1304 | $0.4416$ | $0.0672$ | $0.0617$
resnet50w5 | $0.4526$ | $0.4589$ | $0.1633$ | 0.1363 | $0.4552$ | $0.0696$ | $0.0703$
MoCo | v2_800ep | $0.6931$ | $0.6946$ | $0.2041$ | 0.1584 | $0.6937$ | $0.0943$ | $0.0917$
Table 6: Accuracy of our method comparing to the debug random. Note that the
accuracy of models on debug_seed and debug_heldout was zero before debugging.
These results are for all models that we tested our method on.