# Rectifying Group Irregularities in Explanations for Distribution Shift

Adam Stein Yinjun Wu Eric Wong Mayur Naik
{steinad, wuyinjun, exwong<EMAIL_ADDRESS>
University of Pennsylvania

###### Abstract

It is well-known that real-world changes constituting distribution shift
adversely affect model performance. How to characterize those changes in an
interpretable manner is poorly understood. Existing techniques to address this
problem take the form of shift explanations that elucidate how to map samples
from the original distribution toward the shifted one by reducing the
disparity between these two distributions. However, these methods can
introduce group irregularities, leading to explanations that are less feasible
and robust. To address these issues, we propose Group-aware Shift Explanations
(GSE), a method that produces interpretable explanations by leveraging worst-
group optimization to rectify group irregularities. We demonstrate how GSE not
only maintains group structures, such as demographic and hierarchical
subpopulations, but also enhances feasibility and robustness in the resulting
explanations in a wide range of tabular, language, and image settings.

##  1 Introduction

Classic machine learning theory assumes that the training and testing data are
sampled from the same distribution. Unfortunately, distribution shifts
infringe on this requirement and can drastically change a model’s behavior
[KGB18]. For instance, training a model on data collected from one hospital
may result in inaccurate diagnoses for patients from other hospitals due to
variations in medical equipment [Zec+18, SS20]. Similarly, shifts from daytime
to nighttime or from clear to rainy weather are major obstacles for autonomous
driving [DV18, Wan+20].

When such a distribution shift occurs, it is often useful to understand _why_
and _how_ the data changed. For example, suppose a doctor observes that their
medical AI model’s performance is degrading. Before arbitrarily changing the
model, the doctor should first understand the changes in their patient data
[SS20]. Similarly, a self-driving engineer would have an easier time to adapt
an autonomous driving system to a new environment if it was known that the
shift resulted from changing weather conditions [AI22]. In addition,
policymakers need to understand why and how an event or crisis happens so that
they can adjust their policies appropriately [LST18].

To facilitate better understanding of a distribution shift, it is crucial to
generate appropriate shift explanations. The format of a shift explanation is
a mapping from the original distribution (called source distribution) to the
shifted one (called target distribution) such that their disparity is reduced.
For example, [KI22] find a direct mapping of points from the original
distribution toward the shifted one via optimal transport [PC+17] and its
variant, $K$-cluster transport. Another approach is to use counterfactual
explanation methods such as DiCE [MST20] which explain classifiers. A
counterfactual explanation of a classifier between source and target
distributions will map each source instance such that the model classifies
this instance as from the target distribution.

Shift explanations produced by state-of-the-art methods seek to optimize
global objectives such as minimizing the difference between the target
distribution and the mapped source distribution. However, mappings that merely
satisfy this goal are not necessarily good explanations: they can fail to be
feasible in practice, or lack robustness to perturbations in the source
distribution. This in turn fundamentally limits the practicality of shift
explanations produced by existing methods.

As a concrete example of this phenomenon, Figure 1 shows explanations from
[KI22] that map individuals with low income (source distribution) to
individuals with high income (target distribution) in the Adult dataset from
the UCI Machine Learning Repository [Bla98]. Such explanations can help reveal
insights about income inequalities that enable a policymaker to propose better
policies or an individual to understand how to increase their income. At a
dataset level, $K$-cluster transport [PC+17] can produce a shift explanation
that effectively maps the source distribution to the target, resulting in a
87% reduction in the Wasserstein distance between these two distributions.
However, upon closer inspection, this explanation shifts a majority male
cluster to a majority female cluster. Focusing on the female subpopulation of
the source and target, the explanation only decreases the Wasserstein distance
by 73.6%. Such an explanation is not useful if gender change is infeasible—or
less feasible than changing other attributes such as education level.

Our key insight to achieving high-quality shift explanations is to steer the
generated explanations to respect subpopulations, or groups, in the data.
Since groups are highly context-specific in nature, we seek an approach that
is general and factors this objective jointly with the overall fitness of the
produced mapping from source to target populations. In our running example,
assuming gender-based grouping, such an approach should yield a mapping that
minimizes disrupting the groups while maximizing overall fitness. As depicted
at the bottom of Figure 1(a), it is in fact possible to achieve such a mapping
by using the same underlying K-cluster transport method, that increases the
reduction of Wasserstein distance from 73.6% to 81.8% between source samples
and target samples within the female group, and only slightly impacts the
reduction of Wasserstein distance between the overall source and target
populations (from 87% to 86.6%).

(a)

(b)

Figure 1: (a) shows an example of explaining the distribution shift from a
source (i.e., low-income) population to a target (i.e., high-income)
population in a subset of the Adult dataset [Bla98] using two different
methods: vanilla $K$-cluster transport [KI22] and our GSE $K$-cluster
transport. The shift explanation produced by the vanilla method explains the
shift by subtracting 1 from the “sex” attribute. On the other hand, GSE
generates an explanation that preserves the male and female subpopulations,
and instead requires the samples in the source to change their education level
and age. (b) shows how a particular member of the cluster is modified under
the two methods. The explanation produced by the state-of-the-art method
changes the sample’s sex whereas our method does not.

To this end, we propose Group-aware Shift Explanations (GSE), an explanation
method for distribution shift that preserves groups in the data. We develop a
unifying framework that incorporates heterogeneous methods for producing shift
explanations in diverse settings and allows us to apply GSE to these methods.
In addition, GSE enhances the feasibility and robustness of the resulting
explanations. Through extensive experiments over a wide range of tabular,
language, and image datasets, we demonstrate that GSE not only maps source
samples closer to target samples belonging to the same group, thus preserving
group structure, but also boosts the feasibility and robustness by up to 28%
and 42% respectively.

Our main contributions are summarized as follows:

  1. 1.

We identify and demonstrate group irregularities as a class of problems that
can adversely affect the quality of shift explanations by state-of-the-art
methods.

  2. 2.

We propose one method, Group-aware Shift Explanations (GSE), to rectify group
irregularities when explaining distribution shift.

  3. 3.

We propose a general framework to unify heterogeneous shift explanation
methods such as $K$-cluster transport and show how they can be integrated into
GSE to enforce group structures across varied settings, including tabular,
NLP, and image settings.

  4. 4.

We demonstrate how GSE maintains group structures and enables more feasible
and robust shift explanations occurring in diverse datasets across different
domains.

##  2 Motivation

In this section, we identify issues with existing shift explanations in terms
of group irregularities.

###  2.1 Constructing Mappings for Shift Explanations

A distribution shift is any change from an initial distribution, called the
source, to another distribution, called the target. We follow prior work from
[KI22] to define a shift explanation as a mapping from the source distribution
to the target distribution. For instance, Figure 1 shows a shift explanation,
called a $K$-cluster explanation [KI22] which maps the source distribution to
the target distribution by subtracting 1 from the “sex” attribute among other
changes. Different shift explanation methods can produce different types of
mappings.

###  2.2 Group Irregularities in Existing Shift Explanations

To find a shift explanation, state-of-the-art methods primarily minimize the
disparity between the source distribution and the target distribution. For
example, $K$-cluster transport minimizes an objective depending on the
Wasserstein distance between the source and the target distribution [PC+17].
However, this is not sufficient for finding high-quality explanations. Figure
1 shows such an example with $K$-cluster explanations where a mostly male
group of the source gets mapped to a female group. In this case, the overall
Wasserstein distance is reduced by 87%, but the Wasserstein distance for the
female subpopulation is decreased much less in Figure 1(a).

####  2.2.1 Impact on Explanation Feasibility

Shift explanations which break apart groups of the data are not only
problematic because they degrade on the subpopulation level, but they can also
be overall infeasible. Feasibility is a measure of how useful an explanation
is to a downstream user. For instance, in Figure 1, the sex attribute may be
unactionable, so the $K$-cluster explanation which modifies the sex attribute
would be useless for a policymaker who designs policies to help increase the
income of the low-income population. Overall, the $K$-cluster explanation in
Figure 1 is only feasible for 75.5% of the source distribution, meaning that
24.5% of the source samples have their sex attribute modified by the shift
explanation. Later, we show how our method, which rectifies these group
irregularities, results in more feasible explanations for the overall source
distributions.

####  2.2.2 Impact on Explanation Robustness

(a) Adult dataset.

(b) Civil Comments dataset.

Figure 2: An example of poor robustness of an explanation. Even if an
explanation is feasible (top), small perturbations to the source distribution
can make it become infeasible (bottom). The details on how groups are derived
for the Civil Comments data is in Appendix F.

Group irregularities can also reduce robustness, meaning that small changes to
a source distribution result in large changes to the shift explanation. Figure
2 shows an example of poor explanation robustness in the Adult and Civil
Comments dataset. In Figure 2(a), a small perturbation to the source
distribution leads to the explanation changing from subtracting three from age
to adding one to sex (changing from female to male) and subtracting six from
age. Figure 2(b) shows a shift explanation that maps a non-toxic sample
relating to medicine into the target distribution of toxic sentences. After a
small perturbation, the explanation maps the same sample by adding the words
“shooter” and “stupid” which is an unfeasible change since it changes the
topic of the sample to violence. Ideally, we want a shift explanation to be
robust to very small changes to the source distribution since it should
explain general behavior instead of relying on minute details of a
distribution.

Robustness is problematic even when overall feasibility is high. For example,
in Figure 2(a), the $K$-cluster explanation is feasible for 76.7% of the
source distribution samples. After applying a small perturbation, however, the
$K$-cluster explanation now modifies the sex attribute for an additional 11.7%
of the source distribution samples, reducing feasibility to 65.0%. Thus, even
an explanation with high overall feasibility is not ideal if small changes to
the source distribution can lead to drastic changes to the explanation.

##  3 Group-aware Shift Explanations (GSE)

In this section, we discuss our method, Group-aware Shift Explanations (GSE).
First, we introduce GSE in the context of $K$-cluster transport, and
PercentExplained (PE) a Wasserstein-distance based metric for evaluating the
quality of shift explanations. Then, we present a unified shift explanation
framework which allows GSE to work with arbitrary shift explanations and also
generalizes it from tabular data to NLP and image data. Finally, we formalize
the notions of feasibility and robustness introduced in Section 2 as
additional metrics to evaluate the quality of shift explanations.

###  3.1 Preliminaries on $K$-cluster transport and PercentExplained (PE)

The shift explanations produced by $K$-cluster transport can be denoted by a
mapping function $M(x;\theta_{x})$. The function $M(x;\theta_{x})$ maps a
source sample $x$ towards the target distribution by a distance of
$\theta_{x}$, which is a learnable parameter. As the name $K$-cluster
transport suggests, all the source samples are grouped into a set of clusters,
$C$, with $K$-means clustering algorithm, and within one cluster, $c\in C$,
all the samples share the same $\theta_{c}$. Therefore, the mapping function
for $K$-cluster transport is formulated as follows:

$\displaystyle M(x;\theta)=x+\sum\nolimits_{c\in C}\mathbb{1}_{x\in
c}\theta_{c},\text{in which, }\theta=\\{\theta_{c}|c\in C\\}.$

Optimizing $\theta$ for $K$-cluster transport. According to [KI22], $\theta$
is solved by maximizing PercentExplained (PE). Suppose the source distribution
and the target distribution are denoted by $P$ and $Q$ respectively, then PE
is formulated as follows:

$\displaystyle\text{PE}(\theta;M,P{},Q{})=1-W_{2}^{2}(M(P{};\theta),Q{})/W_{2}^{2}(P{},Q{}),$
(1)

where $W_{2}(\cdot)$ is the Wasserstein-2 distance and $M(P;\theta)$ is the
mapping $M$ applied to every sample in the source, $\\{M(x;\theta)\mid x\in
P\\}$. Intuitively speaking, PE quantifies how much the distance between $P{}$
and $Q{}$ is reduced after $M(\cdot;\theta)$ is applied to $P$. A high PE
means that the explanation closely matches the overall source to the overall
target distribution. Using differentiable implementations of the Wasserstein-2
distance, like the GeomLoss library [Fey+19], allows us to directly optimize
PE using gradient descent.

###  3.2 Worst-group PE for GSE

To rectify the issues identified in Section 2 in existing shift explanations,
we can ideally optimize PE for all pre-specified groups such that all groups
are preserved by the shift explanation. This ideal, however, is not applicable
to finding dataset-level explanations. Instead, we propose Group-aware Shift
Explanations (GSE) to optimize the worst-group PE among all groups, which thus
implicitly improves PE for all groups simultaneously.

Specifically, suppose the source and target are partitioned into $G$ disjoint
groups, i.e., $P{}=\\{P_{1},P_{2},\dots,P_{G}\\}$ and
$Q{}=\\{Q_{1},Q_{2},\dots,Q_{G}\\}$, in which, $P_{g}$ and $Q_{g}$ belong to
the same group, e.g., the male sub-populations from $P$ and $Q$. We can now
evaluate PE on a shared group from the source and target as follows:

$\displaystyle\text{PE}_{g}(\theta;M,P_{g},Q_{g})=1-W_{2}^{2}(M(P_{g};\theta),Q_{g})/{W_{2}^{2}(P_{g},Q_{g})}.$
(2)

The above formula measures how much the distance between $P_{g}$ and $Q_{g}$
is reduced by the given shift explanation, $M$. Then worst-group PE can then
be calculated over all $G$ groups, i.e.,:

$\displaystyle\text{WG-
PE}(\theta;M,P,Q)=\min\nolimits_{g}\text{PE}_{g}(\theta;M,P_{g},Q_{g}).$ (3)

This metric indicates how much the distance between any pair of $P_{g}$ and
$Q_{g}$ is reduced, in the worst case. Instead of learning a shift explanation
which maximizes PE over the entire distributions but may leave some groups
with arbitrarily small PE, GSE learns an explanation maximizing WG-PE.
Optimizing $\theta$ to maximize the WG-PE can guarantee that for every pair of
$P_{g}$ and $Q_{g}$, $\text{PE}_{g}(\theta;P_{g},Q_{g},M)$ is not approaching
0.

Intuitively, GSE regularizes the groups where PE becomes arbitrarily small
even though the overall PE is large. Note that the goal of GSE is still to
learn shift explanations at the dataset level rather than find explanations
for each group separately. This means that the vanila $K$-cluster transport
and GSE $K$-cluster transport produce explanations of the same complexity. As
we will show in Section 4, both feasibility and robustness issues can also be
mitigated with GSE.

###  3.3 A Unified Framework for General Settings

In this section, we propose a generic framework which generalizes GSE from
$K$-cluster transport to broad types of shift explanation methods, and from
tabular data to a wide range of settings, including NLP and image data.

####  3.3.1 Generalizing to other shift explanation methods

##### Generalizing the Mapping $M(x;\theta)$

First of all, recall that the shift explanations produced by $K$-cluster
transport could be represented by the mapping function $M(x;\theta)$, which
can be any function taking the sample $x\in P$ and the moving distance
$\theta$ as input. For example, for optimal transport [KI22],
$M(x;\theta)=x+\theta_{x}$ where the moving distance, $\theta_{x}$, varies
between different $x$.

##### Generalizing the Objective Function beyond PE

Note that one objective function, PercentExplain (PE), is optimized for
solving $\theta$ for $K$-cluster transport. Indeed, any differentiable loss
function, $L(\theta;M,P{},Q{})$ for optimizing $\theta$ can be employed, which
takes the mapping, $M$; the source distribution, $P$; and the target
distribution, $Q$, as input. For instance, for optimal transport and
$K$-cluster transport [KI22], $L$ is $1-\text{PE}$. The details for how to
instantiate $M$ and $L$ for other shift explanation methods, e.g., optimal
transport and DiCE, are given in Appendix A. But note that the feasibility and
robustness metrics introduced in Section 2.2 (will be formalized in Section
3.4) are not suitable due to their non-differentiability. Therefore, they only
serve as post-hoc evaluation metrics.

We can now provide a general form of GSE for any shift explanation method
decomposed as a parameterized mapping $M(\cdot;\theta)$ and an objective
function $L$ for learning $\theta$. First, we extend our formulation of WG-PE
in Equation 3 beyond PE by replacing PE with $1-L$ (recall that $L$ is
$1-\text{PE}$ for $K$-cluster transport), i.e:

$\displaystyle\begin{split}&\text{WG-}L(\theta;M,P{},Q{})=\min\nolimits_{g}(\\{1-L(\theta;M,P_{g},Q_{g})\\}_{g=1}^{G})=\max\nolimits_{g}(\\{L(\theta;M,P_{g},Q_{g})\\}_{g=1}^{G})\end{split}$
(4)

Recall that $P_{g}$ and $Q_{g}$ represent a group of samples from the source
and the target respectively, belonging to the same group. We further
generalize Equation (4) by using an arbitrary aggregation function $F$ in
place of the $\max$ function and regularizing with the loss calculated between
the whole $P{}$ and $Q{}$ to balance the optimization between the worst group
and the overall distribution, i.e.:

$\displaystyle\begin{split}&\text{WG-}L(\theta;M,P{},Q{})=\operatorname*{arg\,min}\nolimits_{\theta}\left(F(\\{L_{g}(\theta;M,P_{g},Q_{g})\\}_{g=1}^{G})+\lambda\cdot
L(\theta;M,P{},Q{})\right)\end{split}$ (5)

where $\lambda$ is a hyper-parameter and $F$ is an aggregation function. The
choice of $F$ and $\lambda$ for our experiments is given in Appendix D.4.

####  3.3.2 Generalizing to language and image data

Figure 3: Pipeline of generating a shift explanation for a raw image and
evaluating the explanation. First, each image from the source and the target
is transformed to its caption with pretrained Img2Text models (e.g., CLIP
Interrogator). Then for the caption data, we derive shift explanations over
interpretable features, i.e., BoW features (denoted by $\widetilde{P}$ and
$\widetilde{Q}$ for source images and target images respectively). After
deriving shift explanations over $\widetilde{P}$ and $\widetilde{Q}$, the
modified caption is produced, which is then fed into a pretrained Text2Img
model (e.g., Stable Diffusion) for reverse featurization.

It is worth noting that shift explanations are built upon interpretable
features, e.g., age or education level for the Adult dataset, which, however,
are not available for image and language data. Therefore, we add two
additional steps in our framework. The first one is a featurization step,
which extracts interpretable features from the language and image data.
Second, we add a reverse featurization step for converting modified features
back to the raw data space for producing mapped source samples.

##### Generalizing to language data

For language data, the featurization step leverages techniques such as Bag-of-
words (BoW) and N-Gram models to produce token-level features. These features
for the source and target data are denoted by $\widetilde{P}$ and
$\widetilde{Q}$ respectively. Then, $\widetilde{P}$ and $\widetilde{Q}$ can be
integrated into $L(\theta;M,P,Q)$ and $\text{WG-}L(\theta;M,P,Q)$ for solving
$\theta$. The resulting mapping function $M(\cdot;\theta)$ is in the form of
removal or addition of words. Therefore, in the reverse featurization step, we
follow the explanations to either remove words from the sentences in the
source distribution or add words to the beginning of these sentences.

##### Generalizing to image data

In comparison to language data, both featurization and reverse featurization
steps over images are even more difficult. To address this issue, we propose
an end-to-end pipeline shown in Figure 3. The featurization step starts by
leveraging image-to-text models such as CLIP Interrogator [22] to produce
captions for each image from the source distribution and the target
distribution. These captions are then processed in the same manner as language
data to obtain interpretable features, such as BoW features, which are denoted
by $\widetilde{P}$ and $\widetilde{Q}$ for the source and the target
respectively. We then follow the way of generating shift explanations over
language data to find shift explanations. Finally, the reverse featurization
step follow the explanation to produce modified captions for each source
image, which is then transformed back to an image using a text-to-image model
such as stable diffusion model [Rom+21].

###  3.4 Feasibility and Robustness Metrics

Despite varied objective functions $L(\theta;M,P,Q)$ across the different
shift explanation methods, to our knowledge, PercentExplained (PE) from
Equation (1) is the only metric to evaluate the quality of shift explanations
in the literature [KI22]. We propose to use feasibility and robustness,
introduced in Section 2, as additional metrics. We formalize feasibility and
robustness as evaluation metrics below.

Feasibility This notion of feasibility has been studied in the literature of
counterfactual explanations [Poy+20]. Formally speaking, feasibility is
defined as the percentage of source samples for which the explanations are
feasible, i.e.:

$\displaystyle\text{\% Feasible}=[{\sum\nolimits_{x\in
P}a(x,M(x;\theta))}]/{\|P\|}$ (6)

where $a(\cdot,\cdot)$ is a function which outputs 1 when the change from $x$
to $M(x;\theta)$ is feasible, and 0 otherwise (say changing education is
feasible while changing sex is almost infeasible for Adult dataset).
Evaluating feasibility becomes indispensable in the presence of unactionable
attributes such as “sex”. Since GSE takes groups into account, we can enhance
an explanation’s feasibility by constructing groups using the unactionable
attributes.

Robustness The notion of robustness is also proposed in prior works such as
[AJ18, Aga+22], which evaluates variations of the explanations with respect to
small perturbation over the distribution of the source data. To add such small
perturbations to the source data distribution, $P$, we randomly perturb
$\epsilon\%$ of the feature values for some pre-specified feature, e.g.,
changing the sex of 1% of the samples from male to female. The resulting
perturbed source distribution is denoted as $P(\epsilon)$. We investigate the
robustness of shift explanations with respect to two types of perturbations,
random perturbations and worst-case perturbations. These two types of
perturbations lead to two robustness metrics (denoted by $\Omega$ and
$\Omega_{\text{worst}}$ respectively) which are quantified with the following
formula adapted from the robustness metrics in [AJ18]):

$\displaystyle\begin{split}&\Omega(\theta;M,P,Q,\epsilon)={\|M(P;\theta)-M(P(\epsilon);\theta(\epsilon))\|_{2}}/{\|P-P(\epsilon)\|_{2}},\\\
&\Omega_{\text{worst}}(\theta;M,P,Q)=\max\nolimits_{\epsilon}\Omega(\theta;M,P,Q,\epsilon),\end{split}$
(7)

in which $\theta$ and $\theta(\epsilon)$ are derived by finding a shift
explanation with the source distribution as $P$, or $P(\epsilon)$,
respectively. Since it is difficult to exactly solve $\Omega_{\text{worst}}$,
we therefore follow [AJ18] to determine $\Omega_{\text{worst}}$ from a finite
set of epsilons. Details are shown in Appendix B.

##  4 Experiments

We present our experiments for evaluating the effectiveness of GSE compared to
shift explanations which ignore group structures. In what follows, we describe
the experimental setup in Section 4.2, the datasets in the experiments in
Section 4.1, and our experimental results in Section 4.3.

###  4.1 Datasets

We perform experiments on three different types of data: tabular, language,
and vision data. For tabular data, we use the Adult and Breast Cancer datasets
(Breast dataset for short) from the UCI Machine Learning Repository [DG17].
For language data, we evaluate on the Civil Comments dataset [Bor+19] (Civil
dataset for short). Finally, for image data we use the version of the ImageNet
dataset from [STM21]. Appendix C provides further details of these datasets.

Distribution shift setup For tabular data and language data, we match the
setup of [KI22] and [STM21], and consider distribution shift between the
different class labels: shift from low-income to high-income for Adult, benign
to malignant for Breast, toxic to non-toxic for Civil dataset, and between
sub-classes of “Mammal” for ImageNet.

Sub-population setup For the Adult dataset, we group samples by their sex
attribute. For Breast dataset, we group by the ratio between “cell radius” and
“cell area” attributes (see Appendix D for details), leading to 3 groups. For
Civil dataset, groups are defined by samples with and without the “female”
demographic feature. For ImageNet, groups are defined by the superclasses
“rodent/gnawer” and “ungulate/hooved mammal” of the ImageNet label. As we show
in Section 4.3.1, despite only a few pre-specified groups across all the
datasets, the state-of-the-art shift explanations still break those group
structures and lead to poor feasibility and robustness.

###  4.2 Experimental Setup

For all datasets described in Section 4.1, we evaluate three shift explanation
methods: $K$-cluster transport ($K$-cluster), Optimal transport (OT), and
DiCE. Due to space limitations, only the results of $K$-cluster transport are
included in this section and other experiments can be found in Appendix C. For
each method, we compare the vanilla explanations and those generated using
GSE. The former one are derived by optimizing group-free objectives such as PE
in Equation (1) while the latter one are constructed by optimizing group-aware
objectives such as WG-PE in Equation (3).

The three different explanation methods in addition to their counterparts
using GSE are evaluated along the following axes:

  * •

PE and WG-PE. Note that for image dataset, PE and WG-PE are evaluated over the
embeddings of images rather than the raw images, produced by leveraging a
pretrained ResNet-50 model.

  * •

% Feasible as shown in Equation (6).

  * •

Robustness and worst-case robustness as shown in Equation (7) by perturbing a
randomly selected 1% of the feature values for six randomly selected features.

Recall that the % Feasible and Robustness metrics are not differentiable, and
thus used as post-hoc evaluation metrics for shift explanations. Further
details of the experimental setup are in Appendix D.

###  4.3 Results

Figure 4:  Comparison of vanilla shift explanation and GSE shift explanation
to explain the sub-population shift in Imagenet dataset with $k$-cluster
transport. This shows how the explanation modifies samples in one cluster for
the image data. The vanilla explanation maps this cluster consisting of mostly
antelopes (which belong to “ungulate/hooved mammal” group) shown on the left
to samples in the target of porcupines shown on the top right. The explanation
of “-2 horns” and “+2 spiky” means that two occurrences of the word “horns”
should be removed and “spiky” should be added twice to the caption. Then we
apply the reverse featurization step introduced in Section 3 to this modified
caption for generating mapped source image. By inspecting its closest target
image, we observe that this mapped image is closer to the group
“rodent/gnawe”, thus breaking the group structure. In contrast, GSE
explanation maps this cluster closer to horse images, which both belong to
“ungulate/hooved mammal”, thus preserving groups (shown on the bottom right).

Our results are divided into quantitative and qualitative results below. We
also perform experiments without prespecified groups for language data, and
the results are in Appendix F.

####  4.3.1 Quantitative Results

Table 1: Comparison of PE, WG-PE and %Feasible between vanilla $K$-cluster explanations and GSE $K$-cluster explanations (Higher is better).  | Dataset | Adult | Breast | Civil | ImageNet
---|---|---|---|---|---
PE | Vanilla | 24.92$\pm$1.26 | 85.35$\pm$0.32 | 12.73$\pm$0.14 |  4.46 $\pm$6.35
GSE |  24.27$\pm$0.61 |  84.54$\pm$0.71 |  6.23$\pm$0.54 | 12.25$\pm$1.96
WG-PE | Vanilla |  6.09$\pm$5.60 |  53.31$\pm$5.54 |  3.85$\pm$0.02 |  -16.55$\pm$5.31
GSE | 21.91$\pm$2.25 | 73.15$\pm$0.37 | 6.23$\pm$5.90 | -4.69$\pm$ 2.06
%Feasible | Vanilla |  84.73$\pm$10.80 | 58.49$\pm$3.71 |  57.50 $\pm$ 0.00 |  20.96 $\pm$9.36
GSE | 100.0$\pm$0.0 | 58.49$\pm$3.71 | 61.83 $\pm$ 0.94 | 48.61$\pm$3.23
Table 2: Comparison of Robustness and Worst-case Robustness between vanilla $K$-cluster explanations and GSE $K$-cluster explanations (Lower is better). | Dataset | Adult | Breast | Civil | ImageNet
---|---|---|---|---|---
Robustness | Vanilla | 78.19 | 436.63 | 1.61 | 19.65
GSE | 66.52 | 251.86 | 1.39 | 18.20
Worst-case Robustness | Vanilla | 312.56 | 325674.59 | 17.22 | 24.18
GSE | 298.25 | 349549.71 | 17.36 | 17.84

The main quantitative results of vanilla $K$-cluster explanations and GSE
$K$-cluster explanations are shown in Table 1-2. First of all, as Table 1
shows, for vanilla $K$-cluster explanations, a huge gap exists between the
overall PE metric and WG-PE, which is up to 32% (see Breast dataset). This
thus indicates that these explanations fail to map at least one group of
source samples to the target samples of the same group, thus causing group
irregularity.

By comparing GSE explanations against the vanilla explanations, GSE almost
always results in a higher WG-PE (up to 20% improvement, see Breast dataset)
than vanilla explanations, while only slightly hurting overall PE on Adult,
Breast and Civil dataset, and even improving it on ImageNet dataset. We also
notice that GSE always produces more feasible explanations in comparison to
vanilla explanations, which improves %Feasible by up to 28%. This is primarily
due to the fact that GSE searches the explanations preserving groups by
nature. Moreover, according to Table 2, GSE improves both the robustness and
worst-case robustness by up to 42% (see the Robustness metric for Breast
dataset) in almost all the cases across all the datasets.

####  4.3.2 Qualitative Results

For a qualitative analysis of vanilla shift explanations compared to our GSE
ones, we first look at some examples of group irregularities in terms of
broken and preserved groups. For image data, Figure 4 shows the shift in an
antelope cluster of the $K$-cluster explanation. We see that the vanilla
explanation maps antelopes to porcupines which breaks the “ungulate/hooved
mammal” group since antelopes are hooved animals while porcupines are rodents.
In addition, observing the generated examples for this cluster shows that
converting an antelope to a porcupine is difficult and yields unusual-looking
results. On the other hand, GSE maps this cluster of antelopes to horses which
preserves the groups since horses are also hooved animals. The resulting
generated images from this explanation are also clearly images of horses which
explains why GSE has higher feasibility than vanilla explanations for image
data.

We also perform qualitative analysis to understand why GSE also improves
robustness, which is included in Appendix E. The qualitative analysis on
language and tabular data is also given in Appendix E.

##  5 Related Work

Explaining distribution shift. [KI22] proposes three different mappings of
varying levels of interpretability and expressiveness as shift explanations. A
related problem concerns finding counterfactual explanations for explaining
model behaviors [MST20]. Counterfactual explanation techniques find the
minimal perturbation which changes a model’s prediction on a given sample
[WMR17, Cha+, Rat19]. Although not originally created to explain distribution
shift, we adapt these methods to our setting (see Appendix A for details).
Note that none of these techniques take group structures into account. It is
also worth noting that some works such as [HZ21] explain how the models are
adapted across distributions rather than explain the shift of a distribution
itself, which is thus outside the scope of this paper.

Worst group robustness. Improving model robustness over sub-populations using
group information is extensively studied in the robustness literature. Here,
the main goal is to minimize the loss on the worst performing sub-population.
This problem can be addressed by directly optimizing worst-group loss [Sag+19,
Zha+], re-weighting sub-populations [Liu+21, BL19], or performing data
augmentation on the worst group [Goe+]. Rather than focus on improving worst-
group model performance, our focus is to find explanations that preserve group
structures.

Domain generalization and adaptation. Common solutions for dealing with
distribution shift include domain generalization and domain adaptation. We
survey them in detail in Appendix H.

##  6 Conclusion and Future Work

We identified a problem with all existing approaches for explaining
distribution shift: the blindness to group structures. Taking group structures
into account, we developed a generic framework that unifies existing solutions
for explaining distribution shift and allows us to enhance them with group
awareness. These improved explanations for distribution shift can preserve
group structures, as well as improve feasibility and robustness. We
empirically demonstrated these properties through extensive experiments on
tabular, language, and image settings.

## References

  * [22] “Clip interrogator”, https://github.com/pharmapsychotic/clip-interrogator, 2022
  * [Aga+22] Chirag Agarwal et al.  “Rethinking stability for attribution-based explanations”  In _arXiv preprint arXiv:2203.06877_ , 2022
  * [AI22] AI  “Why Self-Driving Cars Fail: Computer Vision Challenges for Autonomous Vehicles”, 2022  URL: https://www.manot.ai/why-self-driving-cars-fail-computer-vision-challenges-for-autonomous-vehicles
  * [AJ18] David Alvarez-Melis and Tommi S Jaakkola  “On the robustness of interpretability methods”  In _arXiv preprint arXiv:1806.08049_ , 2018
  * [Ban+18] Peter Bandi et al.  “From detection of individual metastases to classification of lymph node status at the patient level: the camelyon17 challenge”  In _IEEE transactions on medical imaging_ 38.2 IEEE, 2018, pp. 550–560
  * [Bee+21] Sara Beery, Arushi Agarwal, Elijah Cole and Vighnesh Birodkar  “The iwildcam 2021 competition dataset”  In _arXiv preprint arXiv:2105.03494_ , 2021
  * [BL19] Jonathon Byrd and Zachary Lipton  “What is the effect of importance weighting in deep learning?”  In _International conference on machine learning_ , 2019, pp. 872–881  PMLR
  * [Bla98] Catherine Blake  “UCI repository of machine learning databases”  In _http://www. ics. uci. edu/~ mlearn/MLRepository. html_ University of California, Department of InformationComputer Science, 1998
  * [Bor+19] Daniel Borkan et al.  “Nuanced metrics for measuring unintended bias with real data for text classification”  In _Companion proceedings of the 2019 world wide web conference_ , 2019, pp. 491–500
  * [BSC18] Yogesh Balaji, Swami Sankaranarayanan and Rama Chellappa  “Metareg: Towards domain generalization using meta-regularization”  In _Advances in neural information processing systems_ 31, 2018
  * [Cha+] Chun-Hao Chang, Elliot Creager, Anna Goldenberg and David Duvenaud  “Explaining Image Classifiers by Counterfactual Generation”  In _International Conference on Learning Representations_
  * [CS19] Safa Cicek and Stefano Soatto  “Unsupervised domain adaptation via regularized conditional alignment”  In _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 1416–1425
  * [Den+09] Jia Deng et al.  “Imagenet: A large-scale hierarchical image database”  In _2009 IEEE conference on computer vision and pattern recognition_ , 2009, pp. 248–255  Ieee
  * [DG17] Dheeru Dua and Casey Graff  “UCI Machine Learning Repository”, 2017  URL: http://archive.ics.uci.edu/ml
  * [DV18] Dengxin Dai and Luc Van Gool  “Dark model adaptation: Semantic image segmentation from daytime to nighttime”  In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_ , 2018, pp. 3819–3824  IEEE
  * [Fey+19] Jean Feydy et al.  “Interpolating between Optimal Transport and MMD using Sinkhorn Divergences”  In _The 22nd International Conference on Artificial Intelligence and Statistics_ , 2019, pp. 2681–2690
  * [Goe+] Karan Goel, Albert Gu, Yixuan Li and Christopher Re  “Model Patching: Closing the Subgroup Performance Gap with Data Augmentation”  In _International Conference on Learning Representations_
  * [HZ21] Yunzhong Hou and Liang Zheng  “Visualizing adapted knowledge in domain transfer”  In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 13824–13833
  * [KGB18] Alexey Kurakin, Ian J Goodfellow and Samy Bengio  “Adversarial examples in the physical world”  In _Artificial intelligence safety and security_ ChapmanHall/CRC, 2018, pp. 99–112
  * [KI22] Sean Kulinski and David I Inouye  “Towards Explaining Distribution Shifts”  In _arXiv preprint arXiv:2210.10275_ , 2022
  * [Kim+21] Daehee Kim et al.  “Selfreg: Self-supervised contrastive regularization for domain generalization”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 9619–9628
  * [Koh+21] Pang Wei Koh et al.  “Wilds: A benchmark of in-the-wild distribution shifts”  In _International Conference on Machine Learning_ , 2021, pp. 5637–5664  PMLR
  * [Li+18] Da Li, Yongxin Yang, Yi-Zhe Song and Timothy Hospedales  “Learning to generalize: Meta-learning for domain generalization”  In _Proceedings of the AAAI conference on artificial intelligence_ 32.1, 2018
  * [Li+21] Pan Li et al.  “A simple feature augmentation for domain generalization”  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 8886–8895
  * [Liu+21] Evan Z Liu et al.  “Just train twice: Improving group robustness without training group information”  In _International Conference on Machine Learning_ , 2021, pp. 6781–6792  PMLR
  * [LST18] Magnus Lundgren, Theresa Squatrito and Jonas Tallberg  “Stability and change in international policy-making: A punctuated equilibrium approach”  In _The Review of International Organizations_ 13 Springer, 2018, pp. 547–572
  * [Mot+17] Saeid Motiian, Quinn Jones, Seyed Iranmanesh and Gianfranco Doretto  “Few-shot adversarial domain adaptation”  In _Advances in neural information processing systems_ 30, 2017
  * [MST20] Ramaravind K Mothilal, Amit Sharma and Chenhao Tan  “Explaining machine learning classifiers through diverse counterfactual explanations”  In _Proceedings of the 2020 conference on fairness, accountability, and transparency_ , 2020, pp. 607–617
  * [PC+17] Gabriel Peyré and Marco Cuturi  “Computational optimal transport”  In _Center for Research in Economics and Statistics Working Papers_ , 2017
  * [Poy+20] Rafael Poyiadzi et al.  “FACE: feasible and actionable counterfactual explanations”  In _Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society_ , 2020, pp. 344–350
  * [Rat19] Shubham Rathi  “Generating counterfactual and contrastive explanations using SHAP”  In _arXiv preprint arXiv:1906.09293_ , 2019
  * [Rom+21] Robin Rombach et al.  “High-Resolution Image Synthesis with Latent Diffusion Models”, 2021  arXiv:2112.10752 [cs.CV]
  * [Sag+19] Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto and Percy Liang  “Distributionally Robust Neural Networks”  In _International Conference on Learning Representations_ , 2019
  * [Sag+20] Shiori Sagawa*, Pang Wei Koh*, Tatsunori B. Hashimoto and Percy Liang  “Distributionally Robust Neural Networks”  In _International Conference on Learning Representations_ , 2020
  * [Sai+] Kuniaki Saito, Yoshitaka Ushiku, Tatsuya Harada and Kate Saenko  “Adversarial Dropout Regularization”  In _International Conference on Learning Representations_
  * [SS20] Adarsh Subbaswamy and Suchi Saria  “From development to deployment: dataset shift, causality, and shift-stable models in health AI”  In _Biostatistics_ 21.2 Oxford University Press, 2020, pp. 345–352
  * [STM21] Shibani Santurkar, Dimitris Tsipras and Aleksander Madry  “BREEDS: Benchmarks for Subpopulation Shift”  In _International Conference on Learning Representations_ , 2021
  * [Wan+20] Xiao Wang et al.  “When pedestrian detection meets nighttime surveillance: A new benchmark”  In _Image_ 20000.30000, 2020, pp. 40000
  * [WMR17] Sandra Wachter, Brent Mittelstadt and Chris Russell  “Counterfactual explanations without opening the black box: Automated decisions and the GDPR”  In _Harv. JL & Tech._ 31 HeinOnline, 2017, pp. 841
  * [Yao+22] Huaxiu Yao et al.  “Improving out-of-distribution robustness via selective augmentation”  In _International Conference on Machine Learning_ , 2022, pp. 25407–25437  PMLR
  * [Zec+18] John R Zech et al.  “Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study”  In _PLoS medicine_ 15.11 Public Library of Science San Francisco, CA USA, 2018, pp. e1002683
  * [Zha+] Jingzhao Zhang et al.  “Coping with Label Shift via Distributionally Robust Optimisation”  In _International Conference on Learning Representations_
  * [Zha+20] Shanshan Zhao et al.  “Domain generalization via entropy regularization”  In _Advances in Neural Information Processing Systems_ 33, 2020, pp. 16096–16107

##  Appendix A Additional Framework Instantiations

###  A.1 Optimal Transport (OT)

Similar to $K$-cluster Transport ($K$-cluster) [KI22], Optimal Transport (OT)
finds the moving distance $\theta(x)$ for shift explanations directly. In the
next two sub-sections, we discuss how to instantiate $M(x;\theta)$ and
$L(\theta;M,P,Q)$ for OT within our framework.

Mapping function for OT. In OT, the mapping is almost the same as that for
$K$-cluster except that the moving distance $\theta$ now depends on each
individual sample, $x$, from the source. Therefore, the counterfactual mapping
$M(x;\theta)$ can be written as $M(x_{i};\theta_{i})=x_{i}+\theta_{i}$ for
every $x_{i}\in P$.

Objective function for OT. The objective function for OT is exactly the same
as that for $K$-cluster which is the PE metric. The optimization now results
in learning $\theta=\\{\theta_{1},\dots,\theta_{|P|}\\}$, or a separate moving
distance for every source sample such that the PercentExplained is maximized.

###  A.2 DiCE

For vanilla counterfactual explanation methods such as DiCE, model behavior
for a given sample $x$ is explained. To construct such explanations, these
methods perform counterfactual modifications to $x$ such that the model
prediction changes. We adapt these methods to construct a surrogate shift
explanation by finding counterfactual examples for models that classify
between source and target distributions. In this subsection, we investigate
how general methods for finding counterfactual examples can be adapted to fit
within our framework. We take DiCE as an example to describe how to
instantiate $M(x;\theta)$ and $L(\theta;M,P,Q)$ for these methods.

Mapping function for DiCE. The counterfactual examples produced by DiCE depend
on a given model (parameterized by $\theta$). As a consequence, the mapping
function for DiCE, $M(x,\theta)$, is represented as
$M(x,\theta)=x+f(x;\theta)$. Let $h$ denote the fixed model which classifies
between the source and target data. The moving distance, $f(x;\theta)$, used
in the counterfactual explanation relies on this model, $h$, that DiCE is used
to explain.

Objective function for DiCE. As indicated above, it is essential to obtain the
parameter $\theta$ to learn the shift explanation. Since the model, $h$,
discriminates between the source data, $P$, and the target data, $Q$, we
optimize the following objective function for DiCE, in which all source
samples and target samples are labeled as 0 and 1 respectively:

$\displaystyle\begin{split}&\operatorname*{arg\,min}_{\theta}L_{\text{DiCE}}(\theta;M,P,Q)\\\
&=\operatorname*{arg\,min}_{\theta}\sum_{x,y\in
D}\ell(h(x;\theta),y).\end{split}$ (8)

In the above formula, the loss $\ell(\cdot)$ represents the Cross Entropy loss
and $h$ denotes the model which classifies between the source and target data
$D=\\{(x,0):x\in P\\}\cup\\{(x,1):x\in Q\\}$. Note that the above loss
function is an instantiation of the abstract objective function, $L$, used in
Equation (5). This optimization leads to learning a $\theta$ which is the
model parameter for the classifier between the source and target. Once we have
learned the model parameter for the model $h$ to be explained with DiCE, we
derive the moving distance as

$\displaystyle\begin{split}&f(x;\theta)=\text{argmin}_{\delta_{x}}\text{dist}(x,x+\delta_{x})\\\
&\textit{s.t.}\ h(x+\delta_{x};\theta)=1.\end{split}$ (9)

For any $x\in P$ the moving distance $f(x;\theta)$ is found such that it is a
minimal change to $x$ which results in the previously learned classifier, $h$,
classifying the modified sample as a target sample.

##  Appendix B Details on Feasibility and Robustness

(a) % Feasible

(b) Robustness

Figure 5: Visualizations of feasibility and robustness. % Feasible is shown in
Figure 5(a) and it measures the percent of samples which are mapped by
$G(x;\theta)$ to a sample with the same group as the original sample.
Robustness is shown in Figure 5(b) and it measures how small perturbations on
the source data distribution change shift explanation.

Feasibility and robustness are defined in Equation 6 and 7 respectively, but
here we give a visual example of each. A concrete example for calculating
feasibility is shown in Figure 5(a). The source cluster of four males and one
female becomes three males and two females from the mapping, so feasibility is
$\frac{4}{6}=0.667$.

Similarly, we calculate robustness for an example in Figure 5(b). Suppose
there are two clusters in the source distribution and the target distribution
respectively, and each cluster consists of a single sample. After applying
$K$-cluster transport, the moving distance $\theta$ from the source to the
target can be interpreted as “increasing the age by 1 and flipping the sex
attribute”. After perturbing the sex attribute of one source sample from 1 to
0, the magnitude of changes on the source data distribution is
$\|P-P(\epsilon)\|_{2}=1$. This produces a new moving distance
$\theta(\epsilon)$, which is interpreted as “increasing the age by 1 and only
flipping the sex attribute of the first source sample”. By leveraging Equation
(7), the Robustness measure $\Omega$ for this example is
$\frac{\|M(P;\theta)-M(P(\epsilon);\theta(\epsilon))\|_{2}}{\|P-P(\epsilon)\|_{2}}\approx\frac{\|\theta-\theta(\epsilon)\|_{2}}{\|P-P(\epsilon)\|_{2}}=\sqrt{5}$.

The details for how we produce a perturbation, calculate worst-case
robustness, and perform the robustness experiment are given in Appendix D.5.

##  Appendix C Datasets

The tabular, language, and image datasets that we use in the experiments are
described in this section.

###  C.1 Tabular data

Dataset overview. The Adult dataset and the Breast Cancer dataset are standard
tabular datasets from the UCI Machine Learning Repository [DG17]. The Adult
dataset consists of 48,842 samples with categorical and integer features from
census data. The typical task is to predict whether income exceeds $50K per
year. The Breast Cancer dataset contains 569 samples with 10 real-valued
features relating to an imaged cell. This dataset is similarly used for binary
classification between the classes of benign and malignant tumors.

Distribution shift setup. For both the Adult and Breast Cancer datasets, we
match the setup by [KI22] and consider distribution shift between the
different class labels: above 50k and below 50K for Adult, and benign and
malignant for Breast Cancer.

Sub-population setup. For the Adult dataset, we use the existing demographic
feature of “male” to define two groups. For the Breast Cancer data, we define
groups by thresholding on a new attribute which is calculated by using “cell
radius” and “cell area” attributes (see Appendix D for details). This leads to
3 groups in total.

###  C.2 Language data

Dataset overview. The Civil Comments dataset [Bor+19] is used for our language
application. This dataset targets predicting the toxicity of up to 2 million
public comments and it additionally contains annotations of demographic
categories including gender, race, and religion of the authors of each
comment. This dataset is a part of the WILDS [Koh+21] distribution shift
benchmarks and it is used to benchmark subpopulation shift. Subpopulation
shift occurs when the proportions of samples from different demographic
categories changes between the source and target.

Distribution shift, sub-population and featurization setup. We build a
distribution shift setting by splitting the Civil Comments dataset into toxic
and non-toxic text as the source and target respectively as done by [KI22].
After balancing the size of this split, there are 4,437 samples in each of the
source and target. The groups are defined by samples with and without the
“female” demographic feature. The interpretable features for this data are
defined by the bag-of-words representation for each sample. We limit the bag-
of-words to 50 words which helps avoid model overfitting in DiCE based on our
observations.

###  C.3 Image data

Dataset overview. We use BREEDS [STM21], which uses the wordnet class
hierarchy to create subsets of ImageNet [Den+09] for sub-population shift
studies. We use BREEDS to create a subset of the ImageNet validation set which
consists of 50,000 images.

Distribution shift setup. In BREEDS, We start at the subtree under “mammal” in
ImageNet’s wordnet hierarchy and select three ImageNet classes under the
superclass “rodent/gnawer” and three classes under the superclass
“ungulate/hooved mammal” for both the source and target. These three classes
for each superclass are chosen in an adversarial way according to [STM21] to
increase the level of subpopulation shift. In total, this subset consists of
298 samples in each of the source and target.

Featurization and sub-population setup. As described in Section 3.3.2,
features are extracted by using an img-to-text model and then treating the
caption as a bag-of-words representation. We use a total of 50 words in the
bag-of-words as features. Finally, groups are defined by the superclasses
“rodent/gnawer” and “ungulate/hooved mammal” to encourage an explanation which
does not map rodents to hooved mammals. This grouping allows us to define an
infeasible explanation as one which maps rodents to hooved mammals or vice
versa.

Note that for the Civil Comments dataset, the groups are determined by extra
annotations, which are not available for the mapped samples produced by the
shift explanations. To determine the group assignments of the mapped source
sample, we leverage the group annotation of the mapped sample’s closest target
sample as an approximated annotation.

##  Appendix D Datasets and Hyperparameters for Experiments

###  D.1 Tabular data

All categorical features in the Adult data are one-hot encoded resulting in a
total of 35 features. We balance the size of both source and target
distribution in which results in a total of 15,682 samples for the Adult data
and 424 samples for the Breast dataset. Finally, we scale the feature values
of both datasets to range from 0 to 1.

The new meta-feature that is used for grouping the Breast dataset is
calculated by the expression

$\frac{\text{Avg. cell radius}^{2}}{\text{Avg. cell area}},$

and then we group the data by thresholding on this new meta-feature. To find a
good threshold, we compute the meta-feature for the entire source and target
dataset and get the first and third quartiles. Thus, we create three groups:
samples with meta-feature value below the first quartile, between the first
and third quartile, or above the third quartile.

When learning vanilla and GSE $K$-cluster explanations for the Adult data, we
use 10 clusters and optimize for 100 iterations using a learning rate of 10.0.
For the Breast dataset, we use 4 clusters, 100 iterations, and a learning rate
of 10.0. For OT explanations, we use a learning rate of 0.05 for Adult and 1.0
for Breast and use 100 iterations of training for both. For DiCE, we use a
neural network with a single hidden layer of size 16 as the source vs. target
discriminator in the DiCE experiments. This model is trained for 100 epochs
with a learning rate of 0.05 and weight decay 0.0001 for the Adult data. This
network is trained for 500 epochs at learning rate 0.2 and weight decay 0.0001
for the Breast Cancer data. For GSE, we train the neural network using group
DRO [Sag+20] with the same hyperparameters used in the regular training
procedure.

###  D.2 Language data

For the $K$-cluster experiments, we use 4 clusters and optimize for 200
iterations using a learning rate of 20. For OT explanations, we optimize for
200 iterations using a learning rate of 0.1. Finally, for the DiCE
explanations we first train a logistic regression classifier for classifying
the source and target samples using 1000 epochs with learning rate of 0.5 and
weight decay of 0.0001. For GSE with DiCE, we train this logistic regression
classifier using group DRO with the same hyperparameters used in the regular
training procedure.

###  D.3 Image data

For $K$-cluster explanations, we use 5 clusters and optimize for 100
iterations using a learning rate of 150.0. For OT explanations, we optimize
for 100 iterations using a learning rate of 0.5. Finally, for the DiCE
explanations we first train a logistic regression classifier for classifying
the source and target samples using 100 epochs with learning rate of 0.1 and
weight decay of 0.0001, and we use group DRO to train this classifier for GSE.

###  D.4 Framework hyperparameters

For all experiments, we leverage group DRO loss [Sag+20] for the aggregate
function $F$ in Equation 5. We also experimented with $F(X)=\max X$ and
$F(X)=\sum_{x\in X}x$ with $\lambda=0.1$. Note that for the latter $F(X)$, it
is only applicable to the loss function $L$ which does not preserve the
addition operation over groups, such as PE. Otherwise, Equation (5) could be
rewritten as $\min_{\theta}\left((1+\lambda)\cdot L(\theta;M,P{},Q{})\right)$,
which is not group-aware loss.

###  D.5 Robustness experiment

To compute the robustness metric, we use a random small perturbation to the
source distribution. To create this perturbation, we randomly select 75% of
the features and perturb 1% of the feature values for each of these features.
The manner in which we perturb this 1% of the feature values depends on the
type of the feature. For real valued features, we find the standard deviation
of the feature value for the current feature we are perturbing and we randomly
either add or subtract $0.05\cdot\text{stdev}$ to 1% of the feature values.
For integer features, we randomly either add or subtract 1 to 1% of the
feature values. Finally, for boolean features, we randomly either flip the
label of 1% of the True feature values or 1% of the False features values. For
categorical features, we first convert the categories to integers such that
each category is given an integer from 0 to $K$-1 where $K$ is the number of
categories. This allows us to generate a perturbation for categorical features
in the same way as for integer features.

We use the same hyperparameters as above for learning each shift explanation
on the perturbed distribution. To speed up the experiments, we first train the
shift explanation on the original source distribution and then initialize the
parameters of the shift distribution with the parameters learned from the
original source distribution when learning the shift explanation for the
perturbed distribution.

For computing the robustness metric, we use three random perturbations as
described above and average the robustness over the three runs. To compute
worst-case robustness, we calculate robustness from 100 random perturbations
and take the worst (highest) value of robustness from the 100 trials. Since
each calculation of robustness requires learning a shift explanation using the
vanilla method and GSE, this experiment is time consuming, so we don’t report
error bars for the worst-case robustness.

###  D.6 Compute details

For all experiments, we use a local server with four Nvidia 2080 Ti GPUs and
80 Intel Xeon Gold 6248 CPUs. Each experiment required around 2 GB of GPU
memory.

##  Appendix E Additional Qualitative Analysis

##  Appendix F Experiments Without Group Labels

It is possible that group labels are not always available for a dataset, but
we can still use either pretrained models to extract attributes to use for
defining groups or use unsupervised methods for grouping the data. We perform
an experiment on the language data to show that our group-aware method is
still applicable even without group supervision.

To get groups for the language data, we cluster the sentence embeddings of our
source and target data. The sentence embeddings are from a state-of-the-art
sentence embedding model, all-mpnet-v2—111https://huggingface.co/sentence-
transformers/all-mpnet-base-v2, and we use K-means clustering with 10 clusters
to get 10 groups for the source and target. Experimental results are shown in
Table 3, and we see the same trends as for the experiments with specified
groups. In particular, our group-aware explanation always results in higher
worst-group PE and % Feasible than the regular explanation. The most
significant improvement in WG-PE is seen for the OT explanation with a change
from 63.07% to 93.48%. Interestingly, we also see that our group-aware
explanation has slightly improved overall PE over the vanilla DiCE and
$K$-cluster explanations.

Table 3: Comparison of distribution shift explanation methods on language data without groups given. Method | PE | WG-PE | % Feas. | Robustness | Worst-case Robustness
---|---|---|---|---|---
DiCE | $1.08\pm 0.1$ | $-5.84\pm 0.5$ |  54.50 $\pm$ 0.82 |  7.82 $\pm$ 0.01 | 7.93
GSE DiCE | 14.32 $\pm$ 0.9 | 7.52 $\pm$ 0.5 | 56.33 $\pm$ 0.85 | 7.71 $\pm$ 0.08 | 7.92
$K$-cluster |  5.19 $\pm$ 1.75 |  2.64 $\pm$ 0.34 |  66.00 $\pm$ 0.71 | 3.00 $\pm$ 0.20 | 4.60
GSE $K$-cluster | 5.72 $\pm$ 0.88 | 3.79 $\pm$ 0.27 | 67.00 $\pm$ 0.41 |  3.02 $\pm$ 0.05 | 3.21
OT | 99.89 $\pm$ 0.00 |  63.07 $\pm$ 2.97 |  55.17 $\pm$ 4.11 | 1.00 $\pm$ 0.02 | 1.05
GSE OT |  98.34 $\pm$ 0.28 | 93.48 $\pm$ 0.26 | 84.67 $\pm$ 0.24 |  1.06 $\pm$ 0.03 | 1.14

##  Appendix G Results for OT and DiCE Shift Explanations

The full results for tabular data, Civil Comments, and ImageNet are given in
Table 4, 5, and 6 respectively. With DiCE and OT shift explanations, we see
the same trends as previously mentioned in relation to $K$-cluster
explanations. In particular, WG-PE is always improved by GSE, and feasibility
and robustness are improved in most cases.

Table 4: Comparison of distribution shift explanation methods on tabular
datasets.

Method | PE | WG-PE | % Feas | Robustness | Worst-case Robustness
---|---|---|---|---|---
Vanilla DiCE |  2.25 $\pm$ 0.29 |  2.25 $\pm$ 0.24 |  100.0 $\pm$ 0.00 |  23.74 $\pm$ 4.05 | 41.58
GSE DiCE | 26.02 $\pm$ 3.00 | 21.69 $\pm$ 4.77 |  100.0 $\pm$ 1.52 | 22.34 $\pm$ 1.29 | 34.56
Vanilla OT | 95.56 $\pm$ 0.25 |  61.23 $\pm$ 1.06 |  81.00 $\pm$ 0.36 |  54.86 $\pm$ 3.46 | 71.65
GSE OT |  76.55 $\pm$ 0.19 | 76.55 $\pm$ 0.19 | 100.0 $\pm$ 0.00 | 47.01 $\pm$ 3.40 | 65.95
(a) Adult data

Method | PE | WG-PE | % Feas | Robustness | Worst-case Robustness
---|---|---|---|---|---
Vanilla DiCE |  29.6 $\pm$ 2.43 |  25.16 $\pm$ 1.00 |  25.94 $\pm$ 0.00 |  201.00 $\pm$ 21.56 | 566483.01
GSE DiCE | 38.21 $\pm$ 1.58 | 33.48 $\pm$ 0.40 | 27.20 $\pm$ 1.46 | 189.93 $\pm$ 28.37 | 566201.54
Vanilla OT | 93.10 $\pm$ 0.06 |  85.79 $\pm$ 0.12 | 66.98 $\pm$ 0.66 |  190.02 $\pm$ 191.10 | 762966.00
GSE OT |  90.54 $\pm$ 0.02 | 89.10 $\pm$ 0.07 |  61.48 $\pm$ 1.24 | 149.13 $\pm$ 160.04 | 819351.59
(b) Breast data

Table 5: Comparison of distribution shift explanation methods on language data. Method | PE | WG-PE | % Feas. | Robustness | Worst-case Robustness
---|---|---|---|---|---
DiCE |  2.75 $\pm$ 0.19 |  1.11 $\pm$ 0.30 |  63.33 $\pm$ 1.25 |  5.28 $\pm$ 1.72 | 6.75
GSE DiCE | 19.29 $\pm$ 0.80 | 15.12 $\pm$ 2.47 | 64.67 $\pm$ 0.62 | 1.72 $\pm$ 0.06 | 3.40
OT | 99.89 $\pm$ 0.09 |  74.67 $\pm$ 1.19 |  49.83 $\pm$ 3.86 |  4.30 $\pm$ 0.24 | 5.81
GSE OT |  94.62 $\pm$ 1.12 | 94.62 $\pm$ 1.12 | 60.67 $\pm$ 0.24 | 4.05 $\pm$ 0.30 | 5.40
Table 6: Comparison of domain shift explanation methods on image data. Method | PE | WG-PE | % Feas. | Robustness | Worst-case Robustness
---|---|---|---|---|---
DiCE |  -1.09 $\pm$ 1.54 |  -17.25 $\pm$ 2.55 | 50.39 $\pm$ 0.42 | 5.08 $\pm$ 0.36 | 16.24
GSE DiCE | 0.19 $\pm$ 1.63 | -15.27 $\pm$ 3.08 |  49.94 $\pm$ 0.32 |  6.39 $\pm$ 0.64 | 15.73
OT |  7.18 $\pm$ 1.04 |  -17.30 $\pm$ 2.74 |  36.12 $\pm$ 0.55 |  18.77 $\pm$ 2.56 | 24.33
GSE OT | 12.81 $\pm$ 1.34 | -14.70 $\pm$ 2.50 | 48.16 $\pm$ 0.72 | 7.76 $\pm$ 1.73 | 22.79

##  Appendix H Additional Related Work

Domain generalization and adaptation. Common solutions for dealing with
distribution shift include domain generalization and domain adaptation. Domain
generalization assumes that the target distribution is unknown and the goal is
to improve model robustness to unseen out-of-distribution data. In contrast,
domain adaptation aims to adapt a model learned on the source distributions to
some known target distribution. But similar techniques were proposed for
domain generalization and domain adaptation, including augmenting training
data [Li+21, Yao+22, Mot+17], adding regularization terms to the loss function
[Zha+20, BSC18, Kim+21, CS19, Sai+] and meta-learning [Li+18, Mot+17]. There
are also many real world distribution shift datasets such as the iWildCam
dataset [Bee+21] and the Camelyon17 dataset of [Ban+18] as part of the WILDS
datasets [Koh+21].

##  Appendix I Limitations and Societal Impacts

GSE explanations are only as good as the underlying shift explanation method.
For instance, $K$-cluster transport can result in weak explanations that
minimally reduce the Wasserstein distance between the source and target
distributions if too few clusters are used (i.e. $K$ is chosen too small). On
the other hand, the Optimal Transport explanation that we found reduced the
Wasserstein distance the most, is not very interpretable since each source
sample can be mapped differently. This results in the explanation being
interpretable only on a per-sample basis. Improved interpretability of shift
explanations is an area for future work.

In addition to interpretability of the explanation, our shift explanations for
image and language data rely on interpretable feature extraction methods and
methods for counterfactual modification based on changes to the features as
described in Section 3.3.2. We designed a system for interpretable feature
extraction which uses a bag-of-words feature representation, but this method
looses the context that words are used in and it is difficult to make
counterfactual modifications. Creating disentangled embedding spaces for
interpretable embeddings that can also be used for counterfactual modification
is an area of active research, but there is still work left to make these
approaches more general.

We also found that GSE is sensitive to the choice of groups. Even though
unsupervised methods can be used to select groups as shown in Appendix F,
future work can look at how to best select or design groups. For instance, it
may be the case that we know of some groups, but we want the rest of the data
to be grouped appropriately.

Finally, while we evaluated the worst-case robustness, our method sometimes
results in worse worst-case robustness than the vanilla approach. This is
again due to the choice of groups. Future work should investigate how to
extend group robustness to worst-case group robustness of shift explanations
so that a bad choice of groups does not negatively impact robustness.

Explanations which look plausible but are actually wrong can be harmful. This
creates the illusion of understanding, and this can have serious downstream
implications especially if policies are constructed from a shift explanation.
With this work we hope to uncover some properties that a good shift
explanation should have and design metrics and learning procedures based on
group robustness to rectify these issues.