(wacv) Package wacv Warning: Incorrect font size specified - WACV requires
10-point fonts. Please load document class ‘article’ with ‘10pt’ option (wacv)
Package wacv Warning: Incorrect paper size - WACV uses paper size ‘letter’.
Please load document class ‘article’ with ‘letterpaper’ option

# An Empirical Investigation into Benchmarking Model Multiplicity for
Trustworthy Machine Learning: A Case Study on Image Classification

Prakhar Ganesh
Mila, Quebec AI Insitute
<EMAIL_ADDRESS>
###### Abstract

Deep learning models have proven to be highly successful. Yet, their over-
parameterization gives rise to model multiplicity, a phenomenon in which
multiple models achieve similar performance but exhibit distinct underlying
behaviours. This multiplicity presents a significant challenge and
necessitates additional specifications in model selection to prevent unexpe
cted failures during deployment. While prior studies have examined these
concerns, they focus on individual metrics in isolation, making it difficult
to obtain a comprehensive view of multiplicity in trustworthy machine
learning. Our work stands out by offering a one-stop empirical benchmark of
multiplicity across various dimensions of model design and its impact on a
diverse set of trustworthy metrics.

In this work, we establish a consistent language for studying model
multiplicity by translating several trustworthy metrics into accuracy under
appropriate interventions. We also develop a framework, which we call
multiplicity sheets, to benchmark multiplicity in various scenarios. We
demonstrate the advantages of our setup through a case study in image
classification and provide actionable insights into the impact and trends of
different hyperparameters on model multiplicity. Finally, we show that
multiplicity persists in deep learning models even after enforcing additional
specifications during model selection, highlighting the severity of over-
parameterization. The concerns of under-specification thus remain, and we seek
to promote a more comprehensive discussion of multiplicity in trustworthy
machine learning.

##  1 Introduction

Deep learning has experienced a remarkable rise in recent years [32, 29, 30],
with highly sophisticated and over-parameterized models leading the way [11,
31]. Consequently, these cutting-edge models find application across a diverse
set of domains, including image processing [39], natural language [21],
healthcare [14], finance [2], judiciary systems [42], and more, showcasing
their versatility and potential impact. However, the increasing deployment of
these models has sparked concerns about their trustworthiness. To confront
these issues head-on, the global community has embraced a range of trustworthy
machine learning practices and metrics [19, 38]. These efforts are geared
towards ensuring that these models are not only accurate in their predictions,
but are also fair to various groups in the dataset [3, 25], robust to
distribution shifts [36], maintain the privacy of the individuals whose data
was collected [13], and secure against adversarial attacks [7]. These metrics
collectively strive to make deep learning deployments more reliable, fostering
trust and acceptance in its widespread applications.

Alongside the discussion of trustworthy ML, the presence of multiplicity in
deep learning has emerged as a significant concern yet a welcome opportunity
[5, 10]. Model multiplicity is the existence of multiple high-performing
models that achieve similar accuracy on a given task but can display diverse
predictive behaviours due to varying decision boundaries and underlying
learned functions. Model multiplicity is the result of an under-specified and
over-parameterized training regime, and can be affected by design choices like
model architectures, hyperparameters, training configurations, or even
arbitrary choices like the randomness in training.

Model multiplicity in deep learning has significant implications. For
instance, it has been shown that changes in the training configuration can
lead to considerable variations in the biases present in a model [15, 27].
Deploying such models without considering the impact of multiplicity can
result in the unintentional deployment of unfair models in real-world
applications. Conversely, if we manage multiplicity with appropriate
constraints, it presents an opportunity to deploy fairer models without
compromising its utility. Thus, addressing the challenges of model
multiplicity is a crucial step towards creating trustworthy systems.

Existing literature on investigating model multiplicity is limited to
specialized settings that do not generalize. For instance, Somepalli _et al_.
[35] provides an empirical quantification of similarity in the decision
boundary of two models. However, the similarity of decision boundaries may not
necessarily provide any information about its trustworthiness. Models with
significantly different decision boundaries can still provide similar
accuracy, fairness, robustness, security, and privacy. Similarly, Ganesh _et
al_. [15] investigates the impact of random seeds on fairness, but their
discussion focuses on model predictions, and thus may not extend to other
trustworthy metrics like robustness, security, or privacy. Furthermore, these
investigations are not directly comparable to each other. For example, a
$70\%$ agreement between the decision boundaries of two models as defined by
Somepalli _et al_. [35] has no comparative value to a $10\%$ gap in equalized
odds (a fairness metric) between the same set of models [15]. Thus, while
these works provide a deeper investigation into a single metric in isolation,
they fail to provide a comprehensive view of the overall trends of
multiplicity.

In this paper, we address this gap by proposing a framework to measure
multiplicity that can not only dive deeper into multiplicity trends for a
single metric but also provide comparisons across different metrics. We start
by converting various trustworthy metrics to a common scale, which we refer to
as accuracy under intervention, facilitating the comparison of multiplicity
across different metrics. We then create multiplicity sheets that capture the
multiplicity of accuracy under intervention for each metric separately. To
illustrate our framework, we present an image classification case study that
compares multiplicity across model hyperparameters, random seeds, and
architecture choices, repeating the setup for various trustworthy ML metrics,
namely fairness, robustness, privacy, and security.

We end our discussion by presenting the results of combining various metrics
together to improve the model specification and reduce multiplicity. However,
despite following the recommendations of recent literature and providing
additional specifications using trustworthy metrics, our study reveals that
model multiplicity can still create unforeseen failure cases. This highlights
the need for future research to gain a more holistic understanding of model
multiplicity.

#### Setting Expectations and Our Contributions:

Before delving into our contributions, it is essential to first clarify the
scope of our work. Our goal is not to present novel findings on the
multiplicity of any specific metric. In fact, we will revisit many existing
results in the literature during our case study. Rather, we seek to establish
a normative language to record model multiplicity that can be used to
highlight multiplicity trends across different metrics, thus providing an
overall picture of multiplicity in deep learning.

More specifically, our contributions are:

  * •

We propose a standardized framework to measure and study model multiplicity in
deep learning.

    * –

We introduce a new class of metrics called accuracy under intervention. We
showcase techniques to convert any metric into accuracy using appropriate
interventions, thus providing a common scale of comparison.

    * –

We suggest using multiplicity sheets, a comprehensive yet compact method to
record and study model multiplicity for any target metric.

  * •

We present a case study of model multiplicity in image classification, by
providing an empirical benchmark to highlight the advantages of our framework.

    * –

We take an all-encompassing view of model multiplicity and its impact on
trustworthy ML by comparing multiplicity across fairness, robustness, privacy,
and security.

    * –

We study the influence of various axes of model variations on multiplicity,
including model architecture, training randomness, and hyperparameter choices.

  * •

We combine several trustworthy metric specifications to challenge over-
parameterization and assess its impact on multiplicity. Despite this, we see
persistent multiplicity on trustworthy issues not seen during model selection,
underscoring the need for better safeguards against multiplicity when
deploying models in the real world.

92.8592.8592.3792.8992.6492.6092.4792.5192.4393.1792.9192.6492.6092.9892.660.490.300.080.530.380.700.460.30Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

92.8592.8192.1892.8992.9892.6292.4792.6692.6093.1793.0892.6892.6092.3292.070.680.360.190.490.530.700.760.61Batch
Size$\Delta_{max}$128256640

92.8592.5192.8992.8992.4792.8793.1792.7992.6092.870.340.000.400.380.270.700.38Augmentation$\Delta_{max}$RandTrivial

92.8592.6092.8992.5592.4792.6893.1792.8792.6092.450.250.340.210.300.150.700.42Optimizer$\Delta_{max}$SGDAdam

92.8592.2292.1392.8992.4992.0592.4792.2492.4593.1792.0592.2092.6092.3092.180.720.840.231.120.420.700.440.40Architecture$\Delta_{max}$R18R50WR50

$92$$94$

$0$$2$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD; Architecture R18

Metric: Accuracy Dataset: UTKFace g $\Delta_{max}^{all}$: 1.12

Figure 1: Multiplicity sheet for Accuracy on UTKFace dataset. R18/50:
ResNet-18/50; WR50: WideResNet-50x2. This multiplicity sheet records accuracy
scores across different hyperparameter choices and random seeds, representing
the first level of readability without any loss of information. It then
aggregates multiplicity by measuring $\Delta_{max}$ across random seeds for
each hyperparameter and across hyperparameter choices for each random seed.
This is the second level of readability, vital for extracting multiplicity
trends. For instance, by studying the $\Delta_{max}$ values, we see the equal
importance of both random seeds and hyperparameter choices on accuracy
multiplicity. Finally, we aggregate the overall multiplicity
$\Delta_{max}^{all}$, i.e., the third level of readability, condensing
accuracy multiplicity for UTKFace into a single value.

##  2 Measuring Multiplicity

We will start by discussing our framework to study multiplicity. We introduce
the concept of accuracy under intervention to translate any metric into an
accuracy metric, followed by our proposal to use multiplicity sheets to record
and compare the said accuracy under intervention.

###  2.1 Accuracy Under Intervention

We want to establish a standardized way to measure model multiplicity that
would allow easy comparison across different scenarios. However, measuring
multiplicity for various trustworthy objectives relies on vastly different
metrics, making it a complex task. For instance, comparing accuracy
multiplicity (difference in accuracy) with security multiplicity (difference
in minimum adversarial distance to flip the label), is not straightforward.
These two metrics are not comparable since they are based on different
factors, one on performance and the other on distance.

We need a method that can translate these metrics to a common scale for a fair
comparison. To achieve this, we propose converting each metric into accuracy
through appropriate interventions. Simply put, we want to measure model
accuracy under a well-designed intervention that represents a proxy for our
original trustworthy metric. For instance, when testing the security of a
model against adversarial attacks, instead of measuring the minimum
adversarial distance to flip the label, we can measure the model accuracy
under a fixed adversarial distance budget. Once a metric is translated to
accuracy under such an intervention, we can now compare them directly to each
other and get a comprehensive understanding of the multiplicity. More details
on the specific interventions for each metric are present in Section 3.

###  2.2 Multiplicity Sheets

After converting every metric to accuracy under intervention, we propose a
method for recording these values to facilitate easy comparison and
visualization. We want a method that can provide both summaries for a quicker
scan and detailed results for a more in-depth analysis. To achieve this, we
create multiplicity sheets, a straightforward and highly intuitive approach to
documenting multiplicity.

A multiplicity sheet is a collection of tables, where each table compares two
axes of multiplicity. The information in our multiplicity sheets has three
levels of readability. The first level shows the raw metric scores, in this
case, accuracy under intervention, ensuring no loss of information. The second
level aggregates multiplicity across each axis in every table by taking the
maximum difference in scores denoted by $\Delta_{max}$. This allows easy
visualization of various trends and the influence of different hyperparameters
on model multiplicity. Finally, the third level further aggregates the
complete multiplicity sheet by taking the maximum difference across all raw
metric scores to get a single value representing the overall multiplicity of
the given metric, denoted as $\Delta_{max}^{all}$. Given that we use accuracy
under intervention for all metrics, $\Delta_{max}^{all}$ serves as a useful
measure to compare multiplicity across different metrics, i.e., different
multiplicity sheets.

An example of a multiplicity sheet can be seen in Fig. 1, where we record the
accuracy multiplicity on the UTKFace dataset under various training
configurations (more details in Section 3). Throughout our paper, we will
designate one axis of multiplicity in each table to be the random seeds. This
is to filter chance trends when comparing different hyperparameters by
balancing them against multiple runs with changing random seeds. It should be
noted that multiplicity sheets can be created for any metric, not just
accuracy under intervention. However, using accuracy under intervention allows
us to compare the multiplicity trends across different sheets, which wouldn’t
be possible with just any metric. We will now move to our case study to
highlight the benefits of our framework, while also providing an empirical
benchmark of multiplicity in image classification that can be directly useful
for researchers and practitioners.

##  3 Image Classification on UTKFace

To demonstrate the utility of our framework, we will perform a case study of
the model multiplicity in image classification on the UTKFace dataset. We
first outline our experimental setup, followed by a comprehensive discussion
of multiplicity in fairness, robustness, privacy, and security. To provide
diversity in experiments, we also perform a separate case study on the CIFAR10
dataset in Appendix LABEL:sec:;app_cifar10.

89.5688.1588.1589.0088.5889.5687.4588.4388.1590.4189.5689.0089.0089.4289.281.410.990.991.410.422.961.411.41Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

89.5687.3187.4589.0090.1388.4387.4589.4288.4390.4189.7087.8789.0088.0188.582.261.691.982.540.992.962.821.13Batch
Size$\Delta_{max}$128256640

89.5689.0089.0089.2887.4589.1490.4190.5589.0089.140.560.281.690.140.142.961.55Augmentation$\Delta_{max}$RandTrivial

89.5688.8689.0088.0187.4588.2990.4189.7089.0087.730.710.990.850.711.272.961.98Optimizer$\Delta_{max}$SGDAdam

89.5687.5988.0189.0088.7289.0087.4589.5689.0090.4187.4588.1589.0087.7387.451.980.282.122.961.552.962.121.55Architecture$\Delta_{max}$R18R50WR50

$87$$91$

$0$$4$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD; Architecture R18

Metric: Group Accuracy (Asian) Dataset: UTKFace g $\Delta_{max}^{all}$: 3.24

Figure 2: Multiplicity sheet for Group Accuracy (Asian) on UTKFace dataset.
R18/50: ResNet-18/50; WR50: WideResNet-50x2. Among various hyperparameter
choices, the batch size and architecture stand out in their influence on
fairness multiplicity. However, it is the variance across changing random
seeds that overshadows all other sources of multiplicity, making it the most
important factor for fairness during model selection. The overall fairness
multiplicity ($\Delta_{max}^{all}$) is also almost 3 times higher than the
accuracy multiplicity seen earlier.
RaceAgeAllAllWhiteAllBlackAllAsianAllIndianAllAll12-19All19-59All59-116Asian12-19Asian59-116$85$$90$$95$$100$Accuracy
(%) Figure 3: Distribution of fairness multiplicity (i.e., group accuracy)
across different intersections of racial and age groups in the UTKFace
dataset. Each distribution is a condensed representation of a multiplicity
sheet, containing the group accuracy of 65 independently trained models across
all axes of multiplicity described in Section 3.1. Different groups have
varying ranges of multiplicity, with a specially amplified variance from
intersectional groups, highlighting the concerns and opportunities of
multiplicity in fairness. Note: The minor perturbations along the x-axis for
any group are only present for enhanced visualization and do not convey any
additional signal.

###  3.1 Experiment Setup

#### Dataset

We will be studying the UTKFace dataset, containing facial images that have
been labelled according to their perceived gender, race, and age. We will
focus on the binary classification task of perceived gender. We split the
dataset into $80\%$ training and $20\%$ testing, and we maintain the same
split throughout our paper, i.e., we do not consider potential multiplicity
introduced by the train-test splits.

#### Training Details

By default, we train our models using a learning rate of 0.1, a batch size of
128, the data augmentation RandAugment [9], the SGD optimizer, and the
ResNet-18 architecture [16]. For a simpler analysis, all models are trained
from scratch, i.e., without the use of pre-trained weights. We use a single
random seed to control all forms of randomness in model training. We leave the
decoupled analysis of various sources of randomness for future work. Finally,
all models are trained with cross-entropy (CE) loss for 50 epochs, without any
early stopping.

#### Axes of Multiplicity

We will investigate the following different axes of multiplicity, (i) Learning
Rate:  $\\{0.1,0.05,0.01\\}$, (ii) Batch Size:  $\\{128,256,640\\}$, (iii)
Data Augmentation:  RandAugment [9] and TrivialAugment [26], (iv) Optimizer:
SGD and Adam, (v) Model Architecture: ResNet-18 [16], ResNet-50 [16], and
WideResNet-50x2 [43]. We also compare multiplicity across changing randomness
in model training.

###  3.2 Group Fairness

81.9981.5180.1081.3181.3479.9981.8681.8980.0581.1981.8180.0182.2481.3579.911.891.351.841.802.331.050.550.19Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

81.9981.2680.0781.3180.5980.4481.8681.3079.8381.1980.6180.4682.2481.3080.171.920.882.030.732.061.050.700.62Batch
Size$\Delta_{max}$128256640

81.9980.7681.3181.6381.8681.2181.1981.5482.2480.061.230.320.650.362.171.051.57Augmentation$\Delta_{max}$RandTrivial

81.9981.3981.3181.0581.8680.5881.1980.7182.2481.630.600.271.280.480.601.051.05Optimizer$\Delta_{max}$SGDAdam

81.9979.1078.7381.3180.5679.6181.8680.2679.2181.1980.2280.3582.2480.1980.673.261.712.650.972.051.051.461.95Architecture$\Delta_{max}$R18R50WR50

$79$$83$

$0$$4$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD; Architecture R18

Metric: OOD Accuracy (FairFace) Dataset: UTKFace g $\Delta_{max}^{all}$: 3.51

Figure 4: Multiplicity sheet for OOD Accuracy (FairFace) on UTKFace dataset.
R18/50: ResNet-18/50; WR50: WideResNet-50x2. The learning rate and batch size
both seem to noticeably influence the OOD robustness, while it’s the
architecture choice that dominates any other factor for robustness
multiplicity. The overall multiplicity ($\Delta_{max}^{all}$) is slightly
higher than the fairness multiplicity seen previously.

Group fairness is a measure of performance disparity between different
protected groups, rooted in concerns of algorithmic bias propagated from the
dataset to the model [8, 4, 44, 1]. Traditionally, group fairness is measured
as the difference in performance between different groups in the dataset. For
accuracy under intervention in our setup, we calculate the accuracy on the
minority group (which can also be extended to other groups). More
specifically, we consider racial labels for fairness and measure the accuracy
of the racial minority in the UTKFace dataset, i.e., Asians.

In Fig. 2, we present the multiplicity sheet for Group Accuracy (Asian). We
use the sheet to highlight the importance of random seeds in fairness and also
contrast it to the multiplicity sheet for Accuracy in Fig. 1. As can be seen
clearly from the $\Delta_{max}$ values of changing random seeds compared to
different hyperparameter choices, random seeds have the most significant
impact on fairness multiplicity. Moreover, we can also observe that the
overall fairness multiplicity ($\Delta_{max}^{all}=3.24$) is three times
higher than the accuracy multiplicity ($\Delta_{max}^{all}=1.12$). These
trends of fairness variance and the impact of random seed have been previously
noted in literature [15, 34], however here we show the ease with which they
can be spotted in our multiplicity sheets.

We repeat the experiment for various groups and plot the distribution of
fairness multiplicity across all axes of multiplicity, in Fig. 3, with groups
formed at the intersection of two different protected attributes, i.e., race
and age. Our findings reveal that the severity of fairness multiplicity is
even higher for intersectional groups. To put this into perspective, consider
selecting a model from the distribution of models in Fig. 3. While the choice
may only affect the overall accuracy in the range of $92.05\%$ to $93.17\%$,
it can significantly alter the accuracy for older Asian individuals, ranging
from $88.46\%$ to $98.08\%$. To sum up, our analysis clearly shows the
alarmingly high fairness variance, and the need to address this multiplicity
such that deep learning models treat diverse groups fairly during deployment.

###  3.3 Out-of-Distribution Robustness

Out-of-distribution (OOD) robustness refers to the ability of a machine
learning model to perform well on data points that are different from those it
was trained on. Models that lack OOD robustness might make unreliable or
incorrect predictions when faced with new, unfamiliar data, potentially
leading to undesirable outcomes after deployment. Traditionally, OOD
robustness is measured as the model’s performance on an OOD dataset. Since it
is already an accuracy metric, we do not perform any additional intervention
for robustness. More specifically, we simply use the model’s accuracy on the
FairFace dataset [18], a facial image dataset with a different distribution
than UTKFace, as the measure of OOD robustness multiplicity.

In Fig. 4, we present the multiplicity sheet for Accuracy on the FairFace
dataset. We see the impact of learning rate, batch size, and architecture on
robustness emerge from the multiplicity sheets, an unsurprising result based
on existing work on the benefits of smaller batch size, larger learning rate,
and higher risks of overfitting in bigger models [17, 24]. The range of
overall robustness multiplicity ($\Delta_{max}^{all}=3.51$) is of the same
range as fairness multiplicity, i.e. three times higher than accuracy
multiplicity. Thus, addressing multiplicity in OOD robustness is essential to
making sure the model doesn’t fail even under minor distribution shifts.

88.7687.4385.0088.5387.6284.9288.5587.5685.1988.4287.2085.6687.9687.3985.053.763.613.352.762.910.800.420.74Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

88.7686.5284.7188.5387.6883.4688.5587.6284.2288.4287.3485.5587.9686.9284.144.055.064.322.873.820.801.162.09Batch
Size$\Delta_{max}$128256640

88.7687.4788.5387.4388.5587.6888.4287.3987.9686.711.291.100.871.031.250.800.97Augmentation$\Delta_{max}$RandTrivial

88.7686.9088.5387.0388.5586.8888.4287.5187.9687.111.861.501.670.910.840.800.63Optimizer$\Delta_{max}$SGDAdam

88.7686.1686.2388.5387.2486.3788.5586.7886.5088.4286.7586.7587.9686.8487.012.602.152.051.671.120.801.080.78Architecture$\Delta_{max}$R18R50WR50

$84$$89$

$0$$6$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD; Architecture R18

Metric: Pert. Accuracy ($\lambda=5$) Dataset: UTKFace g $\Delta_{max}^{all}$:
5.30

Figure 5: Multiplicity sheet for Perturbation Accuracy ($\lambda=5$) on
UTKFace dataset. R18/50: ResNet-18/50; WR50: WideResNet-50x2. The random seed
has very little influence on the perturbation accuracy, while the default
hyperparameter choices are noticably dominant over other hyperparameters, with
the biggest drop caused by using a large batch size. The overall multiplicity
($\Delta_{max}^{all}$) is also quite high, five times larger than accuracy
multiplicity, but clearly dependent on the choice of the rate parameter
$\lambda$.

###  3.4 Differential Privacy

Deep learning models tend to memorize data points from their training dataset,
compromising the privacy of the individuals in the dataset. For instance, an
adversary with access to only the outputs of the model is capable of
extracting sensitive information from the model [6, 33]. To address this
issue, researchers often study differential privacy [12], which aims to make
models trained on datasets differing at exactly one data point
indistinguishable. One way to achieve this is by adding noise to the model’s
outputs. However, adding noise can also hurt the model’s performance, thus
creating a trade-off between privacy and accuracy. It is this very trade-off
that we will exploit to define our accuracy under intervention, i.e., we will
measure the accuracy of the model under output perturbations from an
exponential distribution with a fixed rate parameter $\lambda$, for privacy
multiplicity.

$\phi$12345$85$$90$$95$Rate Parameter ($\lambda$)Accuracy (%) Figure 6:
Distribution of privacy multiplicity (i.e., perturbation accuracy) across
different values of the rate parameter $\lambda$. The higher the rate
parameter, the larger the output perturbations, which in turn creates larger
drops in accuracy and a larger range of multiplicity. Refer to Fig. 3 for
further details on distribution visualization.

We present the multiplicity sheet for privacy by recording the Perturbation
Accuracy with $\lambda=5$ in Fig. 5. Interestingly, unlike other trustworthy
metrics, the random seed has minimal impact on the privacy multiplicity.
Instead, one hyperparameter choice along each axis is clearly the best when it
comes to the privacy-accuracy trade-off, in line with the existing literature
on practical tips for privacy [28]. Additionally, the overall privacy
multiplicity range ($\Delta_{max}^{all}=5.30$) is almost five times larger
than the accuracy multiplicity. These results are clearly dependent on the
rate parameter $\lambda$ used to calculate accuracy under intervention and to
emphasize this, we plot the distribution of privacy multiplicity for different
rate parameter values in Fig. 6. It is evident that choosing the right model
by accounting for privacy multiplicity is crucial to achieving better privacy-
utility trade-offs during inference, and this choice becomes even more
critical with a decrease in privacy budget (i.e., higher values of rate
parameter $\lambda$).

###  3.5 Security against Adversarial Attacks

84.7985.0083.2784.6984.4882.6885.1384.7582.8584.1884.8182.3284.1484.2482.371.732.002.282.491.880.990.760.95Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

84.7983.7283.7284.6985.6683.9585.1383.3683.5184.1882.9883.4084.1483.4283.721.081.711.771.200.720.992.680.55Batch
Size$\Delta_{max}$128256640

84.7983.9584.6783.9185.1384.7584.1882.9484.1483.590.840.780.381.250.550.991.81Augmentation$\Delta_{max}$RandTrivial

84.7983.6384.6983.4985.1383.7084.1883.5184.1483.891.161.201.430.680.250.990.40Optimizer$\Delta_{max}$SGDAdam

84.7984.7786.9484.6983.3484.7185.1383.3887.5884.1882.0584.4884.1484.0585.402.171.374.202.431.350.992.723.10Architecture$\Delta_{max}$R18R50WR50

$82$$88$

$0$$5$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD; Architecture R18

Metric: PGD Accuracy ($\delta=0.005$) Dataset: UTKFace g $\Delta_{max}^{all}$:
5.53

Figure 7: Multiplicity sheet for PGD Accuracy ($\delta=0.005$) on UTKFace
dataset. R18/50: ResNet-18/50; WR50: WideResNet-50x2. The architecture choice
stands out as the most influential factor in security multiplicity. The
overall multiplicity ($\Delta_{max}^{all}$) is five times larger than the
accuracy multiplicity, dependent on the choice of the adversarial distance
budget $\delta$.

Machine learning models are vulnerable to various attacks that can manipulate
the model’s behaviour to suit the attacker’s desires. One of the most common
adversarial attacks studied in literature is the perturbation-based attack
[37], which takes advantage of the brittle decision boundaries of deep
learning models. In this attack, the objective of the adversary is to perturb
the input image by a minimum amount that can incite adversarial outputs, while
keeping the perturbation imperceptible to the human eye. Instead of measuring
the minimum distance of the perturbed image to the original image (measured as
$L_{\infty}$), we will measure the accuracy under the intervention of a fixed
distance budget represented by $\delta$. Specifically, we use projected
gradient descent (PGD) [23] to progressively move out of the local minima
until we reach the given distance budget, and then measure the accuracy of the
model under these perturbations.

We present the multiplicity sheet for PGD Accuracy with $\delta=0.005$ in Fig.
7. The trends for security multiplicity are similar to the accuracy
multiplicity trends we observed previously, i.e., no single factor dominates
the multiplicity, except for architecture choice. Surprisingly, the larger
model ResNet-50 had a negative impact on security multiplicity, while the even
larger but wider model WideResNet-50x2 improved it, which contradicts previous
findings in literature [41] and raises interesting questions for future
research. Similar to privacy multiplicity, the overall multiplicity range
($\Delta_{max}^{all}=5.53$) for security is almost five times larger than the
accuracy multiplicity and depends on the adversarial distance budget $\delta$.
We plot the distribution of security multiplicity for different adversarial
distance budget values in Fig. 8. Our results have shown that a model’s
robustness to adversarial attacks suffers from severe multiplicity, and needs
to be addressed to provide robust models.

01e-45e-41e-35e-31e-2$70$$80$$90$Adversarial Distance Budget
($\delta$)Accuracy (%) Figure 8: Distribution of security multiplicity (i.e.,
accuracy after PGD) across different adversarial distance budget values
$\delta$. A higher budget corresponds to a more powerful adversary, which in
turn results in lower accuracy and higher security multiplicity. Refer to Fig.
3 for further details on distribution visualization.

##  4 Model Selection

Beforek=75%k=50%$92$$94$Accuracy (%)Group Accuracy (Age 59-116)

Beforek=75%k=50%$50$$60$$70$OOD Accuracy (CelebA)

Beforek=75%k=50%$60$$70$$80$$90$Input Pert. Accuracy ($\lambda=1$)

Beforek=75%k=50%$65$$70$$75$$80$PGD Accuracy ($\delta=0.01$)

Figure 9: Distribution of multiplicity across unforeseen metrics under various
degrees of model selection. The range of multiplicity across unforeseen
metrics might remain unchanged even after we provide additional specifications
for known trustworthy metrics, highlighting the severity of over-
parameterization and the need to address multiplicity beyond just a checklist
of metrics.

In our case study, we found significant multiplicity in various trustworthy
metrics that can hurt model deployment, if left unchecked. To address this
multiplicity, the literature suggests providing appropriate specifications
during model selection [5]. This involves imposing additional constraints
based on some chosen metrics, in our case the trustworthy metrics. For
example, one can measure the fairness scores of the model under different
hyperparameters and only choose the configurations with bias scores less than
some threshold. This ensures that unfair models are not selected.

These recommendations stem from the belief that implementing extra measures
during the selection of a model will decrease its variability, ensuring
predictable behaviour upon deployment. However, as we will demonstrate in this
section, over-parameterized models can still encounter unforeseen failure
cases during deployment, which are not simply solved with appropriate
specifications during model selection.

#### Model Specifications:

We first define the following criteria to simulate model selection. We choose
models that rank in the top k% of every metric under varying training
configurations. We assess fairness by measuring accuracy for the Asian racial
group, robustness by evaluating test performance on FairFace, privacy by
measuring accuracy under output perturbations with a rate parameter
$\lambda=5$, and security by measuring accuracy under PGD attacks with an
adversarial distance budget of $\delta=0.005$. Our approach ensures that we
only select models that meet the high standards for all four metrics mentioned
above.

#### Unforeseen Circumstances:

We now introduce a new set of metrics to account for situations that were not
previously considered in our specifications. To simplify the discussion, we
will just make minor adjustments to the model specifications and create these
’unforeseen circumstances’. To test fairness, we will measure group accuracy
for the age group $59-116$ instead of the Asian racial group. To test
robustness, we will evaluate the performance on the CelebA dataset [22]
instead of FairFace. To test privacy, we will measure accuracy under input
perturbations (with a rate parameter of $\lambda=1$) instead of output
perturbations. Finally, to test security, we will increase the distance budget
from $\delta=0.005$ to $\delta=0.01$, thus creating a stronger adversary.

In Fig. 9, we plot the distribution of multiplicity for all four unforeseen
metrics before any model selection, and then after model selection for
$k\%=75\%$ and $k\%=50\%$ respectively. We see a noticeable drop in unforeseen
fairness and security multiplicities while maintaining decent fairness and
security accuracy scores under intervention. However, we do not see this
improvement in unforeseen robustness or privacy multiplicity. That is, despite
the highly rigorous model selection on four different trustworthy ML metrics,
the overall range of multiplicity in these two unforeseen metrics remains the
same, and thus they will face the same issues during deployment. Clearly,
incorporating additional specifications while selecting models can only
provide limited assistance, leaving a substantial level of multiplicity that
cannot be managed in the same way. Thus, addressing multiplicity with a
checklist of trustworthy requirements is still likely to create models that
face the same risks of failure in unforeseen circumstances, emphasizing the
need for a more fundamental investigation into model multiplicity.

##  5 Related Work

Model multiplicity has been an active subject of research in the deep learning
literature, despite not being in the spotlight. Much of the related work in
multiplicity is indirect, often disguised as research on the impact of
hyperparameter choices or randomness on trustworthy ML [15, 34, 17, 24, 28].

Very few works in literature have focused solely on multiplicity. Black _et
al_. [5] provides a discussion on the opportunities and concerns of
multiplicity within the context of machine learning. However, their work is
highly qualitative and does not provide any framework to quantify and measure
multiplicity. On the other hand, D’Amour _et al_. [10] offer a more
quantitative perspective to underspecification in machine learning.
Nevertheless, their analysis is fragmented across different case studies and
does not provide a common language on multiplicity measurement that can be
adapted for future works on model multiplicity.

##  6 Conclusion and Future Work

In this paper, we contribute to the discussion of model multiplicity,
specifically in the context of image classification. By establishing a
consistent and comprehensive language for multiplicity, we have created a
foundation for more effective communication in the field. Our multiplicity
sheets offer an intuitive and structured approach to capturing the various
facets of multiplicity. Furthermore, through a detailed case study, we
demonstrated the practical implementation of our framework, shedding light on
the complexities that arise when dealing with model multiplicity. The insights
derived from the case study not only showcased the utility of our approach but
also unveiled intriguing trends within the multiplicity scores. Finally, we
show empirically that the challenge of model multiplicity cannot be simply
resolved by providing additional specifications or constraints.

While we emphasize a specific structure for multiplicity sheets in this paper,
it is important to acknowledge that further research is required to develop
more effective methods for recording multiplicity. Moreover, our
recommendation to use accuracy under intervention is primarily applicable to
classification tasks. Nevertheless, the challenge of model multiplicity is a
major issue in deep learning that goes beyond classification alone.
Consequently, it is imperative that the community engages in further
discussion on the topic of model multiplicity. We must shift away from
treating multiplicity as an auxiliary discussion and bring it to the forefront
to address potential unforeseen failures in real-world deployment scenarios
and create truly trustworthy systems.

## References

  * [1] Mohsen Abbasi, Sorelle A Friedler, Carlos Scheidegger, and Suresh Venkatasubramanian.  Fairness in representation: quantifying stereotyping as a representational harm.  In Proceedings of the 2019 SIAM International Conference on Data Mining, pages 801–809. SIAM, 2019.
  * [2] Shamima Ahmed, Muneer M Alshater, Anis El Ammari, and Helmi Hammami.  Artificial intelligence and machine learning in finance: A bibliometric review.  Research in International Business and Finance, 61:101646, 2022.
  * [3] Solon Barocas, Moritz Hardt, and Arvind Narayanan.  Fairness in machine learning.  Nips tutorial, 1:2017, 2017.
  * [4] Solon Barocas and Andrew D Selbst.  Big data’s disparate impact.  Calif. L. Rev., 104:671, 2016.
  * [5] Emily Black, Manish Raghavan, and Solon Barocas.  Model multiplicity: Opportunities, concerns, and solutions.  In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 850–863, 2022.
  * [6] Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al.  Extracting training data from large language models.  In 30th USENIX Security Symposium (USENIX Security 21), pages 2633–2650, 2021.
  * [7] Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay.  A survey on adversarial attacks and defences.  CAAI Transactions on Intelligence Technology, 6(1):25–45, 2021.
  * [8] Kate Crawford.  The hidden biases in big data.  Harvard business review, 1(4), 2013.
  * [9] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le.  Randaugment: Practical automated data augmentation with a reduced search space.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020.
  * [10] Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al.  Underspecification presents challenges for credibility in modern machine learning.  The Journal of Machine Learning Research, 23(1):10237–10297, 2022.
  * [11] Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdulmohsin, et al.  Scaling vision transformers to 22 billion parameters.  In International Conference on Machine Learning, pages 7480–7512. PMLR, 2023.
  * [12] Cynthia Dwork.  Differential privacy.  In International colloquium on automata, languages, and programming, pages 1–12. Springer, 2006.
  * [13] Cynthia Dwork, Aaron Roth, et al.  The algorithmic foundations of differential privacy.  Foundations and Trends® in Theoretical Computer Science, 9(3–4):211–407, 2014.
  * [14] Andre Esteva, Alexandre Robicquet, Bharath Ramsundar, Volodymyr Kuleshov, Mark DePristo, Katherine Chou, Claire Cui, Greg Corrado, Sebastian Thrun, and Jeff Dean.  A guide to deep learning in healthcare.  Nature medicine, 25(1):24–29, 2019.
  * [15] Prakhar Ganesh, Hongyan Chang, Martin Strobel, and Reza Shokri.  On the impact of machine learning randomness on group fairness.  In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 1789–1800, 2023.
  * [16] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [17] Stanislaw Jastrzebski, Zachary Kenton, Devansh Arpit, Nicolas Ballas, Asja Fischer, Yoshua Bengio, and Amos Storkey.  Three factors influencing minima in sgd.  arXiv preprint arXiv:1711.04623, 2017.
  * [18] Kimmo Karkkainen and Jungseock Joo.  Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 1548–1558, 2021.
  * [19] Michael Kearns and Aaron Roth.  The ethical algorithm: The science of socially aware algorithm design.  Oxford University Press, 2019.
  * [20] Alex Krizhevsky et al.  Learning multiple layers of features from tiny images.  2009\.
  * [21] Tianyang Lin, Yuxin Wang, Xiangyang Liu, and Xipeng Qiu.  A survey of transformers.  AI Open, 2022.
  * [22] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.  Deep learning face attributes in the wild.  In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
  * [23] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.  Towards deep learning models resistant to adversarial attacks.  arXiv preprint arXiv:1706.06083, 2017.
  * [24] Dominic Masters and Carlo Luschi.  Revisiting small batch training for deep neural networks.  arXiv preprint arXiv:1804.07612, 2018.
  * [25] Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan.  A survey on bias and fairness in machine learning.  ACM computing surveys (CSUR), 54(6):1–35, 2021.
  * [26] Samuel G Müller and Frank Hutter.  Trivialaugment: Tuning-free yet state-of-the-art data augmentation.  In Proceedings of the IEEE/CVF international conference on computer vision, pages 774–782, 2021.
  * [27] Valerio Perrone, Michele Donini, Muhammad Bilal Zafar, Robin Schmucker, Krishnaram Kenthapadi, and Cédric Archambeau.  Fair bayesian optimization.  In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 854–863, 2021.
  * [28] Natalia Ponomareva, Hussein Hazimeh, Alex Kurakin, Zheng Xu, Carson Denison, H Brendan McMahan, Sergei Vassilvitskii, Steve Chien, and Abhradeep Guha Thakurta.  How to dp-fy ml: A practical guide to machine learning with differential privacy.  Journal of Artificial Intelligence Research, 77:1113–1201, 2023.
  * [29] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever.  Zero-shot text-to-image generation.  In International Conference on Machine Learning, pages 8821–8831. PMLR, 2021.
  * [30] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
  * [31] Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al.  Bloom: A 176b-parameter open-access multilingual language model.  arXiv preprint arXiv:2211.05100, 2022.
  * [32] Koosha Sharifani and Mahyar Amini.  Machine learning and deep learning: A review of methods and applications.  World Information Technology and Engineering Journal, 10(07):3897–3904, 2023.
  * [33] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov.  Membership inference attacks against machine learning models.  In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017.
  * [34] Ioana Baldini Soares, Dennis Wei, Karthikeyan Natesan Ramamurthy, Moninder Singh, and Mikhail Yurochkin.  Your fairness may vary: pretrained language model fairness in toxic text classification.  In Annual Meeting of the Association for Computational Linguistics, 2022.
  * [35] Gowthami Somepalli, Liam Fowl, Arpit Bansal, Ping Yeh-Chiang, Yehuda Dar, Richard Baraniuk, Micah Goldblum, and Tom Goldstein.  Can neural nets learn the same model twice? investigating reproducibility and double descent from the decision boundary perspective.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13699–13708, 2022.
  * [36] Adarsh Subbaswamy, Roy Adams, and Suchi Saria.  Evaluating model robustness and stability to dataset shift.  In International conference on artificial intelligence and statistics, pages 2611–2619. PMLR, 2021.
  * [37] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus.  Intriguing properties of neural networks.  In 2nd International Conference on Learning Representations, ICLR 2014, 2014.
  * [38] Kush R Varshney.  Trustworthy machine learning and artificial intelligence.  XRDS: Crossroads, The ACM Magazine for Students, 25(3):26–29, 2019.
  * [39] Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, Eftychios Protopapadakis, et al.  Deep learning for computer vision: A brief review.  Computational intelligence and neuroscience, 2018, 2018.
  * [40] Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, and Olga Russakovsky.  Towards fairness in visual recognition: Effective strategies for bias mitigation.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8919–8928, 2020.
  * [41] Boxi Wu, Jinghui Chen, Deng Cai, Xiaofei He, and Quanquan Gu.  Do wider neural networks really help adversarial robustness?  Advances in Neural Information Processing Systems, 34:7054–7067, 2021.
  * [42] S Yassine, M Esghir, and O Ibrihich.  Using artificial intelligence tools in the judicial domain and the evaluation of their impact on the prediction of judgments.  Procedia Computer Science, 220:1021–1026, 2023.
  * [43] Sergey Zagoruyko and Nikos Komodakis.  Wide residual networks.  In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
  * [44] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang.  Men also like shopping: Reducing gender bias amplification using corpus-level constraints.  In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989, 2017.

##  Appendix A Image Classification on CIFAR10-Skewed

The UTKFace dataset is an excellent choice for our main paper as it contains
valuable metadata and has been extensively studied in trustworthy ML
literature. However, it does come with certain limitations, such as its focus
on binary classification tasks, and being confined to a highly specialized
domain, i.e., facial images. To address these limitations, we will conduct a
second case study using a skewed version of the CIFAR-10 dataset (more details
below).

###  A.1 Experiment Setup

#### Dataset

We will adopt the CIFAR10-Skewed setup from Wang  _et al_. [40] for our case
study. In this setup, the 10 object classes of CIFAR10 [20] are divided into
two groups, i.e., colour majority and grayscale majority. The first 5 classes
(airplane, automobile, bird, cat, deer) are marked as the colour majority,
i.e., $95\%$ of images from these classes are left as is, while the other
$5\%$ are converted into grayscale images. Conversely, the last 5 classes
(dog, frog, horse, ship, truck). are marked as the grayscale majority, i.e.,
$95\%$ of images from these classes are converted into grayscale images, while
the other $5\%$ are left as is.

#### Training Details

The training details are the same as the UTKFace Setup, except that we train
the models for only 20 epochs on CIFAR10-Skewed.

#### Axes of Multiplicity

The axes of multiplicity are the same as the UTKFace Setup, except for the
model architecture. We will only use a modified ResNet18 model (adapted for
CIFAR10 images of size 32x32) and we will not study multiplicity across
changing architecture in this case study.

###  A.2 Accuracy

92.0491.8791.6591.9391.8791.4391.9092.2191.7891.6591.8991.3891.9891.8191.680.390.500.430.510.300.390.400.40Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

92.0491.7891.2991.9391.9491.2691.9091.6891.0091.6591.5391.3091.9892.1191.130.750.680.900.350.980.390.580.30Batch
Size$\Delta_{max}$128256640

92.0490.8091.9391.0891.9090.5891.6590.8091.9890.961.240.851.320.851.020.390.50Augmentation$\Delta_{max}$RandTrivial

92.0491.9491.9391.4991.9091.6391.6591.8591.9892.010.100.440.270.200.030.390.52Optimizer$\Delta_{max}$SGDAdam

$90.5$$92.5$

$0$$2$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD

Metric: Accuracy Dataset: CIFAR10-Skewed g $\Delta_{max}^{all}$: 1.63

Figure 10: Multiplicity sheet for Accuracy on CIFAR10-Skewed dataset.

The multiplicity sheet for accuracy on the CIFAR10-Skewed dataset is created
in Fig. 10. Note that the test dataset for CIFAR10-Skewed, i.e., the dataset
on which we measure this accuracy, is also skewed and has the same formulation
as the training dataset defined above. The trends of accuracy multiplicity are
quite similar to that of UTKFace, i.e., no significant accuracy variance is
present across any hyperparameter choice or random seeds.

###  A.3 Group Fairness

62.8159.5057.0354.9654.1352.8958.2758.6854.9659.0957.8555.3760.3357.0357.855.792.073.723.723.317.855.374.96Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

62.8158.6861.5754.9655.7954.9658.2761.9854.5559.0955.3757.4460.3359.0954.134.130.837.443.726.207.856.617.44Batch
Size$\Delta_{max}$128256640

62.8156.2054.9657.8558.2751.6559.0954.9660.3358.276.612.896.614.132.077.856.61Augmentation$\Delta_{max}$RandTrivial

62.8157.4454.9653.7258.2760.7459.0959.5060.3358.685.371.242.480.411.657.857.03Optimizer$\Delta_{max}$SGDAdam

$51$$63$

$0$$8$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD

Metric: Group Accuracy (GS Minority) Dataset: CIFAR10-Skewed g
$\Delta_{max}^{all}$: 11.16

Figure 11: Multiplicity sheet for Group Accuracy (GS Minority) on
CIFAR10-Skewed dataset.

For the CIFAR10-Skewed dataset, grayscale images in the ’colour majority’
object classes and colour images in the ’grayscale majority’ object classes
are both minority groups. For this particular case study, we will measure the
group accuracy of the GS Minority, i.e., the grayscale minority in the colour
majority object classes, as the fairness score under intervention. The results
are collected in Fig. 11.

As previously noted, the multiplicity sheet in Fig. 11 highlights the
importance of random seeds in fairness. While other hyperparameter choices do
have a noticeable impact (in order - training data augmentation, batch size,
learning rate, and the optimization algorithm), clearly the most consistently
dominant source of fairness multiplicity is the randomness in model training.
Moreover, the overall fairness multiplicity ($\Delta_{max}^{all}=11.16$) is
almost seven times higher than the accuracy multiplicity
($\Delta_{max}^{all}=1.63$), further highlighting the severe impact of
multiplicity on trustworthy metrics.

###  A.4 Out-of-Distribution Robustness

78.7978.3278.0077.6376.5375.7678.9878.0177.5378.3077.3777.2678.1777.2876.910.791.871.451.041.261.351.792.24Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

78.7978.3976.9677.6376.4875.5078.9878.0076.4678.3077.5676.3678.1777.4175.551.832.132.521.942.621.351.911.46Batch
Size$\Delta_{max}$128256640

78.7975.7877.6376.7178.9874.9778.3076.0678.1776.973.010.924.012.241.201.352.00Augmentation$\Delta_{max}$RandTrivial

78.7977.2077.6375.5478.9877.4378.3078.7078.1777.831.592.091.550.400.341.353.16Optimizer$\Delta_{max}$SGDAdam

$75$$79$

$0$$4$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD

Metric: OOD Accuracy (CIFAR10-GS) Dataset: CIFAR10-Skewed g
$\Delta_{max}^{all}$: 4.01

Figure 12: Multiplicity sheet for OOD Accuracy (CIFAR10-GS) on CIFAR10-Skewed
dataset.

We will use accuracy on a grayscale version of the CIFAR10 dataset [20] as the
measure of our OOD robustness multiplicity. In Fig. 12, we present the
multiplicity sheet for Accuracy on the CIFAR10-GS dataset. The results show
similar trends to robustness multiplicity for UTKFace, with both
hyperparameter choices and the random seed being equally important in
affecting the model’s robustness. The range of overall robustness multiplicity
($\Delta_{max}^{all}=4.01$) is a little more than two times higher than
accuracy multiplicity, which is unsurprising since despite being grayscale,
the test images still belong to CIFAR10. A more severe robustness check on a
dataset that is quite different from CIFAR10 might introduce and even higher
OOD robustness multiplicity.

###  A.5 Differential Privacy

76.5773.1368.4776.4774.3668.8376.0672.0168.2076.2973.7068.2876.8872.9468.418.107.647.868.018.470.822.350.63Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

76.5773.6667.7176.4773.8668.4876.0672.6866.6076.2972.6367.4276.8873.4767.078.867.999.468.879.810.821.231.88Batch
Size$\Delta_{max}$128256640

76.5768.5576.4768.7476.0668.0676.2967.8976.8868.328.027.738.008.408.560.820.85Augmentation$\Delta_{max}$RandTrivial

76.5769.5076.4768.2276.0667.5676.2969.3376.8869.237.078.258.506.967.650.821.94Optimizer$\Delta_{max}$SGDAdam

$67$$77$

$0$$10$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD

Metric: Pert. Accuracy ($\lambda=5$) Dataset: CIFAR10-Skewed g
$\Delta_{max}^{all}$: 10.28

Figure 13: Multiplicity sheet for Perturbation Accuracy ($\lambda=5$) on
CIFAR10-Skewed dataset.

We use the same perturbation and trade-off setup for privacy multiplicity as
done for UTKFace, i.e., we will measure the accuracy of the model under output
perturbations from an exponential distribution with a fixed rate parameter
$\lambda$. We present the multiplicity sheet for privacy by recording the
Perturbation Accuracy with $\lambda=5$ in Fig. 13. The same trends as Fig. 5
are noticed, i.e., the random seed has minimal impact and it’s the
hyperparameters that dramatically influence the privacy multiplicity, in line
with the existing literature on practical tips for privacy [28]. The overall
privacy multiplicity range ($\Delta_{max}^{all}=10.28$) is also almost six
times larger than the accuracy multiplicity but would depend on the rate
parameter $\lambda$.

###  A.6 Security against Adversarial Attacks

84.1084.3284.6283.6784.0684.4784.1184.0283.8783.0883.7884.0283.9884.1885.360.520.800.240.941.381.030.541.49Learning
Rate$\Delta_{max}$0.10.050.01Changing Random Seeds$\Delta_{max}$

84.1084.3484.5183.6784.4084.6584.1184.4484.5983.0884.5784.3983.9884.5685.330.410.980.481.491.351.030.230.94Batch
Size$\Delta_{max}$128256640

84.1082.8783.6781.1084.1182.2783.0880.8583.9881.191.232.571.842.232.791.032.02Augmentation$\Delta_{max}$RandTrivial

84.1083.0683.6781.8784.1184.4483.0882.9783.9883.461.041.800.330.110.521.032.57Optimizer$\Delta_{max}$SGDAdam

$80$$86$

$0$$4$

Default Config: Learning Rate 0.1; Batch Size 128; Augmentation Rand;
Optimizer SGD

Metric: PGD Accuracy ($\delta=0.005$) Dataset: CIFAR10-Skewed g
$\Delta_{max}^{all}$: 4.51

Figure 14: Multiplicity sheet for PGD Accuracy ($\delta=0.005$) on
CIFAR10-Skewed dataset.

We will use the same setup for security multiplicity as UTKFace, i.e., we use
projected gradient descent (PGD) [23] to progressively move out of the local
minima until we reach the given distance budget, and then measure the accuracy
of the model under these perturbations. We present the multiplicity sheet for
PGD Accuracy with $\delta=0.005$ in Fig. 14. The trends are again similar to
the ones seen in the main paper, i.e., no single factor dominates the
multiplicity. The overall multiplicity range ($\Delta_{max}^{all}=4.51$) for
security is almost five times larger than the accuracy multiplicity and
clearly depends on the adversarial distance budget $\delta$.

###  A.7 Discussion

We have provided an additional case study on the CIFAR10-Skewed dataset as a
companion to our main case study on the UTKFace dataset. These results help us
cement certain trends, for example, the impact of random seeds on fairness,
the impact of hyperparameter choices on privacy-utility trade-off, etc., all
of which are unsurprising as these trends have been noted previously in the
literature (albeit in isolated settings). We believe these experiments will
serve as a useful companion to our main paper, and help establish the
importance of multiplicity sheets in image classification.