# Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable
Machine Unlearning

Zheyuan Liu  University of Notre DameSouth BendIndianaUSA<EMAIL_ADDRESS>,
Guangyao Dou  University of PennsylvaniaPhiladelphiaPennsylvaniaUSA
<EMAIL_ADDRESS>,  Yijun Tian  University of Notre DameSouth
BendIndianaUSA<EMAIL_ADDRESS>,  Chunhui Zhang  Dartmouth CollegeHanoverNew
HampshireUSA<EMAIL_ADDRESS>,  Eli Chien  Georgia Institute of
TechnologyAtlantaGeorgiaUSA<EMAIL_ADDRESS>and  Ziwei Zhu  George Mason
UniversityFairfaxVirginiaUSA<EMAIL_ADDRESS>
(2023; 20 February 2007; 12 March 2009; 5 June 2009)

###### Abstract.

Machine Unlearning (MU) algorithms have become increasingly critical due to
the imperative adherence to data privacy regulations. The primary objective of
MU is to erase the influence of specific data samples on a given model without
the need to retrain it from scratch. Accordingly, existing methods focus on
maximizing user privacy protection. However, there are different degrees of
privacy regulations for each real-world web-based application. Exploring the
full spectrum of trade-offs between privacy, model utility, and runtime
efficiency is critical for practical unlearning scenarios. Furthermore,
designing the MU algorithm with simple control of the aforementioned trade-off
is desirable but challenging due to the inherent complex interaction. To
address the challenges, we present Controllable Machine Unlearning (ConMU), a
novel framework designed to facilitate the calibration of MU. The ConMU
framework contains three integral modules: an important data selection module
that reconciles the runtime efficiency and model generalization, a progressive
Gaussian mechanism module that balances privacy and model generalization, and
an unlearning proxy that controls the trade-offs between privacy and runtime
efficiency. Comprehensive experiments on various benchmark datasets have
demonstrated the robust adaptability of our control mechanism and its
superiority over established unlearning methods. ConMU explores the full
spectrum of the Privacy-Utility-Efficiency trade-off and allows practitioners
to account for different real-world regulations. Source code available at:
https://github.com/guangyaodou/ConMU.

Machine Unlearning, Data Privacy, Trustworthy ML, Deep Learning

††copyright: acmcopyright††journalyear: 2023

##  1\. Introduction

Machine Learning (ML) models are often trained on real-world datasets in
various domains, including computer vision, natural language processing, and
recommender systems (Wu et al., 2023; He et al., 2017; Carion et al., 2020;
Brown et al., 2020). For example, many computer vision models are trained on
images provided by Flickr users (Thomee et al., 2016), whereas an amount of
natural language processing and recommender system algorithms have a high
reliance on IMDB (Maas et al., 2011). Meanwhile, privacy regulations like the
General Data Protection Regulation of the European Union (Hoofnagle et al.,
2019) and the California Consumer Privacy Act (Pardau, 2018) have established
the right to be forgotten (Dang, 2021; Bourtoule et al., 2021; Nguyen et al.,
2022). This mandates the elimination of specific user data from models upon
removal requests.

One naive approach to “forget” user data is removing it from the training set.
However, this cannot provide sufficient privacy since ML models tend to
memorize training samples (Feldman, 2020). Organizations must either
expensively retrain the model from scratch without using specified samples or
employ machine unlearning (Cao and Yang, 2015) techniques to protect user data
privacy. Machine unlearning methods are designed to meticulously eliminate
samples and their associated influence from both the dataset and the trained
model. This safeguards the data privacy with unlearning, protecting it from
potential malicious attacks and privacy breaches.

Figure 1.  Privacy, utility, efficiency trilemma in machine unlearning. All
previous works have focused on either one or two extremities of the problem
while ignoring the full spectrum of trade-offs between the trinity (as shown
in blue dots on each subplot). Each of the proposed modules in our ConMU
offers smooth control of a pair of two unlearning aspects specifically.
Together, ConMU is capable of achieving a satisfactory outcome for versatile
practical scenarios, including various degrees of privacy regulations,
efficiency constraints, and utility objectives.

Beyond privacy, utility and efficiency are also important aspects of machine
unlearning problems. For instance, sacrificing utility by naively returning
constant or purely random output ensures privacy but results in a useless
model. On the other hand, retraining from scratch without data subject to
removal guarantees privacy and utility yet is prohibitively expensive.
Designing a method that simultaneously maximizes the privacy, utility, and
efficiency aspects of machine unlearning is critical and needed.
Unfortunately, theoretical machine unlearning research provides evidence that
there is an inevitable privacy-utility-efficiency trade-off even for convex
problems (Guo et al., 2020; Sekhari et al., 2021; Chien et al., 2022; Pan et
al., 2023) and similar phenomena exist in other privacy problems (Chen et al.,
2020; Chien et al., 2023). This leads to a trilemma of trinity aspects of
machine unlearning, which are accuracy, privacy, and runtime efficiency. While
the aforementioned theoretical unlearning solutions provide smooth control
among the trinity, they are restricted to simple models and cannot be
generalized to general deep-learning approaches.

While a number of efforts have been put into machine unlearning, existing
unlearning solutions for deep neural networks mainly focus on maximizing part
of the trinity while neglecting their delicate trade-off. In real-world
scenarios, different applications would require different levels of privacy
regulations, runtime constraints, and utility demand. For example, protecting
user identities in healthcare applications (d’Aliberti and Clark, 2022) might
be stricter than safeguarding friendship data on social networks. However,
autonomous driving (Carion et al., 2020) and fraudulent attack detection in
financial systems (Zhu et al., 2021) would prioritize accuracy and runtime
efficiency more than privacy.

Therefore, a practical machine unlearning solution should be able to easily
account for different levels of privacy, utility, and efficiency requirements
that arise from various tasks in practice. Unfortunately, the literature lacks
a comprehensive examination and controllable MU approach for deep learning to
the intricate dynamics involved in balancing privacy, model accuracy, and
runtime efficiency. A natural yet pivotal research question arises: "How to
resolve the unlearning trilemma for deep neural networks?".

To answer this question, we present Controllable Machine Unlearning (ConMU), a
novel framework that consists of three components: an important data selection
module, a progressive Gaussian mechanism, and an unlearning proxy. Each
component emphasizes one part of the aforementioned trilemma, see Figure 1 for
the illustration. In particular, the important data selection module modulates
the relationship between runtime and model accuracy. The progressive Gaussian
mechanism controls the trade-offs between accuracy and privacy. The unlearning
proxy facilitates a re-calibration between runtime and privacy. We further
underscore that the ConMU is adaptable and can be generalized across diverse
model architectures. Among all conducted experiments, ConMU achieves the best
privacy performance across 10 out of 12 experiments, with competitive model
utility and a 10-15x faster runtime efficiency. Additionally, compared to the
naive control baseline, ConMU has illustrated greater control over the
trilemma by exhibiting superior and stable performance under the influence of
multiple trade-offs. Our main contributions are as follows:

  * •

To the best of our knowledge, this is the first work of tackling the critical
trilemma within the realm of machine unlearning for deep neural networks, with
a specific focus on the delicate balance between privacy, model utility, and
runtime efficiency.

  * •

We propose ConMU, which contains three modules, each designed to reconcile
these competing factors: important data selection, progressive Gaussian
mechanism, and unlearning proxy.

  * •

Extensive experiments demonstrate the effectiveness of our proposed framework
under both class-wise and random forgetting requests.

##  2\. Related Work

###  2.1. Machine Unlearning with Theoretical Guarantees

The concept of machine unlearning was first raised in (Cao and Yang, 2015). In
general, two unlearning criteria have been considered in previous works: Exact
Unlearning and Approximate Unlearning. Exact unlearning requires eliminating
all information relevant to the removed data so that the unlearned model
performs exactly the same as a completely retrained model. For example, the
authors of (Ginart et al., 2019) presented unlearning approaches for $k$-means
clustering. (Bourtoule et al., 2021) proposed the SISA framework that
partitions data into shards and slices, and each shard has a weak learner,
which enables quick retraining when dealing with unlearning requests. However,
exact unlearning does not allow algorithms to trade privacy for utility and
efficiency due to its high requirements for privacy level. In contrast, unlike
exact unlearning, approximate unlearning only requires the parameters of the
unlearned model to be similar to a retrained model from scratch. Therefore, it
is possible for approximate unlearning to sacrifice a portion of the privacy
in exchange for better utility and efficiency. (Guo et al., 2020; Sekhari et
al., 2021) studied the approximate unlearning for the cases of linear models
and convex losses. (Neel et al., 2021) extended the idea and provided a
theoretical guarantee on weak convex losses. (Chien et al., 2022; Pan et al.,
2023) generalize the method of (Guo et al., 2020) to the graph learning
domain. Nevertheless, none of these approximate unlearning solutions apply to
general deep neural networks.

###  2.2. Unlearning in Deep Learning Models

Machine unlearning for deep neural networks is challenging because of the non-
convex nature of the loss function (Basu et al., 2020). (Koh and Liang, 2017)
approximated the model perturbation towards the empirical risk minimization on
the remaining datasets, using the inverse of Hessian. (Golatkar et al., 2020a)
used fisher-based unlearning and introduced an upper bound of SGD-based
algorithms to scrub information from intermediate layers of DNNs. (Golatkar et
al., 2020b) extended the framework and introduced forgetting methods using NTK
theory (Jacot et al., 2018). (Chundawat et al., 2023a) proposed a knowledge
adaptation technique where the unlearned model tries to learn from a competent
teacher model about retained dataset and an incompetent model about forgetting
dataset. (Chundawat et al., 2023b) proposed an unlearning method without using
any training samples, in which they used the error-maximizing noise, proposed
by (Tarun et al., 2023b), to generate an impaired forgetting dataset, and then
used the error-minimizing noise to generate the approximated retained dataset.
However, this method yields poor results. Thus, (Chundawat et al., 2023b)
proposed another unlearning algorithm that uses gated knowledge transfer in a
teacher-student framework. (Wang et al., 2023) also proposed a Knowledge Gap
Alignment method that minimizes the output distribution difference between
models that are trained on different data samples. (Tarun et al., 2023a)
focuses on deep regression unlearning tasks using a partially trained
blindspot model to minimize the distribution difference with the original
model. Lastly, (Jia et al., 2023) showed that applying unlearning algorithms
on pruned models gives better performance.

Though important data selections were largely used in deep learning (Paul et
al., 2021), their implementation in the MU field is still unexplored. We
discovered that the important data selection is able to offer strong control
over the utility-efficiency trade-off. Similarly, Gaussian Noise had been
largely adopted in the field of differential privacy (Balle and Wang, 2018;
Dwork et al., 2006, 2014; Canonne et al., 2020; Bu et al., 2020; Dong et al.,
2019; Gao and Wright, 2023; Abadi et al., 2016), while its usage in machine
unlearning is not yet fully investigated. In addition, the concept of
utilizing partially competent teachers for a privacy-efficiency trade-off has
not been previously examined. Moreover, many of the existing works focused
solely on privacy, overlooking the relationships between accuracy, privacy,
and runtime efficiency. Unlike other machine-unlearning algorithms, our method
gives users exceptional flexibility and control over the trade-offs among
these three factors. In addition, our method imposes no restrictions on
optimization methodologies or model architecture.

##  3\. Preliminaries

Removing certain training data samples can impact a model’s accuracy,
potentially improving, maintaining, or diminishing it (Sorscher et al., 2022).
As noted by (Chundawat et al., 2023a), significant discrepancies between
unlearned and retrained models can lead to the Streisand effect, inadvertently
revealing information about forgotten samples through unusual model behavior.
Therefore, the goal of Machine Unlearning is to erase the influence of the set
of samples we want to forget so that the unlearned model approximates the
retraining one.

Let $D_{o}=\\{x_{i}\\}^{N}_{i=1}$ be the complete dataset before unlearning
requests, in which $x_{i}$ is the $i^{th}$ sample. Let $D_{f}$ be the set of
samples we want to forget as forgetting dataset, and the complement of
$D_{f}$, which we denote as $D_{r}$, is the set of samples retained in the
training samples, i.e. $D_{f}\cup D_{r}=D_{o}$ and $D_{f}\cap
D_{r}=\emptyset$. In the setting of random forgetting, $D_{f}$ may contain
samples from different classes of $D_{o}$. In class-wise forgetting, $D_{f}$
is a set of examples that have the same class labels. We denote $\theta_{o}$
as the parameters of the original model, which was trained on $D_{o}$, denote
the parameters of unlearned models as $\theta_{u}$, and denote the parameters
of retrained model $\theta_{r}$, which is the model completely retrained from
scratch using only $D_{r}$. Lastly, let $\theta_{I}$ denote the parameters of
the unlearning proxy, which has the same model architecture as $\theta_{o}$,
but was only partially trained on $D_{r}$ for a few epochs.

$\theta_{r}$ is the gold standard in our MU problem. The goal of machine
unlearning is to approximate our $\theta_{u}$ to $\theta_{r}$, with less
computational overhead. However, for machine unlearning on deep neural
networks, achieving a balance between utility, privacy, and efficiency has
always been a difficult task.

##  4\. Methods

Figure 2. The overall framework of proposed method ConMU, which is placed
after forgetting request. In (a), an important data selection is implemented
to select data samples that are important to the model. A customized
upper/lower bound is attached to this module to facilitate the selection
process. Then, the selected forgetting data $D_{f}^{\prime}$ is passed to (b),
the progressive Gaussian mechanism, to gradually inject Gaussian noise. More
noise in the image leads to higher privacy. Afterward, the processed
forgetting data $D_{f}^{\prime\prime}$ is concatenated with the selected
retaining data $D_{r}^{\prime}$, which is used for fine-tuning the original
model. The unlearning proxy (c) is partially trained on the retaining data
$D_{r}$ and knowledge is transferred to the original model via KL Divergence.

To address such a trilemma in machine unlearning, we introduce the _ConMU_
(Figure 2), a novel framework that consists of an important data selection, a
progressive Gaussian mechanism, and an unlearning proxy that modulate
relationships among accuracy, privacy, and runtime efficiency. First, the
important data selection module (Figure 2 (a)) selectively discards
unimportant retaining and forgetting data samples that will not be utilized by
subsequent modules. Discarding more samples improves training time while
degrading the model’s accuracy. Next, the Progressive Gaussian Mechanism
module (Figure 2 (b)) injects Gaussian noise into the remaining forgetting
dataset. The amount of noise can control the balance between privacy and
accuracy. Subsequently, an unlearning proxy model (Figure 2 (c)) is trained on
the retained dataset for a select number of epochs. Through knowledge
transfer, the training epoch of the proxy can balance the runtime and privacy.
Finally, by fine-tuning the original model using the concatenated retained and
noised forgetting datasets, it is transformed into an unlearned version. As a
result, by controlling the data volume, the Gaussian noise level, and proxy
training duration, we are able to account for different privacy-utility-
efficiency requirements. Subsequent sections delve deeper into each module’s
capabilities and their influence on the trilemma.

###  4.1. Important Data Selection

Unlearning acceleration is crucial in MU. Since our method uses both remaining
noised $D_{f}$ and remaining $D_{r}$ to perform fine-tuning, the amount of
$D_{f}$ and $D_{r}$ play significant roles in the run time of our proposed
methods. However, the large quantities of $D_{f}$ and $D_{r}$ will likely
result in an inefficient MU algorithm with a long runtime. Therefore, to
facilitate this process, we introduce a novel filtering method using EL2N
scores to determine which samples are important for unlearning scenarios.
Suppose that $f(\theta,x)$ is the output of the neural network $\theta$ with
given data $x$, and denote $y$ as the true class label of $x$. We calculate
the mean and the standard deviation of $l2$ normed loss:

(1) $\mu_{\theta}(x)=\mathbb{E}_{x}||f(\theta,x)-y||_{2},$ (2)
$\sigma_{\theta}(x)=\sqrt{\mathbb{V}_{x}||f(\theta,x)-y||_{2}}.$

A higher $\mu_{\theta}$ means that $x$ is hard to learn and they tend to be
the outliers in the dataset. A lower $\mu_{\theta}$ means that $\theta$ can
fit $x$ well. Therefore, we can keep data samples important for the
generalization of models by keeping data within a certain range of samples
that don’t have a very high or low $\mu_{\theta}$. In our method, we introduce
two controllable hyperparameters $z_{1}$ and $z_{2}$ and calculate a bound:

(3)
$[\mu_{\theta}(x)-z_{1}\times\sigma_{\theta}(x),\mu_{\theta}(x)+z_{2}\times\sigma_{\theta}(x)].$

This bound gives users control of how many important data points we want to
include by tuning $z_{1}$ and $z_{2}$. If we include more data, our accuracy
increases, but the runtime also increases. As a result, the ConMU can have a
greater speed-up while maximally preserving accuracy by utilizing important
data samples.

###  4.2. Progressive Gaussian Mechanism

MU algorithms aim to erase the information about $D_{f}$ from the original
model. In order to forget $D_{f}$, we can continue training the original model
using an obfuscated version of $D_{f}$, prompting catastrophic forgetting of
$D_{f}$. Within this context, we propose the progressive Gaussian mechanism,
which leverages Gaussian noise to obscure the selected $D_{f}$. Moreover, one
of the standout features of this approach is that the magnitude and the shape
of Gaussian noise applied to the dataset serve as tunable hyperparameters,
granting a remarkable degree of control over the process. More formally, after
selecting a subset of important samples:

(4)
$D_{f}^{\prime}\in[\mu_{\theta_{u}}(D_{f})-z_{1}\times\sigma_{\theta_{u}}(D_{f}),\mu_{\theta_{u}}(D_{f})+z_{2}\times\sigma_{\theta_{u}}(D_{f})],$

the ConMU adds Gaussian noise to data samples to balance privacy and accuracy.
More specifically, for each data samples in $D_{f}^{\prime}$, we add Gaussian
noise and obtain:

(5) $D_{f}^{\prime\prime}=D_{f}^{\prime}+\alpha\times
N,\;N\sim\mathcal{N}(\mu,\sigma^{2}\mathbf{I}),$

where $\alpha$, $\mu$, and $\sigma^{2}$ are controllable hyperparameters,
where $\mu$ and $\sigma^{2}$ represent the mean and variance of the Gaussian
distribution, and the $\alpha$ represents the number of times the noise was
added to the sample. With more noise being added to the data samples, we will
get higher privacy, but lower model accuracy. Therefore, the progressive
Gaussian mechanism controls the amount of information they want to scrub away
and the amount of information that they want to preserve to maintain the
accuracy of the model. In Section 5.3, we empirically demonstrated that with
larger $\alpha$, the accuracy decreases and the privacy increases, and vice
versa.

###  4.3. Fine-tuning with Unlearning Proxy

The objective of machine unlearning is to align the output distribution of the
unlearned model closely with that of the retrained model — a model never
exposed to the forgotten data samples. To achieve this, we can utilize an
unlearning proxy model, which is a model that has the same architecture as the
original model and is partially trained on the retained dataset for a few
epochs. By transferring the knowledge of the behavior of the unlearning proxy,
we can obtain an unlearned model that contains less information about the
forgetting datasets.

More formally, the unlearning proxy model $\theta_{I}$ is partially trained on
the retained dataset $D_{r}$ for $\delta$ epochs, in which $\delta$ is a
hyperparameter. Next, we compute the KL divergence between the probability
distribution of $\theta_{I}$’s output on the input data $x$ and that of the
$\theta_{u}$ as:

(6)
$D_{KL}(\theta_{I}(x)\parallel\theta_{u}(x))=\sum_{i}\theta_{I}(x)(i)\log\left(\frac{\theta_{I}(x)(i)}{\theta_{u}(x)(i)}\right),$

where $i$ corresponds to the data class. We want to minimize this KL
divergence, aiming to make the output distribution of the unlearned model
$\theta_{u}$ as close as possible to that of a model that has never seen
$D_{f}$, which is the unlearning proxy. In section 5.3, we demonstrate that if
$\delta$ increases, the $\theta_{u}$ will become more similar to $\theta_{r}$,
but with increasing runtime.

###  4.4. Controlling Machine Unlearning

After discussing the individual modules for important data selection,
progressive Gaussian mechanism, and using an unlearning proxy, we now focus on
how these parts come together. First, we obtain
$D_{new}=D_{f}^{\prime\prime}\cup D_{r}^{\prime}$, in which:

(7)
$D_{r}^{\prime}\in[\mu_{\theta_{u}}(D_{r})-z_{1}^{\prime}\times\sigma_{\theta_{u}}(D_{r}),\mu_{\theta_{u}}(D_{r})+z_{2}^{\prime}\times\sigma_{\theta_{u}}(D_{r})],$

and $z_{1}^{\prime}$ and $z_{2}^{\prime}$ are two hyperparameters for
filtering the retained dataset, as discussed in 4.1. With $D_{new}$, we will
use the cross-entropy (CE) loss to further train $\theta_{u}$ on $D_{new}$,
combined with the KL loss in section 4.3. The loss to train the unlearned
model $\theta_{u}$ is defined as:

(8) $\mathcal{L}=CE(D_{new})+\gamma
D_{KL}(\theta_{I}(D_{new})\parallel\theta_{u}(D_{new})).$

The $\gamma$ in Equation (8) ensures that these two losses are on the same
scale. In summary, the ConMU uses Equation (8) to fine-tune the original model
$\theta_{o}$ to achieve $\theta_{u}$, with way fewer epochs required by
complete retraining, allowing the calibration of the amount of data to
preserve, the amount of noise added to the filtered forget data samples, and
number of times to train the unlearning proxy. With these three modules, the
ConMU allows controllable trade-offs between accuracy, privacy, and runtime.

###  4.5. Forget-Retain-MIA Score

There are many evaluation metrics to determine the privacy of the unlearning
algorithms. For example, many literatures used Retain Accuracy (RA) and Forget
Accuracy (FA) (Becker and Liebig, 2022; Jia et al., 2023; Chundawat et al.,
2023a, b; Nguyen et al., 2022; Golatkar et al., 2021, 2020a; Tarun et al.,
2023b), which are the generalization ability of the unlearned model on $D_{r}$
and $D_{f}$, respectively. Moreover, many previous works have used Membership
Inference Attacks (MIA) (Dwork et al., 2015; Graves et al., 2021; Song and
Mittal, 2021; Jia et al., 2023; Nguyen et al., 2022; Chundawat et al., 2023a)
that determine whether a particular training sample was present in the
training data for a model.

Given this landscape of varied metrics, it becomes imperative to consolidate
them to yield a more comprehensive evaluation. As we have stated in section 3,
our goal for the evaluation of privacy is to ensure minimal disparity between
our model’s outcomes and the retrained model, which is the gold standard of
unlearning tasks. Therefore, we introduce a new evaluation metric called the
_Forget-Retain-MIA_ (FRM) score that considers the differences between the
unlearned model with the retrained model on the trifecta of FA, RA, and MIA,
which is inspired by NeurlPS 2023 machine unlearning challenge
111https://unlearning-challenge.github.io/. Suppose we denote $FA_{r}$,
$RA_{r}$, and $MIA_{r}$ as the FA, RA, and MIA performance of the retrained
model, and denote $FA_{u}$, $RA_{u}$, and $MIA_{u}$ as the FA, RA, and MIA
performance of the unlearning model, we calculate the FRM score as:

(9)
$FRM=exp\bigl{(}-(\frac{|FA_{u}-FA_{r}|}{FA_{r}}+\frac{|RA_{u}-RA_{r}|}{RA_{r}}+\frac{|MIA_{u}-MIA_{r}|}{MIA_{r}})\bigl{)}.$

The FRM score quantitatively compares the normalized differences in FA, RA,
and MIA performances of the unlearning model with that of its retrained
counterpart. The FRM score lies between 0 and 1. An FRM score of an unlearning
model will be closer to 1 if the unlearned model’s privacy is perfectly
aligned with the retained model’s privacy, and it will be closer to 0 if the
model is completely different from the retrained model. An ideal FRM score of
1 signifies that the unlearning algorithm has achieved exact unlearning. We
use the FRM score to evaluate the ConMU and other baseline models’ performance
on privacy in the subsequent experiment sections.

##  5\. Experiments

In this section, we conduct extensive experiments to validate the
effectiveness of the ConMU. In particular, through the experiments, we aim to
answer the following research questions: (1) Can ConMU find the best balance
point given the trilemma? (2) Can each module effectively control a specific
aspect of the trilemma? (3) Can the naive fine-tune method possess the same
control ability as the ConMU?

###  5.1. Experiment setups

####  5.1.1. Datasets and models.

Our experiments mainly focus on image classification for CIFAR-10 (Krizhevsky
et al., 2009) on ResNet-18 (He et al., 2016) under two unlearning scenarios:
random data forgetting and class-wise data forgetting. Besides, additional
experiments are conducted on CIFAR100 (Krizhevsky et al., 2009), and SVHN
(Netzer et al., 2018) datasets using vgg-16 (Simonyan and Zisserman, 2014).

####  5.1.2. Baseline Models.

For baselines, we compare with Fine-Tuning (FT) (Warnecke et al., 2021;
Golatkar et al., 2020a; Jia et al., 2023), Gradient Ascent (GA) (Jia et al.,
2023; Graves et al., 2021), and Influence Unlearning (IU) (Koh and Liang,
2017; Nguyen et al., 2022; Cook and Weisberg, 1980; Jia et al., 2023; Golatkar
et al., 2021). In particular, FT directly utilizes retained dataset $D_{r}$ to
fine-tune the original model $\theta_{o}$. The GA method attempts to add the
gradient updates on $D_{f}$ during the training process back to the
$\theta_{o}$. Lastly, IU leverages influence functions to remove the influence
of the target data sample in $\theta_{o}$. Besides, (Jia et al., 2023) has
shown that pruning first before applying unlearning algorithms will increase
performance. Therefore, we apply the OMP (one-shot-magnitude pruning) (Ma et
al., 2021; LeCun et al., 1989; Jia et al., 2023; Molchanov et al., 2016) to
each baseline model as well as the ConMU. The details of each baseline model
are elaborated in Appendix B.1.

####  5.1.3. Evaluation Metrics.

We aim to evaluate MU methods from five perspectives: test accuracy forget
accuracy (TA), forget accuracy (FA), retain accuracy (RA), membership
inference attack (MIA) (Shokri et al., 2017), runtime efficiency (RTE), and
FRM privacy score. Specifically, TA measures the accuracy of $\theta_{u}$ on
the testing datasets and evaluates the generalization ability of MU methods.
FA and RA measure the accuracy of the unlearned model on forgetting dataset
$D_{f}$ and retaining dataset $D_{r}$, respectively. MIA verifies if a
particular training sample existed in the training data for the original
model. Lastly, we use the FRM privacy score metric to comprehensively evaluate
the privacy level of an MU method. Additional details of evaluation metrics
are illustrated in Appendix A.

####  5.1.4. Implementation Details.

We report the mean and the standard deviation in the form of $a_{\pm b}$ of
ten independent runs with different data splits and random seeds. For random
forgetting, we randomly selected 20% of the training samples as forgetting
datasets. For class-wise forgetting, we randomly selected 50% of a particular
class for different datasets as the forgetting samples. The details will be in
appendix B.2.

Table 1. Overall results of our proposed ConMU, with a number of baselines
under two unlearning scenarios: random forgetting and class-wise forgetting.
Since a retrained model is the golden baseline for unlearning tasks, we
evaluate the performance of MU models based on their differences to the
retrained model. Bold indicates the best performance and underline indicates
the runner-up. The unlearning performance of each MU method is evaluated under
five metrics: test accuracy ($TA$), accuracy on forget data ($FA$), accuracy
on retain data ($RA$), membership inference attack ($MIA$), and running time
efficiency ($RTE$). An additional $FRM$ is added on top of that to thoroughly
evaluate the privacy level of each method. Note that a larger value of FRM
denotes a smaller performance discrepancy with the retrained model, which
means a higher privacy level. A performance difference against the retrained
model is reported in ($\bullet$). The best performance in the metrics with
blue $\downarrow$ has the smallest gap with the retrained model. While the
metrics with black $\uparrow$ favor a greater performance value.

MU Methods | $TA(\%)\uparrow$ | $FA(\%){\color[rgb]{0,0,1}\downarrow}$ | $RA(\%){\color[rgb]{0,0,1}\downarrow}$ | $MIA(\%){\color[rgb]{0,0,1}\downarrow}$ | $RTE(s)$ |  $FRM$ $Privacy$ $\uparrow$
---|---|---|---|---|---|---
Resnet-18 Random data forgetting (CIFAR-10)
retrain | $79.99$ |  $80.46$ ($0.00$) |  $91.47$ ($0.00$) |  $19.62$ ($0.00$) | $933.51$ | 1
IU + Pruning | $41.63_{\pm 0.14}$ |  $41.62_{\pm 0.11}$ ($38.84$) |  $41.21_{\pm 0.07}$ ($50.26$) |  $57.61_{\pm 0.11}$ ($37.99$) | 33.69 | 0.051
GA + Pruning | $64.61_{\pm 0.14}$ |  $66.74_{\pm 0.24}$ ($13.72$) |  $66.15_{\pm 0.13}$ ($25.32$) |  $34.15_{\pm 0.24}$ ($14.53$) | 28.15 | 0.305
FT + Pruning | $\textbf{84.71}_{\pm 0.14}$ |  $\underline{84.13}_{\pm 0.06}$ ($3.67$) |  $\textbf{90.96}_{\pm 0.05}$ ($0.51$) |  $\underline{15.42}_{\pm 0.06}$ ($4.20$) | 475.99 | 0.767
ConMU | $\underline{78.83}_{\pm 0.57}$ |  $\textbf{81.22}_{\pm 0.53}$ (0.76) |  $\underline{81.75}_{\pm 0.69}$ (9.72) |  $\textbf{18.93}_{\pm 0.10}$ (0.69) | $59.59$ | 0.855
Resnet-18 Class-Wise forgetting (CIFAR-10)
retrain | 82.55 | 68.22 ($0.00$) | 89.67 ($0.00$) | 33.92 ($0.00$) | 1241.92 | 1
IU + Pruning | $20.39_{\pm 2.25}$ |  $0.01_{\pm 0.00}$ ($68.21$) |  $20.96_{\pm 2.33}$ ($68.71$) |  $100.00_{\pm 0.00}$ ($66.08$) | 40.15 | 0.024
GA + Pruning | $52.22_{\pm 0.35}$ |  $15.22_{\pm 0.41}$ ($53$) |  $53.88_{\pm 0.38}$ ($35.79$) |  $83.23_{\pm 0.41}$ ($49.31$) | 25.24 | 0.072
FT + Pruning | $\textbf{85.75}_{\pm 0.11}$ |  $\underline{69.89}_{\pm 0.72}$ ($1.67$) |  $\textbf{92.10}_{\pm 0.03}$ ($2.43$) |  $\underline{27.81}_{\pm 0.72}$ ($6.11$) | 565.47 | 0.793
ConMU | $\underline{83.61}_{\pm 1.97}$ |  $\textbf{67.23}_{\pm 2.27}$ ($0.99$) |  $\underline{86.68}_{\pm 2.50}$ ($5.75$) |  $\textbf{32.92}_{\pm 2.21}$ ($1.00$) | $89.4$ | 0.925
VGG Random data forgetting (CIFAR-10)
retrain | $81.10$ |  $81.49$ ($0.00$) |  $92.09$ ($0.00$) |  $19.54$ ($0.00$) | $881.57$ | 1
IU + Pruning | $59.74_{\pm 0.08}$ |  $57.97_{\pm 0.10}$ ($23.52$) |  $57.52_{\pm 0.09}$ ($34.57$) |  $39.32_{\pm 0.09}$ ($19.78$) | 38.36 | 0.186
GA + Pruning | $69.43_{\pm 0.14}$ |  $\underline{69.97}_{\pm 0.04}$ ($11.52$) |  $69.79_{\pm 0.09}$ ($22.30$) |  $\underline{29.20}_{\pm 0.04}$ ($9.66$) | 47.17 | 0.414
FT + Pruning | $\textbf{83.88}_{\pm 0.71}$ |  $58.98_{\pm 0.60}$ ($22.51$) |  $\textbf{90.64}_{\pm 0.77}$ ($1.45$) |  $38.41_{\pm 0.60}$ ($18.87$) | 378.02 | 0.283
ConMU | $\underline{79.09}_{\pm 2.19}$ |  $\textbf{82.52}_{\pm 2.41}$ (1.03) |  $\underline{84.00}_{\pm 2.40}$ (8.09) |  $\textbf{17.53}_{\pm 2.43}$ ($2.01$) | 32.42 | 0.816
VGG Class-Wise forgetting (CIFAR-10)
retrain | 82.41 | 69.02 ($0.00$) | 92.90 ($0.00$) | 33.44 ($0.00$) | 1034.40 | 1
IU + Pruning | $53.06_{\pm 17.55}$ |  $27.16_{\pm 28.87}$ ($41.86$) |  $53.08_{\pm 20.04}$ ($38.82$) |  $65.50_{\pm 14.07}$ ($32.06$) | 46.70 | 0.136
GA + Pruning | $53.18_{\pm 0.25}$ |  $11.96_{\pm 0.28}$ ($57.06$) |  $54.51_{\pm 0.29}$ ($38.39$) |  $86.42_{\pm 0.28}$ ($52.98$) | 30.74 | 0.059
FT + Pruning | $\textbf{83.88}_{\pm 0.91}$ |  $\underline{58.98}_{\pm 8.94}$ ($10.04$) |  $\textbf{90.64}_{\pm 0.82}$ ($2.26$) |  $\underline{38.31}_{\pm 8.94}$ ($4.87$) | 353.97 | 0.729
ConMU | $\underline{81.12}_{\pm 3.27}$ |  $\textbf{63.75}_{\pm 8.33}$ ($5.27$) |  $\underline{87.10}_{\pm 3.79}$ ($5.80$) |  $\textbf{36.20}_{\pm 3.30}$ ($2.76$) | 148 | 0.800
VGG Random data forgetting (CIFAR-100)
retrain | $60.65$ |  $60.54$ ($0.00$) |  $92.49$ ($0.00$) |  $40.60$ ($0.00$) | $823.15$ | 1
IU + Pruning | $7.15_{\pm 0.01}$ |  $5.97_{\pm 0.02}$ ($54.57$) |  $5.83_{\pm 0.01}$ ($86.66$) |  $6.65_{\pm 0.02}$ ($33.95$) | 38.74 | 0.069
GA + Pruning | $14.71_{\pm 0.07}$ |  $13.96_{\pm 0.10}$ ($46.58$) |  $14.24_{\pm 0.09}$ ($78.25$) |  $85.53_{\pm 48.73}$ ($44.93$) | 47.42 | 0.066
FT + Pruning | $\underline{49.78}_{\pm 0.57}$ |  $\underline{47.67}_{\pm 0.81}$ ($12.87$) |  $\underline{60.70}_{\pm 0.79}$ ($31.79$) |  $\underline{50.95}_{\pm 0.81}$ ($10.35$) | 215.22 | 0.444
ConMU | $\textbf{55.22}_{\pm 0.32}$ |  $\textbf{61.53}_{\pm 2.58}$ (0.99) |  $\textbf{71.77}_{\pm 1.16}$ (20.72) |  $\textbf{41.42}_{\pm 2.22}$ ($0.82$) | $65.01$ | 0.771
VGG Class-Wise forgetting (CIFAR-100)
retrain | 63.71 | 65.78 ($0.00$) | 93.00 ($0.00$) | 38.21 ($0.00$) | 1036.40 | 1
IU + Pruning | $7.14_{\pm 0.03}$ |  $3.56_{\pm 0.01}$ ($62.22$) |  $5.89_{\pm 0.92}$ ($87.11$) |  $4.44_{\pm 0.92}$ ($33.77$) | 37.95 | 0.063
GA + Pruning | $10.68_{\pm 0.16}$ |  $0.44_{\pm 0.37}$ ($65.34$) |  $10.21_{\pm 0.14}$ ($82.79$) |  $1.00_{\pm 0.37}$ ($37.21$) | 23.01 | 0.057
FT + Pruning | $\underline{57.33}_{\pm 0.58}$ |  $\textbf{64.00}_{\pm 10.41}$ ($1.78$) |  $\underline{74.60}_{\pm 0.86}$ ($18.40$) |  $\underline{36.45}_{\pm 10.41}$ ($1.76$) | 403.07 | 0.762
ConMU | $\textbf{58.78}_{\pm 0.95}$ |  $\underline{58.39}_{\pm 4.53}$ ($7.39$) |  $\textbf{80.03}_{\pm 1.56}$ ($12.97$) |  $\textbf{38.54}_{\pm 4.61}$ ($0.33$) | $53.07$ | 0.771

(a) Test Acc. vs Runtime

(b) Test Acc. vs FRM Score

(c) FRM Score vs Runtime

Figure 3.  Ablation study results of each module on CIFAR-10 with ResNet-18.
For every module, we fix the other two novel modules while adjusting its own
controllable parameters. Since each proposed module is designed to control one
side of the trilemma, we present the results for each module in a chart with
$x$ and $y$ axes representing their respective controlled factors.

###  5.2. Experiment Results

To answer the first question: Can ConMU find the best trade-off points given
three important factors? We conduct random forgetting and class-wise
forgetting to comprehensively evaluate the effectiveness of a MU method. The
performance is reported in Table 1. Note that a better performance of a MU
method contains a smaller performance gap with the retrained model (except TA
and RTE), which is the gold standard for MU tasks. According to the table, we
can find that IU (influence unlearning) and GA (gradient ascent) with OMP
pruning achieve satisfactory results under unlearning privacy (FA, RA, MIA,
and FRM) and efficiency (RTE) metrics with relatively shorter runtime.
According to the table, IU is usually the fastest baseline, while GA is the
runner-up. However, this outstanding unlearning efficiency comes at a high
cost to the model utility, rendering them the worst baseline models in terms
of test accuracy. Alternatively, FT (fine-tuning) performs well across all
metrics with the exception of unlearning efficiency. As shown in Table 1, FT
is the runner-up in the majority of cases and achieves a high FRM across all
benchmarks. However, this comes with a high sacrifice on runtime efficiency,
making it the slowest baseline method. Finally, we observe that the ConMU can
outperform other baselines by remarkable margins and achieve a good balance on
all privacy metrics and competitive accuracy across CIFAR-10, CIFAR-100, and
SVHN, respectively. Additionally, the ConMU has the highest FRM score among
all baseline models with an acceptable runtime efficiency relative to other
baselines. More experiment results of the different datasets are presented in
Appendix C.

###  5.3. Unlearning Trilemma Analysis

Given our proposed ConMU is to narrow the performance gap with the gold-
retrained model and to better control the trade-off between different metrics,
we conduct further experiments to validate the effectiveness of each module.
The central question addressed is: Can each module effectively govern a
specific facet of the aforementioned trilemma? The associated results are
shown in Figure 3. To better answer this question, we first represent the
unlearning trilemma as a triangle (figure 1), wherein each side corresponds to
a distinct aspect of the trilemma. An effective control module should identify
a balance point anywhere along the side, rather than being confined to the two
endpoints. Since the ConMU contains three modules, to better observe the
compatibility and flexibility of each module in influencing different metrics,
we systematically adjust the input values of each module with random
forgetting requests on the CIFAR-10 dataset, showcasing the ability to control
trade-offs at various levels.

####  5.3.1. Utility vs Efficiency

In our proposed method, the important data selection module is specifically
designed to curate the samples that are later utilized in the fine-tuning
process of the pruned model. The rationale behind this is twofold: (1)
extracting the samples that contribute significantly to model generalization
process, and (2) expediting the runtime of the unlearning process. To further
investigate the trade-off between model utility and runtime efficiency, we
carefully adjust the upper and lower bounds of the important data selection to
incorporate different percentiles of data. In Figure 3 (a), we present the
control ability of the proposed important data selection module by selecting
different portions of data. We start with the inclusion of 5 % of the data and
gradually progress to 90 %. From figure 3 (a), we first discover that a higher
percentile of selected data not only prolongs the runtime but also enhances
the utility performance of the ConMU. For instance, increasing the data
percentile from 5 % to 25% results in a 7.89 % increase in model accuracy from
75.09 to 81.02. However, this improvement comes at the cost of a 68.17 %
increase in runtime, escalating from 38.86 seconds to 65.35 seconds.
Furthermore, we observe diminishing returns as the included data percentage
increases. Take the last two points as an example, including 10 % more data
leads to a mere 1.39 % increase in model utility but incurs a substantial 29.7
% increase in runtime, escalating from 135.43 seconds to 175.71 seconds. This
phenomenon suggests that beyond a certain threshold of included data,
sacrificing runtime yields only marginal improvements in model utility.

####  5.3.2. Utility vs Privacy

In the intricate landscape of the trilemma, another crucial facet involves the
delicate equilibrium between utility and privacy. As mentioned in Section 4.2,
the purpose of such a module is to disrupt the forgetting information in
samples, where a higher noise level indicates that the sample contains more
chaotic information, which represents better privacy. To validate this
hypothesis, we modify the mechanism’s noise level to demonstrate the
relationship between model utility and privacy. Figure 3 (b), illustrates the
performance of the proposed progressive Gaussian mechanism module under
varying noise levels. We begin with a noise level of 0, which uses the
selected data from the previous module, and increase it to a noise level of
10. As shown by the increasing FRM score, a higher noise level results in a
closer privacy level with respect to the retrained model (higher FRM score).
For example, increasing the noise level from 0 to 2 increases the FRM by 4.2
%, from 0.707 to 0.737. This enhancement, however, comes with a 0.72 %
decrease in model utility, from 81.21% to 80.62%. This trend is consistent as
the intensity of noise increases. As the noise level increases from 8 to 10,
the model test accuracy decreases from 79.59 % to 78.22%, a decrease of 1.75
%, while the FRM increases by 3.2 %. This phenomenon demonstrates the
viability of the compromise between model utility and privacy.

(a) Test Acc. vs Runtime

(b) Test Acc. vs FRM Score

(c) FRM Score vs Runtime

Figure 4.  Unlearning performance of ConMU and Naive Fine-tuning on CIFAR-10
with ResNet-18 under random forgetting request using various combinations of
controllable mechanisms. For FT, we adjust the epoch number from 5 to 35 and
try different learning rates ranging from 0.1 to 0.0001. Figure 4 (a) focuses
on evaluating the relationship between utility and runtime efficiency, where
$x$ and $y$ axes denote the test runtime and accuracy, respectively. Figure 4
(b) focuses on the relationship between utility and privacy, where $x$ and $y$
axes denote the FRM score and test accuracy, respectively. Figure 4 (c)
depicts the relationship between privacy and runtime efficiency, where $x$ and
$y$ axes denote the runtime and FRM score, respectively. The red point
represents the performance of the fine-tuning method and the blue point
denotes the ConMU.

####  5.3.3. Privacy vs Efficiency.

Lastly, there is a discernible trade-off between model privacy and runtime
performance. As mentioned in Section 4.3, we introduce an unlearning proxy to
strike a balance between these two crucial factors. This module’s purpose is
to reduce the privacy disparity between the retrained model and the unlearning
model by means of an unlearning proxy. To validate this effect, we
progressively increase the training epoch of the unlearning proxy from 0 to 8,
enabling it closer to the retrained model. As shown in Figure 3 (c), an
increase in the number of training epochs in the unlearning proxy resulted in
improved privacy performance, as indicated by a higher FRM score. Raising the
proxy training epoch from 2 to 3 increases runtime by 23.13 %, from 67.61
seconds to 83.25 seconds. This results in a 26.02 % increase in FRM score.
However, this progress diminishes when FRM exceeds 0.75. Consider the last
three data points as an illustration: a 66.1 % increase in duration from
100.47 seconds to 166.89 seconds results in a 4.64 % increase from 0.776 to
0.812. Similarly to the trade-off between utility and runtime, there exists a
threshold between privacy and runtime where sacrificing one does not result in
a substantial improvement in the other.

###  5.4. ConMU vs. Naive Fine-Tune Method

The overall performance of the rudimentary fine-tune (FT) baseline method, as
shown in Table 1, is comparable to that of the ConMU baseline method.
Consequently, an intriguing question may be posed: Can the naive FT model have
the same control ability over these trilemmas as the method demonstrated by
merely adjusting its hyperparameters? In order to answer this question, we
compare the control ability of the ConMU and naive fine-tuning. To demonstrate
this distinction in a holistic manner, we evaluate the performance of two
models based on three crucial factors: privacy (FRM), utility (TA), and
efficiency (runtime). We primarily demonstrate the control ability of the
naive fine-tuning method by varying two parameters: learning rate and fine-
tuning epochs. For the ConMU, we alter those three proposed modules.

Figure 4 (a) demonstrates the trade-off between the runtime of each sample and
the utility. In addition, figure 4 (b) illustrates the trade-off between
utility and privacy, in which greater $x$ values indicate a higher FRM score,
which corresponds to a closer level of privacy with the retrained model. As
demonstrated in Section 5.3, an expected trade-off would emerge as $x$ values
increase from left to right. Figure 4 (c) demonstrates the relationship
between privacy and runtime efficiency. Ideally, a sample that resolves the
trilemma should be placed in the top left corner of (a), the top right corner
of (b), and the top left corner of (c). Given a similar level of test
accuracy, ConMU can achieve a higher FRM score with a shorter runtime. When
the test accuracy for FT is 77.99 % and the ConMU is 78.08 %, for example, the
FRM is 0.71 and 0.87, respectively. Meanwhile, the runtime is 480.14 seconds
and 52.12 seconds, respectively, which is 9x faster. Furthermore, as test
accuracy improves, the performance of the ConMU remains relatively stable and
consistent. For instance, when the test accuracy for FT increases from 84.22 %
to 85 %, the FT’s FRM falls from 0.77 to 0.74. In contrast, the FRM for ConMU
increases by 0.3 % from 0.799 to 0.801 when the test accuracy changes from
84.32 % to 85.09 %. In terms of the runtime, the FT increases from 600.22
seconds to 720.19 seconds, whereas the ConMU only increases by only 65.55
seconds, from 207.13 to 272.68 seconds. Throughout the resulting chart, ConMU
displays its superiority not only in the stability of controlling the trilemma
but also a significant margin over the overall performance.

##  6\. Conclusion

In this paper, we identify the trilemma between model privacy, utility, and
efficiency that exists in machine unlearning for deep neural networks. To
address this issue and gain greater control over this trilemma, we present
ConMU, a novel MU calibration framework. Specifically, ConMU introduces three
control modules: the important data selection, the progressive Gaussian
mechanism, and the unlearning proxy, each of which seeks to calibrate portions
of the MU trilemma. Extensive experiments and in-depth studies demonstrate the
superiority of the ConMU across multiple benchmark datasets and a variety of
unlearning metrics. Future work could focus on extending our control mechanism
to other fields of study, such as the NLP and Graph domain.

## References

  * (1)
  * Abadi et al. (2016) Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016.  Deep learning with differential privacy. In _Proceedings of the 2016 ACM SIGSAC conference on computer and communications security_. 308–318.
  * Balle and Wang (2018) Borja Balle and Yu-Xiang Wang. 2018.  Improving the gaussian mechanism for differential privacy: Analytical calibration and optimal denoising. In _International Conference on Machine Learning_. PMLR, 394–403.
  * Basu et al. (2020) Samyadeep Basu, Philip Pope, and Soheil Feizi. 2020.  Influence functions in deep learning are fragile.  _arXiv preprint arXiv:2006.14651_ (2020).
  * Becker and Liebig (2022) Alexander Becker and Thomas Liebig. 2022.  Evaluating machine unlearning via epistemic uncertainty.  _arXiv preprint arXiv:2208.10836_ (2022).
  * Bourtoule et al. (2021) Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021.  Machine unlearning. In _2021 IEEE Symposium on Security and Privacy (SP)_. IEEE, 141–159.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _Advances in neural information processing systems_ 33 (2020), 1877–1901.
  * Bu et al. (2020) Zhiqi Bu, Jinshuo Dong, Qi Long, and Weijie J Su. 2020.  Deep learning with Gaussian differential privacy.  _Harvard data science review_ 2020, 23 (2020), 10–1162.
  * Canonne et al. (2020) Clément L Canonne, Gautam Kamath, and Thomas Steinke. 2020.  The discrete gaussian for differential privacy.  _Advances in Neural Information Processing Systems_ 33 (2020), 15676–15688.
  * Cao and Yang (2015) Yinzhi Cao and Junfeng Yang. 2015.  Towards making systems forget with machine unlearning. In _2015 IEEE symposium on security and privacy_. IEEE, 463–480.
  * Carion et al. (2020) Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. 2020.  End-to-end object detection with transformers. In _European conference on computer vision_. Springer, 213–229.
  * Chen et al. (2020) Wei-Ning Chen, Peter Kairouz, and Ayfer Ozgur. 2020.  Breaking the communication-privacy-accuracy trilemma.  _Advances in Neural Information Processing Systems_ 33 (2020), 3312–3324.
  * Chien et al. (2023) Eli Chien, Wei-Ning Chen, Chao Pan, Pan Li, Ayfer Özgür, and Olgica Milenkovic. 2023.  Differentially Private Decoupled Graph Convolutions for Multigranular Topology Protection.  _arXiv preprint arXiv:2307.06422_ (2023).
  * Chien et al. (2022) Eli Chien, Chao Pan, and Olgica Milenkovic. 2022.  Efficient model updates for approximate unlearning of graph-structured data. In _The Eleventh International Conference on Learning Representations_.
  * Chundawat et al. (2023a) Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. 2023a.  Can bad teaching induce forgetting? unlearning in deep networks using an incompetent teacher. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 37. 7210–7217.
  * Chundawat et al. (2023b) Vikram S Chundawat, Ayush K Tarun, Murari Mandal, and Mohan Kankanhalli. 2023b.  Zero-shot machine unlearning.  _IEEE Transactions on Information Forensics and Security_ (2023).
  * Cook and Weisberg (1980) R Dennis Cook and Sanford Weisberg. 1980.  Characterizations of an empirical influence function for detecting influential cases in regression.  _Technometrics_ 22, 4 (1980), 495–508.
  * Dang (2021) Quang-Vinh Dang. 2021.  Right to be forgotten in the age of machine learning. In _Advances in Digital Science: ICADS 2021_. Springer, 403–411.
  * Dong et al. (2019) Jinshuo Dong, Aaron Roth, and Weijie J Su. 2019.  Gaussian differential privacy.  _arXiv preprint arXiv:1905.02383_ (2019).
  * Dwork et al. (2006) Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. 2006.  Our data, ourselves: Privacy via distributed noise generation. In _Advances in Cryptology-EUROCRYPT 2006: 24th Annual International Conference on the Theory and Applications of Cryptographic Techniques, St. Petersburg, Russia, May 28-June 1, 2006. Proceedings 25_. Springer, 486–503.
  * Dwork et al. (2014) Cynthia Dwork, Aaron Roth, et al. 2014\.  The algorithmic foundations of differential privacy.  _Foundations and Trends® in Theoretical Computer Science_ 9, 3–4 (2014), 211–407.
  * Dwork et al. (2015) Cynthia Dwork, Adam Smith, Thomas Steinke, Jonathan Ullman, and Salil Vadhan. 2015.  Robust traceability from trace amounts. In _2015 IEEE 56th Annual Symposium on Foundations of Computer Science_. IEEE, 650–669.
  * d’Aliberti and Clark (2022) Olivia G d’Aliberti and Mark A Clark. 2022.  Preserving patient privacy during computation over shared electronic health record data.  _Journal of Medical Systems_ 46, 12 (2022), 85.
  * Feldman (2020) Vitaly Feldman. 2020.  Does learning require memorization? a short tale about a long tail. In _Proceedings of the 52nd Annual ACM SIGACT Symposium on Theory of Computing_. 954–959.
  * Gao and Wright (2023) Changyu Gao and Stephen J Wright. 2023.  Differentially Private Optimization for Smooth Nonconvex ERM.  _arXiv preprint arXiv:2302.04972_ (2023).
  * Ginart et al. (2019) Antonio Ginart, Melody Guan, Gregory Valiant, and James Y Zou. 2019.  Making ai forget you: Data deletion in machine learning.  _Advances in neural information processing systems_ 32 (2019).
  * Golatkar et al. (2021) Aditya Golatkar, Alessandro Achille, Avinash Ravichandran, Marzia Polito, and Stefano Soatto. 2021.  Mixed-privacy forgetting in deep networks. In _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_. 792–801.
  * Golatkar et al. (2020a) Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020a.  Eternal sunshine of the spotless net: Selective forgetting in deep networks. In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_. 9304–9312.
  * Golatkar et al. (2020b) Aditya Golatkar, Alessandro Achille, and Stefano Soatto. 2020b.  Forgetting outside the box: Scrubbing deep networks of information accessible from input-output observations. In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXIX 16_. Springer, 383–398.
  * Graves et al. (2021) Laura Graves, Vineel Nagisetty, and Vijay Ganesh. 2021.  Amnesiac machine learning. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 35. 11516–11524.
  * Guo et al. (2020) Chuan Guo, Tom Goldstein, Awni Hannun, and Laurens Van Der Maaten. 2020.  Certified Data Removal from Machine Learning Models. In _International Conference on Machine Learning_. PMLR, 3832–3842.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition. In _Proceedings of the IEEE conference on computer vision and pattern recognition_. 770–778.
  * He et al. (2017) Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017.  Neural collaborative filtering. In _Proceedings of the 26th international conference on world wide web_. 173–182.
  * Hoofnagle et al. (2019) Chris Jay Hoofnagle, Bart Van Der Sloot, and Frederik Zuiderveen Borgesius. 2019.  The European Union general data protection regulation: what it is and what it means.  _Information & Communications Technology Law_ 28, 1 (2019), 65–98.
  * Jacot et al. (2018) Arthur Jacot, Franck Gabriel, and Clément Hongler. 2018.  Neural tangent kernel: Convergence and generalization in neural networks.  _Advances in neural information processing systems_ 31 (2018).
  * Jia et al. (2023) Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, and Sijia Liu. 2023.  Model sparsification can simplify machine unlearning.  _arXiv preprint arXiv:2304.04934_ (2023).
  * Koh and Liang (2017) Pang Wei Koh and Percy Liang. 2017.  Understanding black-box predictions via influence functions. In _International conference on machine learning_. PMLR, 1885–1894.
  * Krizhevsky et al. (2009) Alex Krizhevsky, Geoffrey Hinton, et al. 2009\.  Learning multiple layers of features from tiny images.  (2009).
  * LeCun et al. (1989) Yann LeCun, John Denker, and Sara Solla. 1989.  Optimal brain damage.  _Advances in neural information processing systems_ 2 (1989).
  * Ma et al. (2021) Xiaolong Ma, Geng Yuan, Xuan Shen, Tianlong Chen, Xuxi Chen, Xiaohan Chen, Ning Liu, Minghai Qin, Sijia Liu, Zhangyang Wang, et al. 2021\.  Sanity checks for lottery tickets: Does your winning ticket really win the jackpot?  _Advances in Neural Information Processing Systems_ 34 (2021), 12749–12760.
  * Maas et al. (2011) Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis. In _Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies_. 142–150.
  * Molchanov et al. (2016) Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2016.  Pruning convolutional neural networks for resource efficient inference.  _arXiv preprint arXiv:1611.06440_ (2016).
  * Neel et al. (2021) Seth Neel, Aaron Roth, and Saeed Sharifi-Malvajerdi. 2021.  Descent-to-delete: Gradient-based methods for machine unlearning. In _Algorithmic Learning Theory_. PMLR, 931–962.
  * Netzer et al. (2018) Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and A Ng. 2018.  _The street view house numbers (SVHN) dataset_.  Technical Report. Technical report, Accessed 2016-08-01.[Online].
  * Nguyen et al. (2022) Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. 2022.  A survey of machine unlearning.  _arXiv preprint arXiv:2209.02299_ (2022).
  * Pan et al. (2023) Chao Pan, Eli Chien, and Olgica Milenkovic. 2023.  Unlearning graph classifiers with limited data resources. In _Proceedings of the ACM Web Conference 2023_. 716–726.
  * Pardau (2018) Stuart L Pardau. 2018.  The california consumer privacy act: Towards a european-style privacy regime in the united states.  _J. Tech. L. & Pol’y_ 23 (2018), 68.
  * Paul et al. (2021) Mansheej Paul, Surya Ganguli, and Gintare Karolina Dziugaite. 2021.  Deep learning on a data diet: Finding important examples early in training.  _Advances in Neural Information Processing Systems_ 34 (2021), 20596–20607.
  * Sekhari et al. (2021) Ayush Sekhari, Jayadev Acharya, Gautam Kamath, and Ananda Theertha Suresh. 2021.  Remember what you want to forget: Algorithms for machine unlearning.  _Advances in Neural Information Processing Systems_ 34 (2021), 18075–18086.
  * Shokri et al. (2017) Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017.  Membership inference attacks against machine learning models. In _2017 IEEE symposium on security and privacy (SP)_. IEEE, 3–18.
  * Simonyan and Zisserman (2014) Karen Simonyan and Andrew Zisserman. 2014.  Very deep convolutional networks for large-scale image recognition.  _arXiv preprint arXiv:1409.1556_ (2014).
  * Song and Mittal (2021) Liwei Song and Prateek Mittal. 2021.  Systematic evaluation of privacy risks of machine learning models. In _30th USENIX Security Symposium (USENIX Security 21)_. 2615–2632.
  * Sorscher et al. (2022) Ben Sorscher, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari Morcos. 2022.  Beyond neural scaling laws: beating power law scaling via data pruning.  _Advances in Neural Information Processing Systems_ 35 (2022), 19523–19536.
  * Tarun et al. (2023a) Ayush Kumar Tarun, Vikram Singh Chundawat, Murari Mandal, and Mohan Kankanhalli. 2023a.  Deep regression unlearning. In _International Conference on Machine Learning_. PMLR, 33921–33939.
  * Tarun et al. (2023b) Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. 2023b.  Fast yet effective machine unlearning.  _IEEE Transactions on Neural Networks and Learning Systems_ (2023).
  * Thomee et al. (2016) Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016.  YFCC100M: The new data in multimedia research.  _Commun. ACM_ 59, 2 (2016), 64–73.
  * Wang et al. (2023) Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. 2023.  KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment.  _arXiv preprint arXiv:2305.06535_ (2023).
  * Warnecke et al. (2021) Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. 2021.  Machine unlearning of features and labels.  _arXiv preprint arXiv:2108.11577_ (2021).
  * Wu et al. (2023) Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. 2023.  Visual chatgpt: Talking, drawing and editing with visual foundation models.  _arXiv preprint arXiv:2303.04671_ (2023).
  * Zhu et al. (2021) Xiaoqian Zhu, Xiang Ao, Zidi Qin, Yanpeng Chang, Yang Liu, Qing He, and Jianping Li. 2021.  Intelligent financial fraud detection practices in post-pandemic era.  _The Innovation_ 2, 4 (2021).

##  Appendix A Evaluation Metrics

Retain Accuracy (RA) and Forget Accuracy (FA) (Becker and Liebig, 2022; Jia et
al., 2023; Chundawat et al., 2023a, b; Nguyen et al., 2022; Golatkar et al.,
2021, 2020a; Tarun et al., 2023b): Retain Accuracy is the generalization
ability of $\theta_{u}$ on $D_{r}$, which is referred as the fidelity of
machine unlearning. The Forget Accuracy, on the other hand, is the
generalization ability of $\theta_{u}$ on $D_{f}$, which is the efficacy of
machine unlearning. A better $D_{f}$ score for an approximate unlearning
method should minimize the disparity with the retrained model, which is the
gold-standard.

Test Accuracy (TA): Test Accuracy (TA), unlike $D_{f}$ or $D_{r}$, assesses
the generalization ability of $\theta_{u}$ on the test dataset after
unlearning. Besides the task of class-wise forgetting, in which the forgetting
class is excluded from the testing dataset, the accuracy of the test is
determined using the entire testing dataset.

Confidence-based Membership Inference Attack (MIA) (Dwork et al., 2015; Graves
et al., 2021; Song and Mittal, 2021; Jia et al., 2023; Nguyen et al., 2022;
Chundawat et al., 2023a): Membership inference attacks determine whether a
particular training sample was present in the training data for a model. In
our work, we employed confidence-based MIA based on (Jia et al., 2023).
Formally, we first train an MIA predictor using $D_{r}$ and test sets. Then,
we apply the trained MIA predictor to $\theta_{u}$ on $D_{f}$, and we get the
following score called MIA-efficacy: $TN/|D_{f}|$. Here, $TN$ is the number of
forgetting samples that are predicted as non-training samples. A higher MIA-
efficacy score indicates that $\theta_{u}$ has less information about $D_{f}$.

Runtime Efficiency (RTE) (Jia et al., 2023; Nguyen et al., 2022; Cao and Yang,
2015): Runtime efficiency measures the computational efficiency of an MU
method. In Machine Unlearning, we want to achieve the $\theta_{r}$-like model
in less computational time.

##  Appendix B Implementation Details

###  B.1. Baseline Descriptions

Influence unlearning (Koh and Liang, 2017; Nguyen et al., 2022; Cook and
Weisberg, 1980; Jia et al., 2023; Golatkar et al., 2021): The influence
function is a technique that allows us to understand how model parameters
change by up-weighting training data points, and the effect of these data
points can be estimated in closed form. In machine unlearning, we can estimate
$|\theta_{o}-\theta_{r}|$ if $D_{r}$ is removed from $D_{o}$ using influence
functions. Also, influence unlearning assumes that the empirical risk is
twice-differentiable and strictly convex in $\theta_{o}$. However, for deep
neural networks, due to the non-convexity of the loss function and
approximating the inverse-Hessian vector product can be erroneous (Basu et
al., 2020; Koh and Liang, 2017; Nguyen et al., 2022), using influence
functions to approximate $\theta_{r}$ is not effective in practice.

Gradient Ascent (Jia et al., 2023; Graves et al., 2021): Gradient Ascent is
doing exactly the opposite of what gradient descent tries to do. It reverses
the model training on $D_{f}$ during training back to the $\theta_{o}$. To be
more specific, gradient ascent approaches move $\theta_{o}$ towards loss for
data points to be erased.

Regular Fine-tuning (Warnecke et al., 2021; Golatkar et al., 2020a; Jia et
al., 2023): Different from naively retraining from the scratch, FT fine-tunes
the the pre-trained $\theta_{o}$ on $D_{r}$ for a few epochs, with a slightly
larger learning rate. The motivation is that fine-tuning on $D_{r}$ may cause
catastrophic unlearning over $D_{f}$.

###  B.2. Hyperparameters

The hyper-parameters of ConMU are listed in Table 2 for random forgetting and
Table 3 for class-wise forgetting. For random forgetting, we randomly selected
20% of the original training data as the forgetting dataset. For class-wise
forgetting, we randomly selected 50% of a particular class as the forgetting
data. Due to the limitation of the GPU memory, the batch size is restricted to
128. Note that the $\alpha$ is generally larger for class-wise forgetting than
that for random forgetting.

Table 2. Hyper-parameters of ConMU for CIFAR-10, CIFAR-100 and svhn datasets
on ResNet18 and VGG for random forgetting. The $z_{1}$ and $z_{2}$ indicate
the lower and upper bounds of important data selection, respectively. The
$\alpha$ is the amount of Gaussian noise used in the Progressive Gaussian
Mechanism. The Gaussian noise has a mean $\mu$ and a standard deviation
$\sigma$. $\delta$ is the number of epochs that the unlearning proxy model was
trained on. $\gamma$ is the coefficient of the KL loss term.

Model | ConMU (ResNet-18) | ConMU (VGG)
---|---|---
Dataset | CIFAR-10 | CIFAR-100 | svhn | CIFAR-10 | CIFAR-100 | svhn
Fine-Tune Epoch | 5 | 5 | 5 | 5 | 5 | 5
Learning Rate | 1e-2 | 1e-2 | 1e-2 | 1e-2 | 1e-2 | 1e-2
Batch Size | 128 | 128 | 128 | 128 | 128 | 128
Optimizer | SGD | SGD | SGD | SGD | SGD | SGD
Retain, $z_{2}$ | 0.3 | 0.25 | 0.28 | 0.2 | 0.45 | 0.3
Retain, $z_{1}$ | 0.17 | 0.16 | 0.1 | 0.16 | 0.35 | 0.2
Forget, $z_{2}$ | 1.0 | 0.2 | 0.4 | 0.2 | 0.4 | 0.4
Forget, $z_{1}$ | 0.85 | 0.18 | 0.3 | 0.15 | 0.37 | 0.3
$\alpha$ | 3 | 4 | 5 | 3 | 5 | 5
$\mu$ | 0 | 0 | 0 | 0 | 0 | 0
$\sigma$ | 1 | 1 | 1 | 1 | 1 | 1
$\delta$ | 1 | 1 | 1 | 1 | 1 | 1
$\gamma$ | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5

Table 3. Hyper-parameters of ConMU for CIFAR-10, CIFAR-100 and svhn datasets
on ResNet18 and VGG for class-wise forgetting. The forgotten Class is the
class index of the dataset we chose to forget for the experiments. The $z_{1}$
and $z_{2}$ indicate the lower and upper bounds of important data selection,
respectively. The $\alpha$ is the amount of Gaussian noise used in the
Progressive Gaussian Mechanism. The Gaussian noise has a mean $\mu$ and a
standard deviation $\sigma$. $\delta$ is the number of epochs that the
unlearning proxy was trained on. $\gamma$ is the coefficient of the KL loss
term.

Model | ConMU (ResNet-18) | ConMU (VGG)
---|---|---
Dataset | CIFAR-10 | CIFAR-100 | svhn | CIFAR-10 | CIFAR-100 | svhn
Forgotten Class | 5 | 69 | 5 | 5 | 69 | 5
Fine-Tune Epoch | 5 | 5 | 5 | 5 | 5 | 5
Learning Rate | 1e-2 | 1e-2 | 1e-2 | 1e-2 | 1e-2 | 1e-2
Batch Size | 128 | 128 | 128 | 128 | 128 | 128
Optimizer | SGD | SGD | SGD | SGD | SGD | SGD
Retain, $z_{2}$ | 0.3 | 0.3 | 0.3 | 0.4 | 0.45 | 0.3
Retain, $z_{1}$ | 0.2 | 0.1 | 0.2 | 0.3 | 0.35 | 0.2
Forget, $z_{2}$ | 1.0 | 0.4 | 0.8 | 0.8 | 1.0 | 0.8
Forget, $z_{1}$ | 0.8 | 0.3 | 0.5 | 0.5 | 0.85 | 0.5
$\alpha$ | 12 | 1 | 4 | 20 | 10 | 4
$\mu$ | 0 | 0 | 0 | 0 | 0 | 0
$\sigma$ | 1 | 1 | 1 | 1 | 1 | 1
$\delta$ | 1 | 1 | 1 | 1 | 1 | 1
$\gamma$ | 0.5 | 0.5 | 0.5 | 0.5 | 0.5 | 0.5

##  Appendix C Additional Experiments

The additional experiments are reported in Table 4, which has the same setup
as Table 1.

Table 4. Additional results of our proposed ConMU, with a number of baselines
under two unlearning scenarios: random forgetting and class-wise forgetting.
Bold indicates the best performance and underline indicates the runner-up. The
efficacy of each MU method is evaluated under five metrics: test accuracy
($TA$), accuracy on forget data ($FA$), accuracy on retain data ($RA$),
membership inference attack ($MIA$), and running time efficiency ($RTE$). The
performance of ConMU is reported in the form of $a_{\pm b}$, where $a$ is the
mean value of independent 10 trial runs and $b$ denotes the standard
deviation. A performance difference against the retrained model is reported in
$({\color[rgb]{0,0,1}\bullet})$.

MU Methods | $TA(\%)\uparrow$ | $FA(\%){\color[rgb]{0,0,1}\downarrow}$ | $RA(\%){\color[rgb]{0,0,1}\downarrow}$ | $MIA(\%){\color[rgb]{0,0,1}\downarrow}$ | $RTE(s)$ |  $FRM$ $Privacy$ $\uparrow$
---|---|---|---|---|---|---
Resnet-18 Random data forgetting (CIFAR-100)
retrain | $51.45$ |  $40.82$ ($0.00$) |  $99.97$ ($0.00$) |  $49.34$ ($0.00$) | $1066.69$ | 1
IU + Pruning | $6.48_{\pm 0.02}$ |  $6.03_{\pm 0.02}$ ($38.84$) |  $6.02_{\pm 0.00}$ ($93.95$) |  $92.92_{\pm 0.02}$ ($43.58$) | 38.36 | 0.069
GA + Pruning | $31.33_{\pm 0.19}$ |  $\textbf{31.26}_{\pm 0.13}$ ($9.56$) |  $31.64_{\pm 0.18}$ ($68.33$) |  $67.51_{\pm 0.13}$ ($18.17$) | 54.58 | 0.276
FT + Pruning | $\textbf{56.36}_{\pm 0.83}$ |  $54.87_{\pm 0.87}$ ($14.05$) |  $\textbf{70.55}_{\pm 0.89}$ ($29.42$) |  $\underline{45.18}_{\pm 0.87}$ ($4.16$) | 282.09 | 0.484
ConMU | $\underline{48.97}_{\pm 1.78}$ |  $\underline{54.95}_{\pm 4.49}$ (14.13) |  $\underline{61.25}_{\pm 2.96}$ (38.72) |  $\textbf{45.30}_{\pm 4.60}$ ($4.04$) | 27.83 | 0.443
Resnet-18 Class-Wise forgetting (CIFAR-100)
retrain | 56.43 | 60.89 ($0.00$) | 99.98 ($0.00$) | 39.12 ($0.00$) | 1323.98 | 1
IU + Pruning | $6.52_{\pm 0.00}$ |  $8.89_{\pm 1.11}$ ($52.00$) |  $6.05_{\pm 0.01}$ ($93.93$) |  $94.01_{\pm 1.11}$ ($54.89$) | 39.14 | 0.041
GA + Pruning | $20.18_{\pm 0.36}$ |  $7.78_{\pm 0.66}$ ($53.11$) |  $19.53_{\pm 0.40}$ ($80.45$) |  $93.37_{\pm 0.66}$ ($54.25$) | 22.57 | 0.047
FT + Pruning | $\textbf{60.07}_{\pm 0.87}$ |  $\textbf{76.67}_{\pm 5.58}$ ($15.78$) |  $\underline{73.87}_{\pm 0.88}$ ($26.11$) |  $\underline{24.42}_{\pm 5.58}$ ($14.70$) | 364.23 | 0.408
ConMU | $\underline{55.22}_{\pm 0.32}$ |  $\underline{61.53}_{\pm 2.58}$ ($28.27$) |  $\textbf{71.77}_{\pm 1.16}$ ($0.023$) |  $\textbf{41.46}_{\pm 2.31}$ ($2.34$) | 48.23 | 0.704
VGG Random data forgetting (SVHN)
retrain | $93.53$ |  $93.47$ ($0.00$) |  $99.62$ ($0.00$) |  $6.58$ ($0.00$) | $840.33$ | 1
IU + Pruning | $90.18_{\pm 0.00}$ |  $92.30_{\pm 0.00}$ ($1.17$) |  $91.96_{\pm 0.00}$ ($7.66$) |  $\textbf{8.07}_{\pm 0.00}$ ($1.49$) | 49.30 | 0.726
GA + Pruning | $92.26_{\pm 0.04}$ |  $\textbf{94.34}_{\pm 0.03}$ ($0.87$) |  $94.38_{\pm 0.03}$ ($5.24$) |  $94.09_{\pm 0.03}$ ($87.51$) | 26.40 | 0.000
FT + Pruning | $\textbf{94.19}_{\pm 0.14}$ |  ${95.71}_{\pm 0.10}$ ($2.24$) |  $\textbf{99.65}_{\pm 0.00}$ ($0.03$) |  $4.34_{\pm 0.10}$ ($2.24$) | 280.15 | 0.696
ConMU | $\underline{90.95}_{\pm 0.67}$ |  $\underline{91.93}_{\pm 0.97}$ (1.54) |  $\underline{92.51}_{\pm 1.09}$ (7.11) |  $\underline{8.13}_{\pm 0.12}$ ($1.55$) | $79.71$ | 0.716
VGG Class-Wise forgetting (SVHN)
retrain | 94.12 | 90.21 ($0.00$) | 99.55 ($0.00$) | 9.80 ($0.00$) | 1048.65 | 1
IU + Pruning | $90.08_{\pm 0.01}$ |  $\underline{92.83}_{\pm 0.01}$ ($2.62$) |  $91.92_{\pm 0.15}$ ($7.63$) |  $\underline{7.21}_{\pm 0.15}$ ($2.59$) | 35.01 | 0.624
GA + Pruning | $81.38_{\pm 0.26}$ |  $31.64_{\pm 0.76}$ ($58.57$) |  $86.45_{\pm 0.25}$ ($13.10$) |  $68.42_{\pm 0.76}$ ($58.62$) | 27.99 | 0.057
FT + Pruning | $\textbf{95.02}_{\pm 0.15}$ |  $94.87_{\pm 0.57}$ ($4.66$) |  $\textbf{99.99}_{\pm 0.00}$ ($0.44$) |  $5.11_{\pm 0.57}$ ($4.69$) | 320.77 | 0.3772
ConMU | $\underline{92.31}_{\pm 0.61}$ |  $\textbf{88.43}_{\pm 2.89}$ ($1.78$) |  $\underline{94.73}_{\pm 1.23}$ ($4.82$) |  $\textbf{10.59}_{\pm 2.91}$ ($0.79$) | $85.05$ | 0.864
Resnet-18 Random data forgetting (SVHN)
retrain | $91.05$ |  $91.68$ ($0.00$) |  $99.45$ ($0.00$) |  $8.35$ ($0.0$) | $954.66$ | 1
IU + Pruning | $35.04_{\pm 0.00}$ |  $36.45_{\pm 0.00}$ ($55.23$) |  $36.78_{\pm 0.00}$ ($62.67$) |  $63.70_{\pm 0.00}$ ($55.35$) | 40.52 | 0.000
GA + Pruning | $85.35_{\pm 0.05}$ |  $87.87_{\pm 0.03}$ ($3.81$) |  $87.51_{\pm 0.03}$ ($11.94$) |  $12.17_{\pm 0.03}$ ($3.82$) | 57.55 | 0.538
FT + Pruning | $\underline{92.95}_{\pm 2.89}$ |  $\underline{93.34}_{\pm 1.82}$ ($1.66$) |  $\textbf{100.00}_{\pm 0.00}$ ($0.55$) |  $\underline{6.63}_{\pm 1.82}$ ($1.72$) | 729.64 | 0.795
ConMU | $\textbf{92.98}_{\pm 0.43}$ |  $\textbf{93.05}_{\pm 0.38}$ (1.37) |  $\underline{93.84}_{\pm 0.34}$ (5.61) |  $\textbf{7.02}_{\pm 0.42}$ ($1.33$) | 74.36 | 0.796
Resnet-18 Class-Wise forgetting (SVHN)
retrain | 91.40 | 83.73 ($0.00$) | 98.71 ($0.00$) | 16.21 ($0.00$) | 1075.59 | 1
IU + Pruning | $32.68_{\pm 0.05}$ |  $13.14_{\pm 0.41}$ ($70.59$) |  $35.48_{\pm 0.07}$ ($63.23$) |  $86.87_{\pm 0.41}$ ($70.66$) | 41.08 | 0.003
GA + Pruning | $73.96_{\pm 0.10}$ |  $28.25_{\pm 0.79}$ ($55.48$) |  $77.42_{\pm 0.11}$ ($21.29$) |  $71.72_{\pm 0.79}$ ($55.51$) | 19.28 | 0.014
FT + Pruning | $\textbf{93.39}_{\pm 0.32}$ |  $\textbf{90.57}_{\pm 1.21}$ ($6.84$) |  $\textbf{100}_{\pm 0.00}$ ($1.29$) |  $\underline{9.40}_{\pm 1.21}$ ($6.81$) | 458.9 | 0.598
ConMU | $\underline{92.15}_{\pm{0.52}}$ |  $\underline{88.44}_{\pm 2.69}$ ($4.71$) |  $\underline{94.34}_{\pm 1.07}$ ($4.37$) |  $\textbf{11.64}_{\pm 2.72}$ ($4.57$) | $51.37$ | 0.681


# Species Syzygy: Which Animal Has Seen the Most Total Solar Eclipses?

Mark Popinchalk<EMAIL_ADDRESS>
(April 1st 2024)

###### Abstract

A total solar eclipse (TSE) is a shocking and sublime experience. In just a
week hundreds of millions of Homo Sapiens will attempt to see the 2024 eclipse
as it stretches across the North American continent. However, while Homo
Sapiens may be uniquely positioned to understand and predict eclipses, they
are not the only species capable of observing them. The precise alignment of
the Moon, Earth and Sun all existed well before humans. In the same way we
share this planet capable of hosting life, the fantastic astronomical
experiences available on it are not exclusive either. We present a framework
to calculate the number of Total Solar Eclipses experienced by a species at
any point in Earth’s history. This includes factoring in the evolution of the
Sun-Moon-Earth system, the duration the species is extant, and average
population. We normalize over the geographic range by calculating an
Astronomical World Eclipse Surface cOverage MEtric (AWESOME) time. To
illustrate this framework we look at the case study of the family Limulidae
(Horseshoe Crabs) and estimate the number of individuals that have seen an
eclipse. We compare it to the number of current Homo Sapiens that view
eclipses, and predict if it is possible for another species to take the ”top”
spot before the final total solar eclipse in $\sim$ 380 million years.

##  1 Introduction

A total solar eclipse (TSE) is when the alignment of the Earth, Moon and Sun
is such that from a perspective on Earth, the disc of the Sun is completely
obscured by the Moon. It is a geometric arrangement caused by the Moon and Sun
having comparable angular sizes. It is a true marvel that this happens at all,
let alone with regularity when you start to consider all the orbital
variations involved. Apsidial precession of the Earth and the Moon, Nodal
precession of the Moon, eccentricity variations of the Moon and Earth are all
really challenging to keep track of, especially for an April Fools day paper.

In addition, the Earth moon system is evolving. The tidal effects of the Moon
on the Earth create a torque on the Earth that both slows down the day and
causes the Moon to slowly retreat. This change in orbit structure has been
explored in other well regarded papers (Popinchalk, 2023), but it’s effect on
Total Solar Eclipses has not been studied thoroughly.

There has been a lot of wildlife to witness those Total Solar Eclipses. While
there is evidence of microbial life early in the Earth’s history, it is near
impossible to say much about them. However, the Cambrian Explosion $\sim$ 500
million years ago introduced a wide array of animals onto the surface of the
EarthErwin & Valentine (2013).

Determining if an animal is aware of an eclipse can be challenging. Anecdotal
stories of birds going to roost or performing night calls are common, but
quantifying or defining ”observation” from these stories is challenging.
However, recently the study of animal reactions to total solar eclipses has
been possible to study due to eclipse paths aligning with zoos in metropolitan
areas. Hartstone-Rose et al. (2020) tracked the response of 17 taxa of animals
during thee 2017 TSE in Riverbanks Zoo and Garden in Columbia, South Carolina.
13 of the 17 acted differently from their baseline behavior, with 8 performing
nighttime routines. The next most frequent behavior was anxiety, specifically
noted in a few primates (both gorillas and baboons), matching the common
reaction of humans prior to the last few centuries.

These are only the outward behaviors of the animals, and do not consider the
inner workings of their minds. For example, Hartstone-Rose et al. (2020)
observed the Galapagos turtles turning to look skyward. Were they
contemplating the eclipse? While there are contested reports of understanding
animal communication Lofting & Dolittle (1920), we are unable to know for
sure. Even if they don’t understand what a Total Solar Eclipse is, they have
borne witness.

We therefore aspire in this paper to provide a framework for calculating how
many members of a species have witnessed a Total Solar Eclipse. In Section 2
we present the biological considerations necessary to discussing animals
witnessing TSE on astronomical timescales ($\sim$ 10 Myr). Section 3 we
describe our method for calculating the number of TSE that will happen in a
certain epoch in Earth’s history. In Section 4 we calculate the number of TSEs
experienced by Horseshoe Crabs and by Humans as a species, and in Section 5 we
discuss our findings and offer conclusions in Section 6.

##  2 Lifespan of Species and Estimation of Animal Populations

We will start by taking a liberal approach to the definition of ”observing” to
include animals capable of distinguishing between light and dark, and carrying
behaviors associated with each. Some might complain that this is too broad,
but if you’ve ever been in the path of totality, you can’t help but notice it.
We hope that any critics of this choice will some day see the lack of light of
day for themselves.

###  2.1 Species Extinction Rate

99.99$\%$ of all species have gone extinctLawton & May (1995). While this
sounds like a bummer, it is a natural part of evolution. Most animals tend to
exist on the order of 1-10 million years, and so that is the characteristic
timescale we will consider. The exception being our case study Horseshoe Crabs
(see Section 2.4).

###  2.2 Geographic Range of Species

Animals obviously live in a variety of conditions and geographic regions on
the Earth. Most are conducive to witnessing a TSE, as while they are varied
they are all generally considered to be ”outside”. However, bodies of water
represent something one can be inside of, and therefore we must account for
the time spent in water for a given animal.

Attempting to calculate the number of eclipses for a specific location would
be extremely challenging, especially since tectonic shift would move those
locations or render them below the crust on the timescales we are concerned
with. Thankfully, our method is independent of specific location, as you will
see in Section 3.2.1.

###  2.3 Animal Population and Anthropogenic Effects thereof

To accurately gauge the number of TSEs members of a species have witnessed,
the population of said species needs to be known. In particular, we need an
estimate of the average population of a species over 10 Million year time
spans. One issue for estimating this is that since humans have been able to
count wild animals, they have been killing them.

This is not hyperbole.

100,000 years ago, after modern humans existed (Vidal et al., 2022), there was
20 million tonnes of carbon in wild land mammals of which humans were
negligible amount. By 2015 Humans consisted of 60 million tonnes and wild land
mammals only 3 million (Ritchie, 2022). Humans have without debate diminished
populations and extinguished species, not only of mammals but across the
animal kingdom. This makes present day population counts of our fellow species
vast underestimates of any prior epoch.

###  2.4 Case Study: Horseshoe Crabs

We now apply the following considerations to a case study, Horseshoe Crabs.
First and foremost, Horseshoe Crabs are capable of observing a Total Solar
Eclipse. Horseshoe crabs have 10 total eyes, including two compound eyes and
two sensitive to UV light, specifically used to track the Sun and reflected
light from the Moon (Walls et al., 2002).

They spawn on the beach with the spring tide during the months of March and
July (Shuster & Botton, 1985). The rest of the time they spend approximately
100 km offshore scavenging for food on the ocean floor (Zhu et al., 2020).

Most importantly, Horseshoe Crabs are considered living fossils. Since they
emerged in the fossil record 480 million years ago, they have changed very
little in function or form (Werth & Shear, 2014). In the last 200 million
years they have essentially seen next to no evolution. There are currently 4
extant species, located on the east coasts of North America and Asia. While no
single species can be traced back to 480 million years ago, to serve as a
demonstration of our method we consider Horseshoe Crabs together, the family
Limulidae.

In Figure 1 we present a typical specimen of Limulus polyphemus, the Atlantic
Horseshoe Crab, wearing eclipse glasses on 8 of its 10 eyes.

Figure 1: An image of an Atlantic Horseshoe Crab, Limulus polyphemus, wearing
proper eye protection for viewing the Sun before a Total Solar Eclipse.

##  3 Frequency of Total Solar Eclipses over Earth’s History

As outlined in the introduction, a Total Solar Eclipse requires a precise
alignment of the Earth Moon and Sun. However, while the orbital mechanics of
the Earth Moon and Sun vary over time, we are going to sweep most of them
under the rug. Precession rates, Saros cycles and eccentricity variations are
all on the order of tens to hundreds of thousands of years. We will be working
in 10 million year time steps, and so any effect due to the mentioned will be
averaged out (e.g. Laskar et al., 2004).

We will state the average eccentricities of the Earth and Moon orbit,
currently 0.017 and 0.0549 respectively. These are important and fun when
figuring out when a TSE happens because the angular size of the Moon is so
close to that of the Sun, the slight difference in distance between the Sun-
Earth and Earth-Moon due to these eccentricities will change the angular size
of the Sun or Moon. This is optimized for a TSE when the Earth is at aphelion,
and Moon is at perigee, making the Sun its smallest and the Moon its biggest.

###  3.1 An Orbital Argument to define TSE Frequency

Figure 2: A model of the Moon’s orbit around the Earth for the present day.
The blue line denotes the elliptical orbit. The gray dashed line is the
maximum radial distance the Moon can be for their to be a TSE if the Earth is
at its average distance from the Sun. The grey shell represents how much this
radius changes as the Earth moves from aphelion to perihelion. Green points
are sampled position on the orbit that are within the minimum radius, and red
points are those outside of it. The blue x represent the Earths postion.

The common heuristic is that TSE currently happen roughly every 18 months. Not
satisfied in doing things quickly, we made a simple model of the Moon’s orbit
(don’t worry, we’re gonna need this later). In Figure 2 we see the elliptical
orbit of the Moon at its current semi major axis of 239000 miles. The grey
dashed line is a maximum orbital distance from Earth for which the Moon’s
angular size allows for a Total Solar Eclipse when the Earth is its average
distance from the Sun, approximately 234400 miles.

We then place representative ”Moons” around the orbit, and note if they are
within this minimum radius or outside of it. If we assume that each point in
the Moon’s orbit is equally likely to be aligned during syzysgy with the Earth
and Sun, them the fraction of number of points within divided by the total
number of points is a simple means of calculating how frequently a TSE occurs.
In our simple model, 18 points are outside, and 10 are in, so if any apsis in
the orbit is taken at random approximately 35$\%$ of them could be a TSE.
However, there are 2 opportunities for syzygy in a year so at random there are
0.7 TSE per year, or one TSE every 1.4 years.

###  3.2 Current World TSE Rate

With our estimate roughly matching the heuristic for TSE frequency, we need to
check if there is a location bias for TSE occurrence. Meeus (1982) used a
statistical approach, tracking how often 408 standard points on the globe
experienced an eclipse between 1799 and 2299 AD. In this 600 year span, they
found that on average any given point on the earth experiences a TSE every 375
years. Furthermore, Wright (2024) took the NASA catalog of 5000 years of
eclipses Espenak & Meeus (2009) and tracked how often any of the 3743 Total or
Hybrid eclipse paths landed on each of the $\sim$ few square mile pixels. They
found a similar average of 366, mostly independent of longitude.

From these results we take two conclusions. In the 5000 years eclipse catalog,
there were $\sim$ 3000 Total eclipses (Hybrid eclipses are edge cases we
choose to ignore). This is a rate of 0.6 TSE per year, which roughly matches
our value from Section 3.1. Furthermore, the average of 366 years between TSE
from Meeus (1982) and 375 from Wright (2024) inspire us to create a new term
which we dub the Astronomical World Eclipse Surface cOverage MEtric (AWESOME)
time.

####  3.2.1 AWESOME Time

The AWESOME Time represents how long it takes for the entire surface of the
Earth to experience a TSE on average. As a proof, we start by estimating the
surface area of paths of totality from all the TSE in the Espenak & Meeus
(2009) catalog. Most TSE paths are 100 miles across, and stretch 10,000 miles
from one end to the other across the curvature of the Earth’s surface.
Therefore, on average a TSE path covers $\sim$ 1 million square miles. The
Earth’s surface area is 196 square miles, so the 3000 TSEs from the 5000 years
catalog covered the Earth’s surface $\sim$ 15 times. If we divide the number
of years to find how long on average it takes to cover the Earth’s surface one
time, we get an AWESOME time of 326 years.

We compare our work to others. Using the same catalog, Wright (2024) mapped
all of the paths of totality and found the average number of times each pixel
was on a path was 16 over the 5000 years, comparable to our 15. Wright (2024)
calculated his own version of the AWESOME time at 375 years, and Meeus (1982)
found a value of 366 years.

Satisfied with out definition, we must realize the AWESOME time is extremely
powerful. Now we have a time period at which the entire globe will experience
a TSE within, as long as we average it out over a much greater time period.
This allows us to ignore the geographical ranges of animals as we can assume
their entire range will experience on average one TSE at some point within an
AWESOME time. Whether their population is located at a single point or
dispersed evenly around the globe, by the end of an AWESOME time, on average
their entire population will have experienced 1 TSE across the species.

###  3.3 Lunar Orbital Retreat effects on AWESOME time

The lunar distance from the Earth will dramatically effect the AWESOME time
over the course of Life’s history. A closer Moon than today’s will 1) have a
TSE more often and 2) have a wider path of totality.

To account for this, we use the orbital model from 3.1 along with a linear
estimation of the Moon’s semi major axis evolution from Williams & Boggs
(2016) of 3.8 cm/yr to calculate the Moon’s semi major axis over time. This
rate is consistent with more advanced models of lunar recession (e.g. Farhat,
Mohammad et al., 2022) over the billion years centered on today.

Figure 3: Same as for Figure 2 but using a Moon semi-major axis for 480
million years ago. Notice the different number of red and green points.

Figure 3 shows the same set up as Figure 2 but places the Moon as the semi-
major axis when Horseshoe crabs first appear in the fossil record 480 million
years ago Werth & Shear (2014). Notice the frequency of total solar eclipses
is far higher ($\sim$ 1.2 TSE per year). The much closer Moon means the Umbra
and path of totality will be larger too. We estimate the Umbra radius on the
Earth’s surface using

$r=R_{m}-\frac{d_{m}-Re}{d_{e}-d_{m}}*(R_{s}-R_{m})$

This is a geometric definition and doesn’t account for the curvature of the
Earth surface beneath the shadow, which can project the shadow over a greate
distance. When we applied it to our modern model of the Moon orbit, we found
we need to multiply the diameter of our Umbras by a fudge factor of 1.723 to
match the $\sim$ 100 mile typical path of totality width. We apply the same
fudge factor to totality paths at other epochs.

##  4 Calculating TSE per Species

We can now calculate the AWESOME time across the history of life on Earth. We
divide up the last 500 million years in to 10 million year bins to calculate
AWESOME times during. We also predict 500 million years into the future.

At each time step we make a Moon orbit model presented in Sections 3.1 and
3.3. From this we calculate the number of orbit positions within the minimum
radius for a TSE, and calculate the time between eclipses (Sections 3.1) and
average Umbra shadow radius which we convert with the fudge factor in to a
surface area covered by the path of totality (Section 3.3). We then divide the
Earth’s surface area by the surface area of an average path of totality and
multiply by the time between eclipses to arrive at an AWESOME time.

###  4.1 Results

Figure 4: A plot of AWESOME Time (the average time between Total Solar
Eclipses for a point on the surface of the Earth) over 10 Myr periods in the
500 million years preceding and succeeding the present.

We present the AWESOME time, the average time between Total Solar Eclipses for
a spot on the Earth over 10 Million year increments of Earth’s history in
Figure 4. The AWESOME time was much shorter earlier in Earth’s history. 500
million years ago the frequency of eclipses was an average of every 7 months
with 200 mile wide paths of totality. On average it took 77 years for every
place on the Earth to experience a True Solar Eclipse.

The AWESOME time exponentially grows over the course of the billion years
displayed. In 370 million years, it will take 56000 years on average between
successive coverings of the Earth by Total Solar Eclipses. The time between
eclipses will be 7.5 years on average, but the width of the path of totality
will only be $\sim$ 3 miles wide. The final Total Solar Eclipse will occur
approximately 380 million years from now.

Figure 5: A plot of the number of Global Total Solar Eclipses Coverages for
the Earth during a 10 Myr period in the 500 million years preceding and
succeeding the present. This is how many times the surface of the Earth will
be covered by TSE, and also the average of how many TSE a single point on the
Earth will experience in that 10 Myr period.

In Figure 5 we present the number of TSEs a point on the Earth will experience
on average during 10 million year bins in Earth’s history, or how many times
the Earth will be wrapped in TSE during that time. We do this by dividing the
10 million year bins by their AWESOME Time. We see the number of eclipses drop
exponentially, until at 380 million years in the future there is a final Total
Solar Eclipse, and then no more.

####  4.1.1 Horseshoe Crabs and Humans

In Figure 6 we present the cumulative number of TSEs observed by Horseshoe
Crabs and Humans. It’s important to note that we attempted to factor in
Horseshoe Crab behavior into this result. Horseshoe Crabs leave the ocean
floor and spend time on the beach to mate over the mating season, March
through July (Shuster & Botton, 1985). We therefore applied a factor of 5/12
to the totals in Figure 5 to account for the months out at sea. Horseshoe
crabs time their beach rendezvous with the spring tide caused by the New and
Full Moon, so it is not unrealistic that they are likely to be on the beach
during TSEs that fall during mating months.

Figure 6: A plot of the cumulative number of Total Solar Eclipses the family
Limulidae, Horseshoe Crabs has experienced. Starting from 480 million years
ago, the number of TSE in Figure 5 in each bin is added to the total. A factor
of 5/12 is also applied, representing that only 5 of the Months of the year
are Horseshoe Crabs attempting to mate on the beaches and able to observe a
TSE. The total number of TSE for Humans is also presented.

To date, the family Limulidae has witnessed 1.5 million eclipses across the
species. This number is not the total that have happened, but given a
dispersed population, their entire range has been covered that many time.
Therefore, we can multiply this number by an assumed steady population and get
a number for the total number of individual eclipse experiences. Good
Horseshoe population data starts in what must be assumed is the middle of a
massive drop off (see Section 2.3) but in 1871 4 million Horseshoe Crabs were
”harvested” in Delaware and New Jersey. We can assume the population was much
higher before records of fishing were kept, and while this is a major breeding
ground for Atlantic Horseshoe Crabs, they do spread from Maine to Mexico.
Therefore we estimate a conservative standing population of 40 million
Horseshoe crabs for that species, and naively assume the same for the other 3
extant species.

If a standing population of 120 million Horseshoe Crabs witnessed each of the
1.5 million eclipses, that would make a total of 138 trillion Total Solar
Eclipse experiences for Horseshoe Crabs.

In comparison, Humans have witnessed just 32000 TSE across the species, and
that includes extending the definition of Humans back to the last common
ancestor of Humans and Chimpanzees 10 million years ago. Human population
numbers have increased exponentially in the last 10,000 years, but prior to
that were on the order of tens to hundreds of thousands (Sjödin et al., 2012).

We assume a standing population of 1 million to keep things simple, meaning
across the Human species there have been 32 billion TSE experiences.

##  5 Discussion

Given that Horseshoe Crabs have experience over 3 orders of magnitude more
TSEs than Humans, can we catch up? The 2017 Total Solar Eclipse had an
estimated 215 million participants. If humanity can mobilize that number of
people to each upcoming eclipse (including those over the ocean) we can
overtake Horseshoe Crabs in $\sim$ 400000 years.

This does not seem feasible however, and if we return to the approach above
and set a new standing population of 10 billion individuals and predict the
number of eclipses across species, we would overtake the current Horseshoe
Crab TSE experiences in under 10 million years.

But at what cost does that Human population entail? Although a potential
synthetic replacement is being developed, the population of Horseshoe Crabs
continues to suffer due to harvesting for their blood as an important tool in
pharmaceuticals (Maloney et al., 2018). The Delaware population is struggling
to maintain 700000 members, down from estimated tens of millions. Our global
civilization exists due to the exploitation and extermination of resources and
environments that have sustained other species for millions of years. A total
solar eclipse is worth sharing.

##  6 Conclusions

The AWESOME time is the average time between TSEs for a given point on the
Earth, as well as describing the average time it takes the whole Earth to
experience one TSE.

Total Solar Eclipses have evolved over the Earth’s history, and the final
Total Solar Eclipse will be in approximately 380 Million Years. If we want
more eclipses after that, we will need to move the Earth, Moon or Sun.

We have presented a framework to calculate the number of Total Solar Eclipse
experiences for any given species in Earth’s history.

We found that Horseshoe Crabs across the species have had 138 Trillion Total
Solar Eclipse experiences, compared to Humans with only 38 Billion. However,
if population levels maintain, Humans will overtake them in less than 10
million years.

The author would like to thank J.W., E.G., W.D., D.V., and A.M. for humoring
him 7 years ago and agreeing to go witness his first total eclipse. He would
also like to proactively thank all those he will be dragging to future ones.

## References

  * Erwin & Valentine (2013) Erwin, D., & Valentine, J. 2013, The Cambrian explosion: the construction of animal biodiversity (Greenwood Village, Colorado: Roberts and Company)
  * Espenak & Meeus (2009) Espenak, F., & Meeus, J. 2009, NASA technical publication TP-2009-214174, NASA.  https://eclipse.gsfc.nasa.gov/SEpubs/5MKSE.html
  * Farhat, Mohammad et al. (2022) Farhat, Mohammad, Auclair-Desrotour, Pierre, Boué, Gwenaël, & Laskar, Jacques. 2022, A&A, 665, L1, doi: 10.1051/0004-6361/202243445
  * Hartstone-Rose et al. (2020) Hartstone-Rose, A., Dickinson, E., Paciulli, L. M., et al. 2020, Animals : an open access journal from MDPI, 10, doi: 10.3390/ani10040587
  * Laskar et al. (2004) Laskar, J., Robutel, P., Joutel, F., et al. 2004, A&A, 428, 261, doi: 10.1051/0004-6361:20041335
  * Lawton & May (1995) Lawton, J. H., & May, R. 1995, Extinction Rates (Oxford: Oxford University Press)
  * Lofting & Dolittle (1920) Lofting, H., & Dolittle, J. 1920, The Story of Doctor Dolittle (New York: Stokes)
  * Maloney et al. (2018) Maloney, T., Phelan, R., & Simmons, N. 2018, PLoS biology, 16, e2006607, doi: 10.1371/journal.pbio.2006607
  * Meeus (1982) Meeus, J. 1982, Journal of the British Astronomical Association, 92, 124
  * Popinchalk (2023) Popinchalk, M. 2023, arXiv e-prints, arXiv:2303.17697, doi: 10.48550/arXiv.2303.17697
  * Ritchie (2022) Ritchie, H. 2022, Our World in Data
  * Shuster & Botton (1985) Shuster, C. N., & Botton, M. L. 1985, Estuaries, 8, 363.  http://www.jstor.org/stable/1351874
  * Sjödin et al. (2012) Sjödin, P., E. Sjöstrand, A., Jakobsson, M., & Blum, M. G. B. 2012, Molecular Biology and Evolution, 29, 1851, doi: 10.1093/molbev/mss061
  * Vidal et al. (2022) Vidal, C. M., Lane, C. S., Asrat, A., et al. 2022, Nature, 601, 579, doi: 10.1038/s41586-021-04275-8
  * Walls et al. (2002) Walls, E., Berkson, J., & Smith, S. 2002, Reviews in Fisheries Science - REV FISH SCI, 10, 39, doi: 10.1080/20026491051677
  * Werth & Shear (2014) Werth, A. J., & Shear, W. A. 2014, American Scientist, 102, 434.  http://www.jstor.org/stable/43707865
  * Williams & Boggs (2016) Williams, J. G., & Boggs, D. H. 2016, Celestial Mechanics and Dynamical Astronomy, 126, 89, doi: 10.1007/s10569-016-9702-3
  * Wright (2024) Wright, E. 2024, NASA Scientific Visualization studio, NASA.  https://svs.gsfc.nasa.gov/5222#section_credits
  * Zhu et al. (2020) Zhu, G., Yuan, X., & Fan, J. 2020, Global Ecology and Conservation, 22, e00896, doi: https://doi.org/10.1016/j.gecco.2019.e00896