# A Comprehensive Review of Machine Learning Advances on Data Change: A Cross-
Field Perspective

Jeng-Lin Li, Chih-Fan Hsu, Ming-Ching Chang, Wei-Chao Chen Jeng-Lin Li, Chih-
Fan Hsu, and Wei-Chao Chen are with Inventec Corporation, Taipei 111, Taiwan.
EMail<EMAIL_ADDRESS><EMAIL_ADDRESS>chen.wei-
<EMAIL_ADDRESS>Chang is with University at Albany – State
University of New York, Albany, NY 12065, USA. EMail:
<EMAIL_ADDRESS>received April XX, 2023; revised August XX,
2023.
Corresponding authors: Jeng-Lin Li, Chih-Fan Hsu, Ming-Ching Chang, and Wei-
Chao Chen

###### Abstract

Recent artificial intelligence (AI) technologies show remarkable evolution in
various academic fields and industries. However, in the real world, dynamic
data lead to principal challenges for deploying AI models. An unexpected data
change brings about severe performance degradation in AI models. We identify
two major related research fields, domain shift and concept drift according to
the setting of the data change. Although these two popular research fields aim
to solve distribution shift and non-stationary data stream problems, the
underlying properties remain similar which also encourages similar technical
approaches. In this review, we regroup domain shift and concept drift into a
single research problem, namely the data change problem, with a systematic
overview of state-of-the-art methods in the two research fields. We propose a
three-phase problem categorization scheme to link the key ideas in the two
technical fields. We thus provide a novel scope for researchers to explore
contemporary technical strategies, learn industrial applications, and identify
future directions for addressing data change challenges.

###### Index Terms:

data change, domain shift, concept drift, domain adaptation, transfer
learning, few-shot learning, robust learning, continual learning, foundation
models, data-centric analysis, trustworthy AI.

††publicationid: pubid: 0000–0000/00$00.00 © 2021 IEEE

##  1 Introduction

Data-centric Machine Learning (ML) paradigms have emerged into important
technical approaches to address many challenges in various Artificial
Intelligence (AI) application fields, including the automotive industry [1],
internet-of-things (IoT) [2], and healthcare [3]. Data is pivotal in driving
model learning, influencing performance, and serving as an infrastructure in
real-world systems. However, data may change due to differences in the
sources, unexpected sensor corruption, and seasonal variations. New data
distributions might gradually deviate from the original distributions. As a
result, the predictive capability of the learned model can degrade
accordingly. In the literature, there are two scenarios widely investigated
with respect to data changes: domain shift [4, 5, 6, 7] and concept drift [8,
9, 10, 11, 12, 13]. The frameworks for domain shift aim to handle data changes
caused by the change of data sources with the unchanged model’s predicted
performance. In comparison, concept drift methods aim to handle data changes
caused by time-varying phenomena leading to the obsolescence of existing
models. Although the original problem definitions differ in the two scenarios,
recent ML advancements in the two fields share commonalities and trends, due
to the similar underlying properties of data change.

Figure 1:  Taxonomy of the data change corresponds to this paper’s sections
and sub-sections. We illustrate our three-phase problem categorization scheme
in different colors and icons. The topics of domain shift and concept drift
are categorized into these phases.

Figure 2:  Overview of research topics related to the data change problems.
The corresponding section is written in parentheses brackets followed by the
research problem. The source domain model $M_{S}$ is trained with the data
$D_{S}$ and the target domain model and data are denoted as $M_{T}$ and
$D_{T}$. Data from $D_{t}$ to $D_{t+w}$ denote the data in an observed window
for concept drift. Refer to $\S$3 for the corresponding notations and
relations.

Domain adaptation aims to bridge the gaps between the source and target
domains [14]. The problem intricately encompasses distribution shifts in both
data and labels, arising from changes in data quality, annotating criteria,
and the targeted label classes. Substantial research endeavors have been
dedicated to enhancing the generalization capability of models in various
scenarios, leveraging the concept of distribution learning [15, 16, 17]. A
common strategy is to train a robust and general source model and then fine-
tune it for downstream tasks.

Concept drift arises from the need to monitor model performance throughout the
lifecycle of a system. Unexpected drift often occurs in data streams in non-
stationary environments, where data distributions evolve over time,
potentially leading to catastrophic failures in real-world applications. A
common strategy to address concept drift is estimating temporal changes by
analyzing historical data and adapting the model to accommodate forthcoming
changes.

Regarding the underlying data change, concept drift represents a form of
temporal distribution shift that requires timely adaptation. Therefore, it is
natural to align with the concept of domain shift. Researchers have expanded
and begun to explore similar approaches to handle the two problems. For
example, continual learning, which is specifically designed to mitigate
catastrophic forgetting, can help address domain shift problems towards
universal model learning. Simultaneously, continual learning techniques were
also developed to incrementally learn new knowledge without forgetting
important historical knowledge from the data stream [18] in the context of
concept drift. Similarly, regularization and ensemble learning techniques are
frequently leveraged in both domains. Fast domain adaptation and sequential
test-time training are two sub-fields within the domain shift methods that
emphasize efficiency and temporal factors during learning. Interestingly,
these factors align with the primary challenges addressed in concept drift.
Although the original assumptions in the two fields have shaped distinct
algorithmic framework developments, emerging topics have led to an
intertwining of technical approaches across these fields.

In this paper, we propose a unified perspective to bridge the literature gap
between domain shift and concept drift. We further provide insightful
industrial perspectives on the challenges associated with this gap [19] for
real-world model deployment applications. This paper aims to answer the
following research questions (RQs):

  * •

(RQ1) What problem settings have been proposed to tackle and consolidate the
complicated data change issues in the research fields?

  * •

(RQ2) What are the current state-of-the-art approaches that address the
problems under these settings?

  * •

(RQ3) How best to contribute to the integration of research fields revolving
around the data change problems in a unified categorization scheme?

  * •

(RQ4) What are the perspectives, challenges, and practical applications in
industries associated with data change?

Based on their underlying shared data change characteristics, we associate
domain shift and concept drift with a new and unified three-phase scheme to
make evident the advantages of techniques in the two fields, facilitating
comparisons of framework design ideas across the fields. Specifically, Fig. 1
shows the three-phase scheme including problem detection, problem handling,
and other related factors. These phases suggest the logic of the model
deployment procedure starting from detecting the problem to adapting models
and finally incorporating other factors to further relax usage constraints.
Efficiency is an important factor in adaptation, as it helps prevent the
burden of extensive model retraining. Continual learning for the forgetting
effects has also been studied to achieve sustainable model deployment in
concept drift. Fig. 1 lists detailed terms used in the fields of domain shift
and concept drift. For instance, in the case of domain shift, data undergo
changes in batches, whereas, in concept drift, data changes occur in streams.
Consequently, in domain shift, the source and target domains represent the
data before and after the change, whereas in concept drift, the current and
next states are utilized to capture the evolving data dynamics.

Overall, this paper shows a comprehensive overview of the global landscape,
cutting-edge designs, and emerging trends of domain shift and concept drift.
Specifically, we summarize the key research and surveys in both research
fields and introduce a problem-oriented taxonomy to categorize the state-of-
the-art studies. We examine the major difference and similarities between the
methods and datasets, thereby identifying potential avenues for future
research. Additionally, we highlight other important topics, such as model
bias and fairness, as well as advances in the related communities that should
be considered in addressing data change problems. We expect this overview
paper can stimulate technical improvement to address the challenging data
change problems and guide researchers and practitioners for future
development.

Our contributions are summarized in the following.

  * •

Our work bridges the gaps of key ideas for handling data change in modern deep
learning that spans multiple research fields around domain shift and concept
drift.

  * •

This study reviews the state-of-the-art methods and illustrates the
characteristics of the two fields, thereby bringing researchers to the
frontiers of cutting-edge advancements.

  * •

We introduce a three-phase scheme including problem identification, problem
handling, and extended factors, to categorize the research topics and
illuminate key developments and emerging topics in the two fields.

  * •

We propose future research directions regarding the challenges of model
deployment in industrial applications.

The rest of the paper is organized as follows: $\S$2 summarizes the recent
survey papers on domain shift and concept drift, and $\S$3 describes the
notations used in this paper. $\S$4 and $\S$5 introduce the domain shift and
the concept drift problems with a series of research topics using our proposed
problem categorization scheme. $\S$6 discusses the shared and complementary
techniques in the fields of domain shift and concept drift. $\S$7 provides an
industrial perspectives for the state-of-the-art studies. Finally, $\S$8
concludes this data change study with the current status and future trends.

TABLE I: Recent survey papers published in 2018-2023. Domain Shift | Concept Drift
---|---
Transfer Learning [4, 5, 6] | General [10, 11, 13]
Unsupervised Adaptation [20, 21, 22] | Unsupervised Learning [23, 24, 25]
Source-free Adaptation [26] | Machine learning [27, 28]
Domain Generalization [29] | Regression [30]
Vision [31, 22] | Class imbalance [32]
Language [33, 34, 35] | Deep learning [36]
Acoustic and Speech [37, 38] | Performance-based approach [39]
Reinforcement Learning [40] | Distance measurement [41]

##  2 Related Surveys and Datasets

Table I lists the existing survey papers up to 2023 that are on domain shift
and concept drift. We briefly introduce their key contributions and the
featured topics to summarize the surveys in both fields. Table II and Table
III summarize the commonly-used public datasets for the evaluation of domain
shift and concept drift, respectively.

###  2.1 Domain Shift Survey Papers

Domain shift involves the transfer learning across domains in addressing the
data distribution shift problems. With the identified source and target
domains, the domain shift problem can easily be found when we need to deal
with multiple datasets. Therefore, the techniques are extensively investigated
in multiple prominent machine learning applications, such as vision, language,
and speech. Table I lists the recent survey papers along with the key topics
that shows the trend of domain adaptation development. The general focus on
transfer learning has garnered significant attention, with notable
contributions from four survey studies [4, 5, 6, 7]. Additionally,
unsupervised domain adaptation has emerged as a widely discussed topic [20,
21, 22]. Vision [31, 22], language [33, 34, 35], and speech [37, 38]
applications have also been organized to lighlight the technologies developed
in the contexts, each with its own task and data properties. Review studies
have also covered topic such as reinforcement learning [40], providing a
comprehensive understanding of the research findings.

TABLE II: Existing benchmark datasets for domain shift. We describe the major
data change factors along with the modality of the datasets.

Vision Attribute Data Corruption Noisy Data Style Environment Colored MNIST
[42], MNIST-R [43] CIFAR10-C [44], CIFAR100-C [44], ImageNet-C [44],
ModelNet40-C [45], VisDA-C [46] ANIMAL-10N [47], CIFAR-10N [48], CIFAR-100N
[48], Clothing1M [49], WebVision [50] PACS [51], Office-Home [52], DomainNet
[53], ImageNet-R [54], VisDA [46] VLCS [55], Wilds [56], NICO [57] Language
Speech Language (Translation) Query Dialog Intention Sentiment Data Source
Language CoCoA-MT [58] OSQ [59] SNIPS [60], CLINC150 [61], MultiWOZ 2.2 [62]
ACE2005 [63], Amazon MDSD [64] LibriAdapt [65] Common voice [66]

###  2.2 Domain Shift Datasets

We investigate the publicly available datasets commonly used in domain shift
research. Existing datasets are organized into three setup types according to
their covering domains, the use of synthetic data, and the data sources.
First, to cover broad domain changes, samples were collected in-the-wide from
multiple domains [67, 68]. Second, data change can be simulated by changing
the color and viewing angle, or by adding noise or blurriness [42, 43, 54] and
other data corruption [44, 45, 46]. Third, a simple approach to prepare a
cross-domain dataset is to combine two or more naturally different datasets
collected from multiple sources or institutions [69, 15, 70, 71].

Table II provides an overview of datasets collected with domain shift data in
different modalities. Sub-fields of domain shift such as out-of-domain
detection ($\S$4.1) and domain generalization ($\S$4.2) are developed on
large-scale benchmark datasets to include different types of data change. The
commonly considered domains are style and environmental changes, such as
clipart and infographic styles in DomainNet [53], and speaker accents and
acoustic environment in LibriAdapt [65]. Changes in language tasks naturally
involve cross-lingual contents [63] and conversation topics [62]. These
studies frequently use cross-institution setups while turning into the multi-
domain setup as the scale of data collection grows. Fast adaptation ($\S$4.4)
follows a similar data configuration, yet it selects the dataset altered with
distinct label classes. Studies on label shift ($\S$4.3) and test-time
training and adaptation ($\S$4.5) often utilize synthesized corrupted data
(e.g., CIFAR100-C or ImageNet-C [44]) to represent the target domain with
presumed change types.

###  2.3 Concept Drift Survey Papers

Numerous studies have investigated the issue of concept drift that occurs in
various machine-learning applications. Common categories of algorithms include
performance-based detection [39], distribution-based detection [41],
hypothesis-based detection, and contextual-based detection [39]. The majority
of previous review studies have identified similar taxonomies for categorizing
typical concept drift approaches [8, 9, 10, 11, 12, 13]. Table I lists the
topics highlighted in the review papers between 2018 and 2023. Unsupervised
approaches are widely used due to their ability to effectively handle concept
drift without relying on labeled data. Fast-growing machine learning
techniques are then employed to address concept drift [27, 28]. Meta-learning
has emerged as a promising direction for concept drift adaptation [27].
Algorithms tailored to specific usage scenarios, such as regression [30] and
class imbalance [32] problems, are developed taking into account the
properties of the respective problems. Furthermore, studies in deep learning
as presented in [36], have focused on learning network parameters and
architectures to address concept drift in the data stream.

TABLE III:  Commonly used datasets for the concept drift. We describe the
major data change factors along with the task descriptions.

Dataset Prediction Goal Descriptions (Change Factor) Weather [72] raining
prediction climate change over 1949-1999 Electricity [73] daily price changes
(rise or fall) change of prices affected by supply and demand Forest [74]
seven cover types change of ecological processes Outdoor Objects [75] forty
objects change of lighting conditions on a mobile in a garden environment
Rialto Bridge Timelapse [76] ten colorful buildings change of weather and
lighting conditions over 20 consecutive days Luxembourg [77] frequency of
internet usage (high or low) change of personal habits and political tendency
Chess [77] outcome of the game (won, lost, or draw) change of player skills
and opponent selection by system

###  2.4 Concept Drift Datasets

Current studies on concept drift have used both real and synthetic datasets,
with a focus on continuous temporal data. Tabular and structured data are more
commonly employed in these cases. Given the diverse conditions under which
concept drift occurs in different data collection scenarios, we report
commonly used real datasets in Table III. In line with the trend of utilizing
real-world data with an unbiased estimation of the drift, we exclude synthetic
datasets and prioritize the frequently used datasets in the table. The
selected datasets comprise the prediction targets that are influenced by
various potential drift factors. For example, the weather observations exhibit
varying frequencies of rainy days as a consequence of climate change [72].
Similarly, ecological changes impact forest coverage [74]. Video recordings
often exhibit changes in lighting conditions and recording environments [75,
76]. Human behaviors and habits also exhibit temporal variations, which can be
observed in internet usage patterns [77] and chess playing [77]. For research
aiming at comprehensive studies across various applications, Souza et al. have
presented additional benchmark datasets [78].

TABLE IV: Commonly referred terms in domain shift and concept drift.

Domain Shift Concept Drift Data Access Batch Stream Original Data Source
Domain ($D_{S}$) Previous State ($D_{t}$) Changed Data Target Domain ($D_{T}$)
Next State ($D_{t+w}$)

TABLE V:  Problem definition and properties for the data change technical
fields. $D_{m}$ indicates the $m$-th domain dataset used for model training.

Domain Shift Training Change Source Data Source Model Target Data Target Model
Task $\mathcal{Y}$ $P(Y)$ $P(Y|X)$ Temporal Ouf-of-Domain Detection $D_{1}$, …
, $D_{M}$ $D_{1}$, … , $D_{M}$ $D_{M+1}$, Batch ✗ ✗ ✓ ✓ ✗ Domain
Generalization $D_{1}$, … , $D_{M}$ $D_{1}$, … , $D_{M}$ $D_{M+1}$, Batch ✗
✓11footnotemark: 1/✗ ✗ ✓ ✗ Domain Adaptation $D_{i}$ $D_{i}$ $D_{j}$, Batch
$D_{j}$ ✓22footnotemark: 2/✗ ✗ ✓ ✗ Label Shift $D_{i}$ $D_{i}$ $D_{j}$, Batch
$D_{j}$ ✗ ✓ ✓ ✗ Fast Adaptation $D_{1}$, … , $D_{M}$ $D_{1}$, … , $D_{M}$
$D_{M+1}$, Batch $D_{M+1}$ ✓ ✗ ✓33footnotemark: 3/✗ ✗ Test-Time Training ✗
$D_{i}$ $D_{j}$, Stream $D_{j}$ ✗ ✗ ✓ Short Concept Drift Training Change
Source Data Source Model Target Data Target Training Task $\mathcal{Y}$ $P(Y)$
$P(Y|X)$ Temporal Drift Detection $D_{i}$ $D_{i}$ $D_{j}$, Stream ✗ ✗ ✓/✗ ✓/✗
Long Drift Adaptation $D_{i}$ $D_{i}$ $D_{j}$, Stream $D_{j}$ ✗ ✓/✗ ✓/✗ Long
Label Drift $D_{i}$ $D_{i}$ $D_{j}$, Stream $D_{j}$ ✗ ✓ ✗ Long Continual
Learning $D_{1}$, … , $D_{M}$ $D_{1}$, … , $D_{M}$ $D_{j}$, Stream $D_{j}$ ✓ ✗
✗ Short

11footnotemark: 1

Zero-shot learning 22footnotemark: 2Incremental learning 33footnotemark:
3Cross-domain few-shot learning

##  3 Notations and Definitions

We next introduce the notations used in this paper. We aim to establish
consistent notations and definitions to describe the data change associated
with domain shift and concept drift for all papers that are reviewed. A
specific domain is characterized by a distribution $P(X)$, which represents a
set of data samples $X=\\{x_{1},x_{2},...,x_{N}\\}$. Here, $N$ denotes the
number of samples in the collected dataset $D$. For a given machine learning
task $\mathcal{T}$, which involves a collections of labels
$Y=\\{y_{1},y_{2},...,y_{N}\\}$ in the dataset $D$, supervised learning can be
conducted using the labeled samples ${x_{i},y_{i}}$ for $i\in N$, $x_{i}\in X$
and $y_{i}\in Y$.

We generalize the notations in the context of domain shift to describe the
data change for concept drift. Specifically, we define the mismatch between a
source domain and a target domain for domain shift and further extend this
definition to address concept drift. This allows the dynamic updating of the
two domains to accommodate the data stream setting. Table IV shows the unified
definition for the various terms commonly used in the fields of domain shift
and concept drift. The original data correspond to the source domain dataset
$D_{S}$ and the changed data correspond to the target domain dataset $D_{T}$.
Concept drift is usually defined on a particular time step $t$ and a time
window $w$, to specify the data in the current state $D_{t}$ and the
occurrence of drift data at $t+w$, denoted as $D_{t+w}$. Here, $D_{t}$ can be
regarded as a special case of $D_{S}$, where the time stamp is specified, and
thus $D_{t+w}$ can be represented by $D_{T}$.

The data change problem is formulated as the distribution shift between
$P_{S}(X,Y)$ and $P_{T}(X,Y)$ which contains the special case of the concept
drift problem. Table V provides an overview of the sub-field problems
addressed in this paper. We highlight the differences in data and models
between the source and target domains to illuminate the changes. As there
might be multiple source or target domain datasets, we denote the dataset as
$D_{m}$, where $m$ represents the index of the domain. The checklist showing
the data change of the distribution is also presented in Table V.

##  4 Domain Shift

We categorize the research efforts addressing the distribution shift between
the source domain distribution $P_{S}(X,Y)$ and target domain distribution
$P_{T}(X,Y)$ in three phases of problem categorization, as shown in Fig. 1.
The problem detection phase focuses on out-of-domain detection technologies
($\S$4.1). The problem handling phase encompasses domain adaptation and
generalization approaches ($\S$4.2). Additional factors including label
quality, adaptation speed, and temporal dynamics are considered in $\S$4.3,
$\S$4.4, and $\S$4.5.

###  4.1 Out-of-domain Detection

Out-of-domain (OOD) detection is vital in real-world applications, as it can
avoid data abnormality that deviates from the typical distribution. This topic
was recently identified in 2017 [79] due to the limitations of common domain
adaptation approaches in ensuring safe and robust deployment in rapidly
changing environments. From the perspective of domain adaptation, the OOD
detection problem aims to identify unknown data sample that differs from the
source domain dataset $D_{S}$, which is typically unavailable during the test
phase. $D_{S}$ is referred to as in-domain data and the unknown samples are
OOD data. The goal is to identify samples that do not belong to the source
distribution $P_{S}(X,Y)$. Therefore, the OOD problem can seen as an early-
phase monitoring problem, as the target dataset $D_{T}$ is not accessible for
retraining. The scope of OOD detection has expanded to include various related
topics such as anomaly detection, novelty detection, open set recognition,
out-of-distribution detection, and outlier detection [79]. This broader
perspective considers not only changes in sample features but also unknown
classes and noises within the domain shift context. Differentiating between
relevant information and irrelevant noises is a complex task, as the
definition of in-domain distribution must be unbiased. Overconfident
predictions can undermine model trust.

Classification-based methods are intuitive in thresholding the prediction
score to distinguish in-domain and out-of-domain distributions. Advanced
methods extend the scoring functions to the gradient space, offering better
flexibility for observing abnormal patterns. The design of the scoring
function, by estimating the probabilistic properties of true likelihood
distribution, can keep sensitive scores for model performance [80]. State-of-
the-art OOD detection approaches favor agnostic solutions that are applicable
to any network structure. For instance, the Simple Activated Function designed
for deep networks makes no assumptions on the model architectures [81].
Another effective approach is the non-parametric nearest-neighbor distance,
which is distribution assumption-free, test data agnostic, and model agnostic
properties [82]. The success of the distance measurement approach can be
attributed to techniques like contrastive learning [83] and self-supervised
learning [84], which come with enhanced generalization ability for in-domain
training.

OOD detection in visual applications aims to detect domain shift factors, such
as spatial location accuracy, appearance diversity, image quality, and aspect
distribution [85]. Speech and language applications may encounter challenges
related to cross-language problems or content differences influenced by
cultural or contextual factors [59, 86]. In a recent survey study [87], open
source codes are released to facilitate comprehensive benchmarking of the
generalized OOD detection frameworks. Other libraries are also available to
accelerate the R&D in this area [88]. These OOD detection approaches have led
to the development of corresponding strategies in machine learning systems in
two directions: learning with rejection and domain adaptation. Rejecting
prediction on unknown data maintains the quality of the services and engaging
humans to rectify the data promotes effective collaboration between humans and
machines for further model learning. The direction of adaptation described in
$\S$4.2 aims for a more automatic process to adapt models as soon as OOD is
identified.

###  4.2 From Domain Adaptation to Domain Generalization

Traditional domain adaptation methods assume that we have access to the target
domain dataset $D_{T}$ to estimate the distribution $P_{T}(X,Y)$, which
usually restricts real-world applications. Therefore, unsupervised domain
adaptation has emerged, leaving to the promising field of domain
generalization (DG). DG encompasses an ambitious goal to generalize learning
without accessing to the target domain dataset $D_{T}$. It can be seen as a
more generalized problem of domain adaptation, which aims to handle unseen
domains without retraining models [29]. Both domain generalization and domain
adaptation are designed in the problem-handling phase which has attracted
abundant research. In contrast to the OOD problem, the DG problem focuses on
detecting changes in data across different domain distributions and adapting
models to maintain predictive ability even on unseen classes [89].

####  4.2.1 Source Domain Settings

The source domain data for training yields in-depth impacts on the model
generalization ability and transferability. Herein, we regard two source
domain training data settings, including multiple-source and single-source
settings. The typical DG studies use multiple training datasets for multiple
sources and attempt to generalize the source domain distribution $P_{S}$ [29].
The single source DG is more difficult because there is less data variability
for the source model. Some prior works deem the single source DG as an OOD
generalization problem [90] to focus on the samples’ learning once the OOD has
been detected. The typical domain adaptation studies use the single source
setting that assumes the availability of the target domain labels for
adaptation.

Aside from the two data source settings for training, source-free domain
adaptation prohibits the source data to be used for adaptation once the source
model has been trained [91, 92]. The adaptation is only based on the trained
model parameters without access to the source domain data $D_{S}$.
Technically, clustering along with pseudo labels can help estimate the
distribution shift [93]. Xie et al. propose contrastive category matching to
exploit the relationship between pairs of targeted images [94]. Source-free
domain generalization has been identified as a harsh problem. Although
teacher-student learning [95] and using text prompts as a proxy can be
possible solutions [96], there still remains substantial space for further
research.

####  4.2.2 Transfer Learning Strategies

Research in both domain adaptation and domain generalization has grown with
three branches of learning strategies to conquer the domain gap problem
including domain finetuning, domain adversarial learning, and domain matching
approaches. Their distinct underlying ideas shape the framework designs with
more focus on model refining, distribution difference elimination, and
distribution alignment techniques, respectively.

Domain Finetuning: The domain finetuning is the most fundamental approach for
domain adaptation which can then be introduced to domain generalization with
additional algorithm adjustment. Researchers are meant to construct a large
and robust source model for initialization and design fine-tuning techniques
for downstream tasks. The knowledge distillation approaches extract the
knowledge in the source domain network for further reuse in the target domain
[69]. Teacher-student learning enhances the knowledge distillation approaches
by keeping using the distilled teacher network to supervise the finetuning
steps [97]. With the knowledge distillation technique, the frameworks can deal
with source-free cases [95]. Self-supervised learning loss acts as a
regularization term to map the same-class samples across domains close to each
other [98].

Domain Adversarial Learning: The domain adversarial learning is a mainstream
technique that is renowned for Domain Adversarial Neural Network (DANN)
introducing the reverse gradient with an additional discriminator to avoid one
domain being differed from another [15]. This setting naturally results in an
adversarial condition in which one branch learns to recognize the main task
and the other branch disables the ability to discriminate different domains.
The adversarial approaches further improve the convergence by using mixup to
create continuous label space instead of hard domain labels [16]. Although
adversarial learning is designed to learn domain invariant latent features,
over-optimization on domain discriminator can degrade the recognition
performance on the target domain. Accordingly, explicitly designing an
interplay of the classifier and the domain discriminator attains better
adversarial optimization [99]. Recent visual transformer (ViT) takes advantage
of the attention module to re-weight image patches to boost transferability
during the adversarial learning [100]. The adversarial learning can also be
extended to domain generalization studies by seeking a common ground of
multiple datasets in the domain discriminator [101].

Domain Matching: The domain matching approaches minimize the domain
discrepancy by optimizing feature consistency across domains. The focus of the
research turns to the design of distribution measurement approaches, such as
maximum mean discrepancy (MMD) [102], correlation alignment [103], contrastive
domain discrepancy [104], and Wasserstein distance [105]. Other than the
distribution measurement approaches, batch normalization can adaptively
dissociate bias and variance of the datasets [106]. In the source-free
scenario, measuring classifier discrepancy with pseudo-labeled samples
facilitates self-training [68].

###  4.3 Label Shift and Noisy Labels

Most machine learning models highly rely on labeled data for supervised
learning. However, unreliable labels might be observed in situations of label
shift and noisy labels. Label shift refers to the change of the underlying
label distribution $P_{S}(Y)\neq P_{T}(Y)$. Although noisy labels would also
give rise to the difference between the label distribution. We differ the
label shift and noisy labels with the corresponding learning strategies that
mainly keep the target distribution or the source distribution
characteristics, respectively.

Current studies blindly estimate the label shift problem and performance drop
as it occurs unexpectedly. For example, Lipton et al. have proposed a black
box shift estimation on an invertible confusion matrix and the distribution
ratio before and after label shift [107]. To generalize the estimation in the
low target sample setting, an importance weighting approach with
regularization to address three kinds of synthesized label shift [108]. These
studies quantify the shift with mathematical forms and propose a statistical
bound to guarantee the bound of generalization. However, the presumed labeled
shift might be oversimplified. An advanced work proposes a model to jointly
estimate sparse covariate shift and label shift [109]. Recently, online
learning algorithms can be used to actively estimate the shift [110].

One of the situations of label shift is that the source domain is contaminated
with noisy labels. Data annotation engages in the cognitive status and past
experiences of the annotators which yields potential labeling errors when the
task is complex or the annotators are biased. The noisy labels in the source
domain impede the model’s robustness to generalize across domains. These
corrupted labels might occupy 8.0% to 38.5% in real-world datasets [111]. The
label quality issues are amplified as crowd-sourced labeling solutions can
reduce data annotation costs. Technically, learning with noisy labels aims to
handle the problem by enhancing the robust source domain model learning [111,
112] including framework designs on architecture, regularization, loss design,
and sample selection. Curriculum learning is a remedy to recover noisy data as
clean data by the estimation of empirical risk [113]. One of the loss designs
is the denoising MMD loss to estimate the target domain label distribution
with noisy source data [17]. Sample selection by measuring the divergence of
two classifiers for each sample can alleviate the adverse effect of noisy
samples and perform partial alignment on the cross-domain distributions [114].

A special case related to the learning from noisy labels is the source-free
domain adaptation in that recent studies have generated pseudo labels as noisy
labels for the target data using a pre-trained source model [115, 116].
Regularizing the source model to avoid overly adapting to the noisy target
data provide a bound of effect from the noise [116]. Separating target domain
samples into clean and noisy data also regularizes the learning in a self-
supervised manner [115]. Considering the label shift as noise has opened up
new avenues for robust learning across domains.

###  4.4 Fast Adaptation

Frequent change in real-world data poses challenges to the domain adaptation
models with the need for data efficiency and convenience. This section
includes algorithms resolving the issues with inconsistent label space and
target domain tasks. Few-shot learning and zero-shot learning are notable
solutions to deal with learning with limited access to new data. A well-
trained source domain model is the key to generalizing without complicated
operations on the target domain data. Sophisticated techniques are proposed to
improve this source domain model training to derive the so-called foundation
model. The idea is that the foundation model aggregating massive knowledge
provides strong initialization for any tasks to transfer with minimal effort.
To allow fast adaptation with limited testing data, the technical approaches
are optimized on either pretraining or finetuning phases. The pretraining aims
to learn a generic model with large-scale datasets and the finetuning requires
several techniques to efficiently adapt model parameters.

####  4.4.1 Few-Shot Learning

Few-shot learning is a problem set upon a harsh condition that the target
dataset $D_{T}$ contains only very few available labeled samples for
adaptation. We formally define a task
$\mathcal{T}_{S}=\\{\mathcal{Y},P(Y|X)\\}$ with the label space $\mathcal{Y}$
and conditional distribution $P(Y|X)$. The common setup includes the source
domain dataset $D_{S}$ for typical supervised training and validation. Then, a
new $C$-way $K$-shot task $\mathcal{T}^{\prime}$ is set up in the situation
that $\mathcal{T}\neq\mathcal{T}^{\prime}$. That is, there are $C$ unseen
classes in the target domain, resulting in disjoint label space. However, only
a very limited number of $K$ samples are available as the target domain
support set $D_{T}^{S}$ for model training, and the final task is to predict
on a query set $D_{T}^{Q}$. In the context of a typical pre-trained model
utilizing dataset $D_{S}$, the occurrence of domain shift towards dataset
$D_{T}$ poses a challenge known as cross-domain few-shot learning. This shift
is characterized by the dissimilarity between the probability distributions
$P_{S}(X)$ and $P_{T}(X)$.

A prior survey has categorized few-shot learning research into meta-learning
and non-meta-learning methods [117]. The task change naturally leads the few-
shot learning algorithm to seek more tasks for pretraining. Meta-learning
becomes a major approach, mimicking the prediction of new tasks with sampling
strategies (e.g., MAML [118]). Data augmentation and prototypical networks are
notable examples of non-meta-learning methods [119]. The design principle of
these approaches is to expand the feature space with sufficient sample
diversity, providing a robust semantic structure for the new task to finetune
delicate information. The cross-domain few-shot learning problem is more
challenging with the change of both the domain and task. Associating
prototypical network and adversarial learning serves as a way to capture new
task prototypes in the new domain [120]. Moreover, training with a contrastive
loss enhances the semantic structure of the self-supervised prototypes [121].
Domain-Switching Learning embeds the knowledge from multiple domains to
realize a fast switch to the target domain [122]. The setting is similar to
domain generalization but allows a few data in the support set to tackle the
task differences. Zero-shot learning is an extended field restricting the use
of the support set while introducing semantic features (attributes) to bridge
to the new tasks [123].

####  4.4.2 Foundation Models

The progress of the pre-trained model with continuously enlarged source domain
data size has achieved promising success in the natural language processing
[124], vision [125, 126], and audio [127] domains for fast adaptation to
various downstream tasks. The core idea of the foundation model is to
aggregate as much knowledge as possible into a pre-trained model for generic
use.

Pretrained Model:  Large-scale self-supervised learning is a dominated source-
domain model training approach with the idea that predicts the latent features
of masked inputs with self-training. Contrastive learning can minimize the
distance between positive pairs of samples and maximize negative pairs of
samples while the non-contrastive approach only considers the positive pairs
[128]. Intriguingly, the recent research indicates that the covariance
regularization-based non-contrastive methods can be regarded as performing
contrastive pairs on dimensions rather than samples [129]. The learning
methods for pre-trained models might share similar optimization goals with
different techniques. In addition, Data2vec demonstrates that the same self-
supervised training approach can be applied to different modalities [130]. The
foundation model can not only support multimodal tasks [125, 131], but also
encompass huge information with even cross-domain samples. For example, cross-
lingual acoustic and language models have been introduced by training with
multiple source domain datasets [132, 133].

Finetuning:  With a pre-trained foundation model, parameter efficiency in
finetuning becomes crutial that requires minimal effort for deployment in new
tasks, which is also termed as “delta tuning” [134]. Typically, researchers
train an adapter by inserting a small learnable module to avoid updating the
whole bulky network [135]. Prefix tuning manipulates the keys and values
attention heads [136] and prompt tuning simplifies the manipulation to the
input tokens [137]. Instead of inserting an adapter module, researchers also
introduce a learnable low-rank matrix to modify the projection of intermediate
layers [138]. A unified view has been proposed to cast the problem as a
learning problem on modification vectors of hidden layers [139].

###  4.5 Test-time Training and Adaptation

One of the emerging topics of unsupervised domain adaptation, test-time
training (TTT) or test-time adaptation (TTA), has been consolidated recently
that the generalization of learning in the testing phase can leverage
unlabeled testing data as an indicator for updating the model on the new data
distribution [140]. This setting breaks the traditional stationary assumption
and indicates that the testing data distribution might change over time. In
practice, we observe a small batch of testing data and update the model in an
unsupervised manner. Then, we evaluate the performance on-the-fly with the
adapted model. TTA can be thought of as an online domain adaptation problem
combining the techniques from domain generalization and fast adaptation
described in $\S$4.2 and $\S$4.4, respectively. During training, researchers
design the learning of source domain models for generalization while during
testing, self-training with the unlabeled test data at hand needs to adjust
the model for a better fit. In this task, the adaptation at test time is
usually conducted without access to the source domain data. This setting
connects TTA with source-free DG described in $\S$4.2.

####  4.5.1 Online Learning

TTT depends on the powerful foundations of source domain training and the
strategies of the target domain unsupervised adaptation. The designed loss
function with self-supervised learning techniques dominates the success of
TTT. In addition, meta-learning is another strategy to expand the source
domain learning space for a generic model, and thus the model simultaneously
exploits the learned knowledge from other tasks [141]. In the testing phase,
computing the self-supervised learning loss on the unlabeled testing data can
update the model to handle new data distribution. The distribution mismatch
often causes failure of adaptation due to the missing link across domains.
Therefore, selecting an appropriate loss and matching the samples located in a
similar feature space are techniques to diminish the distribution shift. For
example, Goyal et al. examine training losses’ convex conjugate to facilitate
to seek for the best TTA loss for a given source model and distribution shift
[142]. Chen et al. refine online pseudo labels with soft voting among their
nearest neighbors and learn to contrast positive and negative samples [143].
Liu et al. propose a feature alignment approach by computing statistics to
represent the distribution characteristics [144]. Gandelsman et al. reframe
the TTT problem as one-sample learning which aims to perfectly fit each
testing sample point for tailored prediction [145]. Sequential test-time
training (sTTT) is recently defined to strictly include only one-pass
adaptation on the data stream [146]. That is, refining the model with the
accumulated testing data is not allowed to minimize computational overhead.
Even further, adjustment on the classifier completely avoids the back-
propagation and minimizes the cost at inference time [147].

####  4.5.2 Learning without Forgetting

The model updating over time raises the catastrophic forgetting issue which
learns the pattern from the new data samples but loses the discriminative
power on the old samples. The new pattern is usually attributed to the out-of-
domain samples described in $\S$4.1. Continual learning, also known as
lifelong learning, has been exploited as a remedy to consolidate the learned
knowledge over time. For example, Niu et al. propose to actively select
reliable samples with regularized updating to avoid dramatic network
parameters change [148]. Qin et al. reduce the error accumulation by using
weight-averaged and augmentation-averaged prediction and introducing
stochastic restoring partial neurons during training iterations to avoid the
forgetting effects [149]. The model updating efficiency is also emphasized in
these studies because the frequent back-propagation during the testing phase
leads to great burdens on computational consumption. Samples redundantly
generating similar gradients can be discarded to ensure sample efficiency
during adaptation [148]. The studies imply the data-efficient learning trend
for data stream with minimal updating and maximal knowledge consolidation.

##  5 Concept Drift

For the input of data streams, the ability for models to adapt on the fly is
crucial when facing dynamic changes in data distribution. Typical machine
learning models are ill-suited for dealing with non-stationary scenarios,
which significantly impacts their reliable deployment across diverse
application domains. In the studies [39, 11], the term “concept drift” is
defined as the occurrence of a change in the joint probability of variables
$X$ and $Y$ at time $t$, within a specified time window $w$. Specifically,
$\exists t,w:P_{t}(X,Y)\neq P_{t+w}(X,Y)$. To maintain consistency, we refer
to the distribution at time $t$ as the source distribution and the
distribution at time $t+w$ as the target distribution. Consequently, the
concept drift is expressed in a similar manner as the domain shift problem,
where $P_{S}(X,Y)\neq P_{T}(X,Y)$, based on the notation defined in $\S$3.

We identify three distinct types of potential sources of drift from the
literature survey. The first type is referred to as virtual draft,
characterized by the condition where $P_{S}(X)\neq P_{T}(X)$ while
$P_{S}(Y|X)=P_{T}(Y|X)$. This form of drift does not explicitly impact the
decision boundary. The second type, known as actual drift, occurs when
$P_{S}(Y|X)\neq P_{T}(Y|X)$ while $P_{S}(X)=P_{T}(X)$. In this case, there is
a direct influence on the posterior probability for predicting the target
label, thus affecting the decision boundary. Lastly, the third type, termed
prior probability drift, arises when $P_{S}(Y)\neq P_{T}(Y)$. This drift
significantly alters the learning process, often involving novel classes or
the disappearance of existing classes.

Figure 3:  Examples of the common drift types. The figure is adapted from the
review paper [39].

Research topics on concept drift generally concern the why, how, where, and
when the drift happens [11]. Factors behind the observed drift can be
intricate. For example, social trends involving newly-created terms and
preferences can influence text recognition tasks in recommendation systems.
Transient, seasonal, and circumstantial changes are sometimes subject to
domain-specific and user differences, such as policy changes, pandemics, and
personality. Most existing studies categorize the drifts into the following
types [11] as in Fig. 3: sudden drift, gradual drift, incremental drift, and
recurring concepts. The general implementation setting is accumulating
historical data, training models, and performing testing while the demand for
collecting historical data leads to a trade-off between usability and model
stability. Prior studies are inclined to neglect the concept drift issues in
common settings. Recent studies have focused on the importance of temporal
generalization, arguing that the random split used in the previous works might
unintentionally neglect the issue of concept drift, which hinders their real-
life usage [150].

According to the proposed three-phase problem categorization in Fig. 1, we
investigate the drift detection ($\S$5.1) and drift adaptation ($\S$5.2) for
the first two phases. We discuss other factors including label drift ($\S$5.3)
and continual learning ($\S$5.4), which offer solutions for unstable human
annotations and the forgetting effects, respectively.

###  5.1 Drift Detection

We organize the technical developments in drift detection into three main
categories, namely, performance-based, distribution-based, and meta-learning-
based methods. Performance-based and distribution-based methods have long
development history, and meta-learning is an emerging technique using machine
learning algorithms to handle concept drift.

####  5.1.1 Performance-based Detection

The most intuitive means to detect drift is monitoring the performance change
with the assumption that the drift directly results in the degradation of the
model accuracy. Bayram et al. have provided an in-depth review of this kind of
performance-based detectors [39]. The rule-based, statistical-based, and
ensemble-based methods are widely used due to the easiness of deployment.
Traditional methods usually set thresholds and rules to capture the pattern of
different types of drift. For example, monitoring performance drop within an
adaptive time window (e.g., DDM [151] and ADWIN [152]) to detect the drift by
selecting appropriate rule parameters. Recent advances focus on selecting the
proper threshold as an index instead of developing new algorithms [153].
Statistical-based methods formulate the rules into multiple hypothesis tests
to alleviate false alarms from insignificant data change cues. Hierarchical
Hypothesis Testing was designed upon Heoffding’s inequality and two
Kolmogorov-Smirnov (KS) tests to monitor the defined class uncertainty [154].
Ensemble-based methods detect drafts by examining the performance degradation
of multiple learners. One example is to adaptively maintain different levels
of ensemble diversity, in a way that ensembles of both high and low diversity
can mix detecting patterns to reduce bias [155].

Ensemble-based methods can be combined with rule-based and statistical-based
methods. The EnsembleEDIST2 [156] uses EDIST2, a statistical hypothesis test
on distribution distance in the defined windows, for decision criteria update.
Meta-learning is then built based on the multiple-learner setting in the
ensemble-based methods. The meta-learner can reuse or replace the base
learners by tracking their behaviors [157]. Recently, Yu et al. [158] pre-
trained an offline prototypical network as a meta-detector with meta-features
extraction based on error rates; they conducted online finetuning for improved
prediction accuracy.

####  5.1.2 Distribution-based Detection

Distribution-based detection approaches utilize distance functions to measure
the difference between the distribution of historical and new data. This
measurement is usually examined via hypothesis tests to ensure the
significance of the detected results. The underlying concept is similar to
anomaly detection which can detect the distribution variation in an
unsupervised manner. Under this circumstance, studying clustering algorithms
along with a family of distance functions can properly quantify the
distribution relations, where the representation space is shaped by the
distance function. Zubaroğlu and Atalay surveyed [159] the clustering
approaches for data stream with detailed characteristics regarding the time
window, computational complexity, and accuracy.

Centroids as the cluster centers can represent the average characteristics of
a subset of data in a distribution, which evolves with the techniques to learn
“prototypes”. The window of focus becomes a parameter for measuring bias from
the prototypes. For example, regional drifts can be observed within a larger
window. The well-known Gaussian Mixture Model (GMM) allows the measurements to
be maintained as soft clusters over time [160]. The nearest neighbor-based
density variation identification (NN-IDV) [161] employs a k-Nearest Neighbor
(kNN) model with distance measurements, and statistical tests are used to
specify the region with drift occurrence. Other studies attempt to alleviate
the unsatisfactory results of unsupervised detection. The supervised embedding
training, optimized through the incorporation of intra-class and inter-class
relationships with cluster centroids, enables the prediction of real drift by
means of distance comparison [162]. Recent advances have tied in with deep
learning. Learning a generative model based on the learned historical latent
distribution can detect possible drift clues [163]. The data generator can
adapt fast by forecasting the drift before it happens rather than detecting
the drift that is already happened.

###  5.2 Drift Adaptation

The drift adaptation can be conducted either actively or passively. Active
adaptation ($\S$5.2.1) explicitly performs detection on the data stream and
adapts to the new data once detects the drift. Passive adaptation ($\S$5.2.2)
methods instead update the model continuously to ensure predictive ability
without degradation. As for the model updating scheme, both active and passive
learning approaches have adopted either partial updating, full updating, or
ensemble updating. Note that passive adaptation approaches are tightly
associated with the development of partial updating because frequent updates
might increase computational overhead.

####  5.2.1 Active Adaptation

These approaches heavily rely on the drift detection approaches described in
$\S$5.1. Right after the drift is detected, most studies focus on model
retraining straightforwardly. Vector quantization approaches are commonly used
to generate prototypes which can then be adapted with the parameters after
drift detection [164]. A recent study in [165] moved toward temporal deep
learning, by combining the Long Short-Term Memory (LSTM) with an ensemble
learning strategy, to estimate a drift factor and assist the adaptation of
short-text classification. The main issue of the active adaptation method is
that false alarms might trigger inefficient adaptation. Identifying different
types of drift might be a direction to customize the algorithm for the
characteristics of the drift. For example, Liu et al. [166] proposed a diverse
instance-weighting ensemble approach for regional drift, which resulted in an
excessive number of false alarms during adaptation.

####  5.2.2 Passive Adaptation

These methods often involve the utilization of evolution algorithms, which
gradually adapt the model to ensure robustness in new environments [167].
Automatic adjusting of the learned parameters of the clusters using stochastic
gradient descent techniques can enhance model capability against variation
[168]. The design of network modules that can automatically determine adapted
parameters is advantageous for modern deep networks. For instance, in [169],
hybrid spiking neurons embedded in LSTM can provide an attention mechanism for
multivariate time series. This approach avoids the requirement of pre-set
variance into the standard surrogate gradient. Automatic parameter adjustment
can also be achieved by carefully designing the loss function. In [170], the
group optimization algorithm regularizes the disparity between temporally-
split groups. This approach is inspired by group disparity among different
attributes of data (e.g., gender and age). Due to the limited accessed data in
a time window, switching the adaptation between the minority and majority
classes is essential for the class imbalance issue [171].

Passive adaptation approaches usually accommodate active adaptation schemes
through updates triggered by an additional drift detector. Passive adaptation
approaches usually come with more complex issues such as adaptation efficiency
and class imbalance problems.

###  5.3 Label Drift

While previous research on concept drift has predominantly focused on changes
in $P(X|Y)$, the drift of labels where $P(Y)$ changes, has received relatively
less attention. Moreover, there are certain factors not adequately addressed
in previous concept drift studies, such as manual labeling drift. This drift
is intricately connected to the properties of the task at hand and the
cognitive states of the annotators involved. Therefore, annotation quality
control is crucial to effectively detect instances where the annotators’
performance declines ($\S$5.3.1). Other research further proactively involves
human in the loop ($\S$5.3.2) to foster workflows in real-world scenarios.

####  5.3.1 Annotation Quality Control

Robustly detecting and adapting to label drift typically involves human manual
annotations and designed algorithms to assess annotation quality. Modeling
labeling quality control involves gathering a set of gold standard labels to
assess and enhance the quality of annotation workers as well as other labels
[172]. To maintain continuous labeling quality [173], it is important for
multiple annotators to maintain consistent rules and criteria for the labeling
procedure that can maintain inter-annotator variability. However, online
detection of annotation bias is non-trivial and requires automatic algorithms
for large-scale evaluation. Tracking the annotation workers on crowdsourcing
platforms is essential to maintain quality and enable the modeling of labeling
behaviors to facilitate automatic real-time monitoring [174]. Worker features
are generated from their actions, including mouse clicks, scrolling, key
presses, and focus changes. These features capture temporal behaviors by
calculating event durations and time intervals between events. Certain
attributes exhibit varying time scales influenced by human life experiences,
biological status, and internal states. Preference modeling for recommendation
systems is inherently shaped by personal thoughts and societal events. Time-
aware recommendation systems utilize matrix factorization techniques to
decompose and identify latent factors associated with temporal dynamics [175].
Human manual annotation of time series data involves continuous labeling
within a streaming process, which is influenced by cognitive load and response
delay. For instance, modeling physiological data [176] using aggregated labels
from multiple annotators leads to improved performance.

####  5.3.2 Human In The Loop

We now explore techniques relevant to human-in-the-loop model learning, which
includes research on detecting human annotator performance and incorporating
humans’ next actions. In particular, active learning plays a significant role
by providing data sampling and adaptive model learning approaches to guide the
labeling of the next set of data in an iterative process. Initially, a model
is trained with a small amount of data and selects a subset of data for the
annotator to label, subsequently updating the model with the newly labeled
data.

Developing effective active learning strategies ensures stable model learning
while considering labeling, storage, and computational costs [177]. Key
techniques in active learning include uncertainty-based, representation-based,
and hybrid approaches [178]. These approaches offer optimal strategies for
efficient and cost-effective model learning. Active learning strategies need
to consider different data input forms. In the pool-based setting, a pool of
unlabeled data is available, while the stream-based setting has limited
choices of query samples due to the data being obtained in a stream. In terms
of addressing label drift, the setting is similar to the stream-based setting.
The pool-based approach can be seen as an adaptive strategy within the semi-
supervised domain adaptation framework, involving humans in the process. One
prevalent issue in active learning approaches is the cold start problem, where
the initial model has limited data to accurately select key unlabeled samples.
Another human-engaged approach, prompt learning, also faces similar challenges
when model updating is guided by inaccurate queries [179]. Recently, Hacohen
et al. highlighted that low-budget and high-budget scenarios require different
sample selection strategies [180]. In the low-budget case, actively selecting
similar samples to the current dataset can aid model initialization.
Conversely, the high-budget case involves the concept of hard-sample mining to
enhance the model’s discriminative power.

###  5.4 Continual Learning

Catastrophic forgetting in data streams can result in significant failures in
model prediction [181]. To mitigate this issue, continual learning, also known
as life-long learning, has been developed to update models without causing
forgetting effects. Continual learning enables the consolidation of knowledge,
allowing models to handle new information while retaining previously learned
knowledge. One approach, Neural Network with Dynamically Evolved Capacity
(NADINE), employs adjustable layers to perform partial updating of the
network, preventing severe forgetting during updates [182]. A survey on these
techniques can be found in [36]. Interestingly, in the context of long-term
adaptation where accumulating vast amounts of new data is cumbersome, some
forgetting can be beneficial [183]. Consequently, the goal of these algorithms
is to update new knowledge smoothly while minimizing the rapid loss of
previously acquired knowledge.

The comprehensive exploration of continual learning frameworks in the context
of concept drift is still relatively limited. Most existing research on
continual concept drift primarily focuses on adapting to upcoming new
instances using incremental learning techniques. However, the development of
adapted models for new classes and tasks, in addition to handling domain
differences over time, has received little attention [184]. Only a small
number of recent studies have begun to unify continual learning with new
classes in the concept drift framework, particularly for incrementally
learning class-imbalanced new data [185]. For example, Darem et al. utilize an
incremental learning approach to integrate new malware detection capabilities
without sacrificing the detection performance on existing instances [186].
Similarly, Taufique et al. redesign domain shift datasets to simulate gradual
changes inspired by the concept drift setting [187]. Continual learning serves
as an intermediate research topic that bridges the fields of domain shift and
concept drift, as researchers from both domains strive to advance knowledge in
this area.

TABLE VI:  Techniques commonly used in each section. Superscript number
denotes the occurrence count of each technique in the literature. Referred to
Fig. 1 for the three-phase problem detection scheme and Table VII for the
detailed organization of the techniques. Bold texts highlight techniques used
in both fields of domain shift and concept drift.

Domain Shift Phase Section Techniques I Out-of-Domain Detection Self-
Supervised Learning5 [83, 84], Distribution Learning3 [82, 86], Performance
Monitoring1 [80] II Domain Generalization Domain Adaptation Data Manipulation3
[93], Self-Supervised Learning5 [94, 104], Regularization4 [98], Distribution
Learning3 [15, 16], Feature Alignment2 [102, 103], Sample Selection2 [68],
Teacher-Student Learning3 [95, 97], Knowledge Distillation1 [69], Partial
Network Update3 [96], Network Architecture Design1 [100] III Label Shift Data
Manipulation3 [115, 116], Self-Supervised Learning5 [115], Regularization4
[108, 116], Distribution Learning3 [17], Teacher-Student Learning3 [113],
Sample Selection2 [114], Loss Adjustment2 [110] III Fast Adaptation Data
Manipulation3 [119], Self-Supervised Learning5 [128], Regularization4 [129],
Meta Learning2 [118], Teacher-Student Learning3 [122], Partial Network Update3
[188, 135, 136, 137, 138] III Test-Time Training/ Test-Time Adaptation Self-
Supervised Learning5 [140, 144], Regularization4 [148, 144], Meta Learning2
[141, 142], Feature Alignment2 [146], Partial Network Update3 [149] Loss
Adjustment2 [148] Concept Drift Phase Section Techniques I Drift Detection
Distribution Learning1 [159, 161, 162, 163], Ensemble3 [189, 190, 162, 155],
Meta Learning1 [157, 157], Performance Monitoring2 [39, 154], Time Series
Model1 [163, 169] II Drift Adaptation Regularization2 [170], Ensemble3 [165,
166], Partial Network Update2 [168], Network Architecture Design1 [169] III
Label Drift Ensemble3 [171], Factorization1 [175], Performance Monitoring2
[172, 174], Sample Selection1 [177, 179, 180], Loss Adjustment1 [170] III
Continual Learning Partial Network Update2 [182]

TABLE VII:  Technical terms referred in the data change problems and their
corresponding techniques. The focused domain categorizes algorithms based on
their emphasis on either pretraining (source domain) or finetuning (target
domain). The feature and label columns represent the primary technical design
spaces for learning, as depicted in Fig. 4.

Technical Term Focused Domain Feature Label Related Techniques Data
Manipulation Source ✓ Data Augmentation, Pseudo Labels, Batch Normalization
Self-Supervised Learning Source ✓ ✓ Contrastive Loss Regularization Source ✓
Stochastic Weights Averaging Distribution Learning Source ✓ Generative Model,
Density Estimation, Clustering, Data Imputation Ensemble Source ✓ ✓ Mixture-
of-Expert Meta-Learning Source ✓ Auxiliary Learning Time Series Model Source ✓
Autoregression Network Architecture Design Source ✓ Transformer Feature
Alignment Both ✓ Cluster Matching Factorization Both ✓ Disentangle
Representation Adversarial Learning Both ✓ Generative Adversarial Network,
Gradient Adversarial Learning Performance Monitoring Both ✓ Statistical Test
Reinforcement Learning Target ✓ Domain Adaptive Imitation Learning Sample
Selection Target ✓ Active Learning Teacher-Student Learning Target ✓ ✓
Curriculum Learning Knowledge Distillation Target ✓ ✓ Self-distillation,
Subspace Learning Partial Network Update Target ✓ Prompt Learning, Prefix
Learning, Adapter Loss Adjustment Target ✓ Uncertainty Estimation Continual
Learning Target ✓ Life-Long Learning

##  6 Cross-Field Discussion of Data Change

We utilized the proposed three-phase problem categorization scheme to review
research on domain shift and concept drift in $\S$4 and $\S$5. We identified
several overlapping techniques that have been recently developed in both
fields, and we will discuss these in $\S$6.1, where we will frequenly make
correspopndences to the respective sections. $\S$6.2 presents distinct methods
that address the original problems of each field, which may complement one
another. To provide a comprehensive overview of the technology trends bridging
these two fields within the context of data change, Table VI presents a list
of state-of-the-art algorithms.

Figure 4:  Common learning schemes on the feature and label spaces. (a)
Feature alignment: The encoder learns to map samples from different domains
onto a shared latent space. (b) Adversarial learning: An additional
discriminator is employed to distinguish between real and fake labels. (c)
Teacher-student learning: Constraints are imposed on feature embeddings and
classifiers. (d) Self-supervised learning: Pre-trained networks are
generalized using contrastive loss to facilitate downstream tasks.

###  6.1 Similar Underlying Structures and Techniques

The research topics presented in Fig. 1 are categorized into three phases
based on the shared structure of domain shift and concept drift problems. The
first phase, problem detection, contains the OOD ($\S$4.1) in-domain shift and
drift detection ($\S$5.1) for concept drift. Once the data change is detected,
the second phase, problem-handling, involves adaption techniques. Extensive
efforts have been made in both domain shift ($\S$4.2) and concept drift
($\S$5.2) to adapt models for up-to-date usage. The third phase focuses on
other factors, such as labeling issues recognized as label shift in the domain
shift field ($\S$4.3) and label drift in the concept drift field ($\S$5.3).
Time-related factors are emerging topics in domain shift, including Target
Task Transfer (TTT) ($\S$4.5) and Fast Adaptation ($\S$4.4), which
incidentally align with concept drift for efficient model updating. Continual
learning in concept drift ($\S$5.4) even leverages similar technical
approaches as fast adaptation and TTT.

Table VI provides an overview of cutting-edge techniques proposed in state-of-
the-art studies. The listed methods offer valuable insights into addressing
the challenges in domain shift and concept drift. Distribution learning and
performance monitoring are common strategies employed during the detection
phase. Regularization and partial network updating have proven to be effective
approaches for adaptation in both domain shift and concept drift. Sample
selection, particularly in the context of active learning, has been widely
explored to mitigate label shift and label drift. Overall, both domain shift
and concept drift offer a rich array of methods to address the negative
effects of changes. The aforementioned techniques serve as general solutions
applicable across different phases of these problems.

###  6.2 Unique and Complementary Properties

The domains of domain shift and concept drift have historically emerged from
separate communities, driven by distinct problem setups that naturally led to
different intuitive directions. Domain shift arises when encountering a new
batch of data, resulting in the development of frameworks centered around
distribution-based learning. In Table VI, self-supervised learning and
teacher-student learning have been extensively developed, indicating a focus
on distribution modeling. Concept drift is characterized by the inherent
streaming constraint, requiring sequential processes for detection,
adaptation, and learning. Consequently, statistical-based performance
monitoring, coupled with ensemble methods, offers a straightforward approach
to robustly maintain an up-to-date model. Note that in Table VI, distribution
learning primarily pertains to drift detection rather than adaptation, as
manipulating the entire distribution during adaptation can be costly given the
streaming constraints. These inherent differences in technical advancements
reflect the original targeted problems within the respective fields,
showcasing the unique challenges and solutions specific to the two fields.

Table VII and Fig. 4 illustrate the primary learning space of framework
designs and major focused domains, providing guidance for selecting
appropriate algorithms to address specific data change problems. Algorithms
targeting the domain shift problem often consider constraints such as limited
retraining resources and inaccessible source domain data. In contrast,
approaches focusing on the source domain benefit from leveraging powerful pre-
trained models trained on large-scale data to capture generic patterns. While
the domain shift field typically emphasizes the source domain, recent studies
have emerged that develop target domain-focused approaches built upon well-
trained source domain models. Additionally, within the domain shift research,
there is a growing consideration of temporal change as a factor influencing
data distribution shift. On the other hand, the concept drift field leverages
deep learning to expedite model performance. The potential to integrate these
research advancements and reframe the problem is becoming increasingly
evident.

##  7 Industrial Perspective to Data Change

In dynamic environments of the industries, data undergo rapid changes aligned
with market trends. $\S$ 7.1 discusses the challenges associated with
deploying models in such contexts. $\S$ 7.2 explores two real-world case
studies.

###  7.1 Model Deployment

Deploying models for real-world industrial applications presents substantial
challenges. Modern research efforts are addressing concerns regarding model
size, data availability, and usability issues. These issues become more
complicated as data undergoes change, creating new dimensions for model
development.

####  7.1.1 Efficiency Constraint

As the number of model parameters and the frequency of model training continue
to grow, efficient management of resources becomes increasingly crucial.
Solutions for domain shift and concept drift often necessitate the development
of intricate frameworks to effectively handle rapid data changes over time.
However, the trend of training large models has led to a significant increase
in computational demands, raising concerns about sustainability. The research
focus has shifted towards data-efficient learning, as highlighted in [191].
Cutting-edge advancements now underscore the significance of efficient
frameworks, prioritizing factors such as low carbon footprints, compact model
sizes, low computational complexity, and transparent logic. The objective is
to achieve maximized model performance while incorporating reusable and
traceable characteristics. Beyond enhancing model design, there is a growing
emphasis on the human engagement throughout the model’s usage process.

For the data change problem, a handful of research has laid the groundwork for
efficient learning. Notably, techniques such as few-shot learning and
foundation models discussed in $\S$4.4 on the fast adaptation can be leveraged
to reuse pre-trained models with little adaptation for reducing the required
model size. However, the demand concerning timely adaptation, such as test-
time adaptation ($\S$4.5) and concept drift ($\S$5), introduces a new
dimension in resource consumption control. The cumulative consumption over
time can become substantial. Hence, we emphasize that research fields focusing
on temporal aspects should prioritize green learning as a primary direction in
the future.

To enhance sustainability further, an important aspect is to harness human
expertise, which directly aligns with the research field of human-in-the-loop
($\S$5.3.2). Current studies have explored efficient label acquisition, aiming
to reduce human effort while improving model performance. Explainable AI
emerges as a promising field that involves humans in the process, ensuring
transparent results. By incorporating humans and machines in a loop, we
envision the model being updated through human guidance and humans expanding
their knowledge through systematic updates. Despite the progress made in
addressing data change, there remains a limited number of studies that
investigate interpretable reasons for changes and the resulting adaptations,
as most evaluations still rely on prediction accuracy, leaving room for
exploration in the interpretability.

The findings in our study reveals a growing convergence in the research fields
of domain adaptation and concept drift. The use of pre-trained models and
representation learning techniques in domain adaptation can complement the
statistical tests and temporal modeling approaches employed in concept drift.
The combination of transferable pre-training models and interpretable
statistical analysis holds the potential to establish a new paradigm for
trustworthy collaboration between humans and machines. Technical advancements
in temporal efficiency can encourage fine-grained sustainability assessment.
Contemporary research should adopt the accuracy, model size, and speed for the
performance assessment regarding adaptation. A systematic metric to assess the
level of human-machine collaboration in these solutions is still lacking.

####  7.1.2 Model Robustness and Maintenance

Model deployment presents a range of challenges for sustainability,
specifically in terms of model robustness and iteration maintenance. Building
robust accuracy that can withstand diverse noises in features and labels is
crucial for establishing trust in machine predictions. These noises can
originate from natural variations or deliberate insertions.

The field of learning from noisy labels ($\S$4.3) is an example of robust
knowledge extraction from noisy data in the source domain. Adversarial
training algorithms are proposed to keep samples away from decision
boundaries, preventing unforseeing changes in model predictions under
adversarial perturbations [192, 193]. The OOD challenges described in $\S$4.1
can be viewed as another kind of robustness volunability that might lead to
unexpected results. Further research is needed to investigate the robustness
of models against adversarial attacks and out-of-distribution (OOD) scenarios
across various contexts. For example, prompting ChatGPT [194] with typos,
distracted sentences, or abnormal prompts should not yield socially
detrimental outcomes.

Implementing algorithms to address the challenges of data change relies
heavily on the infrastructure, as insufficient deployment speed can lead to
delays in detecting and adapting to these changes. Continuous Training (CT) is
a concept established upon standard Machine Learning Operations (MLOps) to
provide model retraining as a pipeline. When data change occurs, a series of
retraining steps can directly update the model with new data for deployment.
Moreover, the development of algorithms is significantly influenced by public
resources, such as open-sourced frameworks that ensure code consistency, as
well as the collaborative efforts within communities.

####  7.1.3 Usability

We organize the usability challenges related to data change into three main
categories: privacy, fairness, and decision-making schemes. These factors
significantly influence the user’s willingness to adopt AI technologies. We
divide the AI users into two groups: professional users and general users.
Professional users leverage AI models to streamline their work tasks and
prioritize decision-making schemes that enhance task value creation. General
users are more inclined to enjoy AI services as long as privacy protection and
fairness awareness designs are integrated.

Privacy: Maintaining privacy during data change is crucial due to the
increased risks associated with reusing source domain data, adapting to target
domain data, and the specific application scenarios. The level of privacy
protection needed depends on the context. Most studies assume that public data
from the source domain are readily available, while the target domain data are
scarce and private. Differential Privacy (DP) [195] and Federated Learning
(FL) [196] are two technical branches to tackle the data privacy and model
privacy issues. Another research direction of source-free domain adaptation
discussed in $\S$4.2.1, aims to enhance the privacy conditions by prohibiting
access to the source domain data. This approach aligns with the principles of
data-efficient learning discussed in $\S$7.1.1, which seeks to maximize
performance while minimizing data usage. In terms of time considerations, FL
has been explored for concept drift [197, 198]. Additionally, privacy leakage
can be seen as a potential source of drift that requires detection and
adaptation.

Fairness: The fairness issue is naturally tied to data change, as it involves
biased data distributions. The bias can occur across various batches of data
(domain shift) or data streams (concept drift). The main difficulty is on how
best to define fairness among individuals or groups. A recent study [199]
treats fairness issues as problems of domain shift and concept drift. It is
worth noting that data change introduces potential risks of unfair
distributions, even when the source domain has been calibrated [200].
Adversarial learning has emerged as a promising technique in addressing
fairness in both domain adaptation and fairness representation learning,
although the choice of objective function remains crucial in determining the
final performance [201]. Therefore, identifying appropriate definitions and
assessment metrics for fairness remains an ongoing research topic in the
context of data change. It is important to consider fairness assessment during
the timely detection and adaptation processes discussed in the concept drift
section. While research has focused on addressing temporal dynamics for robust
accuracy, the drift of fairness over time has been overlooked.

Decision-Making Scheme: The integration of automatic models has foundamentally
transformed the decision making process. When faced with rapidly changing
data, how best to leverage data-driven AI models for decision making becomes
challenging. Studied reviewed in $\S$5.3.2 have provided a basic process under
active learning. However, existing human-machine collaboration schemes are
limited to the enhancements of annotation strategies. In practice, guidelines
for human feedbacks can greatly facilitate human engagement and effective
acquisition of human expertise.

Our discussions regarding concept drift (specifically label drift in $\S$5.3)
imply that model prediction can come with unawared bias that may corrupt the
outcome. Therefore, the decision-making scheme involving humans in the loop
should not be limited to a one-way feedback loop, but should instead encompass
a mutually corrected iterative procedure.

Addressing the aforementioned topics is crucial for establishing trust for
human users. While existing literature has made progress, the solutions and
strategies to maintain usability during data change remains an ongoing
endeavor. It is evident that there is a need for additional evaluation metrics
beyond accuracy-based measures, to assess the effectiveness of data change
solutions. These issues will continue to be a focus of research as the fields
of AI advance and evolve.

TABLE VIII: Common data change scenarios in real-world applications.

Domain Shift Concept Drift Manufacturing Product variability Product line
Operating pipeline* Machine configuration* Decision variability Environment
Healthcare Disease category Region, Race, Gene Device, Equipment* Decision
variability Population change Diagnosis rule* Robotics Use case Environment

* denotes that the cases rarely occur.

###  7.2 Real-world Applications

We categorize research works concerning real-world applications based on their
task and algorithmic characteristics. We note that there is universal solution
for all real-world scenarios. The broad range of applications include smart
manufacturing [202, 203], smart healthcare [4, 7], smart automotive [203],
internet-of-things [204, 4], voice assistants [203], and recommendation
systems [10, 11]. Data change occurs in different frequencies and degrades the
learned model in various levels. We next focus discussion on smart
manufacturing in $\S$7.2.1 and smart healthcare in $\S$7.2.2. Table VIII lists
common scenarios of data change in real-world scenarios.

####  7.2.1 Smart Manufacturing

Smart manufacturing tasks typically involve defect detection, process
optimization, and inventory forecasting. In Table VIII, common data change
problems can pose significant challenges in manufacturing applications. For
example, the introduction of new product lines and the variability within a
product category result in domain shifts. Even different versions of the same
product category exhibit domain shifts. Additionally, changes in the operation
pipeline or machinery can lead to rare yet substantial domain shifts. The
renewal of working policies or hardware infrastructures also contributes to
dramatic shifts. Concept drift, on the other hand, is often associated with
human factors such as tiredness or cognitive biases, leading to label drift.
Environmental changes over time, such as background noise and lighting
conditions, can be overlooked by humans.

It is important for smart manufacturing models to operate with high
robustnessẆhile out-of-distribution events might be rare in the product line
due to the unified pipeline control, it remains crucial to establish
protective mechanisms to mitigate the potential impact of unknown data
classes.

Industrial machines often possess superior computational power compared to
consumer devices, allowing for the deployment of models without compromising
the performance or accuracy. In such case, fairness and privacy issues are
thus less influential as the users are not external individuals. Decision-
making schemes would focus on incorporating automatic machine-assisted
predictions with the human operating pipeline.

####  7.2.2 Smart Healthcare

In healthcare applications and clinical practice, stringent data quality
control and meticulous model deployment requirements are important. Table VIII
highlights various scenarios encountered in these contexts, such as diverse
disease categories, regional disparities, and variations in equipment [205].
Most diseases exhibit subtypes characterized by distinct phenotypic
attributes. Meanwhile, data collection across different regions introduces
diverse subject groups with variations in race and genetics. Inconsistent
expert opinions can lead to discrepant labels, resulting in label noise
stemming from biased individual annotations [206]. These instances of domain
shift contribute to the existence of datasets with varying properties.

Concept drift in healthcare encompasses not only variations in human decision-
making but also changes in population dynamics over time [205]. An example of
a relatively uncommon concept drift case is the update of diagnostic rules
prompted by newly recognized consensus within the medical community.

We next consider a clinical scenario in a medical center to illustrate the
data change impacts. The centralized services run on the server machines of
the hospitcal servers typically come with high performance. On the other hand,
the requirements regarding robustness, privacy, and fairness concerns become
critical, as they can directly impact treatment feasibility. The effectiveness
of AI models in clinical practice hinges on a well-designed decision-making
scheme that incorporates medical specialists. To achieve successful
integration into the clinical pipeline, interpretable AI and feedback
mechanisms are preferred.

##  8 Conclusion

Ensuring the model’s generalization capability is crucial for leveraging AI in
real-world applications. The challenge of data change encompasses the study of
domain shift and concept drift across various fields. This paper provides a
systematic framework that guides readers in understanding these two fields
through a unified three-phase problem categorization scheme. Despite the
distinct research perspectives that have shaped the development of these
fields, we have highlighted the presence of common underlying properties that
can complement each other. By recognizing and leveraging these shared
characteristics, valuable insights are obtained to better address the data
change challenges.

Through an investigation of state-of-the-art studies, we have identified
several trends across various technical fields. Notably, self-supervised
learning has emerged as a key factor in enhancing models’ capabilities to
handle diverse data changes. Additionally, innovative designs in
regularization approaches, including novel loss functions and network layer
techniques, have proven effective. Temporal modeling techniques have gained
significance, driven by the need for efficient solutions, leading to the
emergence of new research branches. Partial network updating strategies, such
as prompt and adapter methods, have become valuable for adapting to changes
and are closely associated with continual learning. More and more domain shift
methods consider temporal change as a factor of data distribution shift.
Regarding concept drift, model performance keeps increasing as new deep
learning approaches are developed. This progress opens up possibilities for
integrating these developments and reframing the problem at hand.

This paper also addresses an important research question regarding the
development of model deployment in applications. We emphasize the multifaceted
challenges associated with model efficiency, robustness, and other usability
concerns, including privacy, fairness, and decision-making schemes. By
reviewing accessible datasets and benchmarking evaluations, this paper
facilitates model deployment developments. We also select two industrial
applications to carry out case studies and show the different needs among
real-world applications.

Future Directions. Our comprehensive cross-field review has identified
important future directions for addressing the data change problem. Firstly,
further research is needed to develop advanced modeling approaches that
efficiently capture temporal dynamics. This involves exploring network
architectures and adaptation strategies that minimize frequency and parameter
quantity during network updating. Effective data change detection methods can
indicate the optimal timing for adaptation, preventing unexpected model
failures. Secondly, achieving model generalization for adaptation to any data
change is a major focus, leveraging self-supervised learning and continual
learning techniques. The aim is to establish a large foundation model that
reduces efforts for downstream tasks, aligning with efficient learning
approaches. Thirdly, incorporating humans in the loop is a critical but
challenging next step. Understanding human behaviors in the context of label
shift and drift problems is currently neglected, limiting the sustainability
of real-world applications involving human interaction. Exploring a human-
machine cooperation paradigm is essential for effectively addressing data
change challenges.

Studies on domain shift have yielded remarkable results in modeling
distribution, offering a foundation for constructing pre-trained models
applicable to various tasks. In the realm of concept drift research,
considerations of efficiency and explainability are key factors when designing
methods for timely adaptation. This review paper uncovers both shared
techniques and distinct characteristics between these two fields. We advocate
for further research to advance reliable model learning, thereby accelerating
the realization of trustworthy AI.

## References

  * [1] D. Alvarez-Coello, D. Wilms, A. Bekan, and J. M. Gómez, “Towards a data-centric architecture in the automotive industry,” _Procedia Computer Science_ , vol. 181, pp. 658–663, 2021.
  * [2] U. Jayasinghe, A. Otebolaku, T.-W. Um, and G. M. Lee, “Data centric trust evaluation and prediction framework for iot,” in _ITU K_ , 2017, pp. 1–7.
  * [3] M. H. Jarrahi, A. Memariani, and S. Guha, “The principles of data-centric ai (dcai),” _arXiv:2211.14611_ , 2022.
  * [4] L. Zhang and X. Gao, “Transfer adaptation learning: A decade survey,” _IEEE TNNLS_ , 2022.
  * [5] J. Jiang, Y. Shu, J. Wang, and M. Long, “Transferability in deep learning: A survey,” _arXiv:2201.05867_ , 2022.
  * [6] C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, “A survey on deep transfer learning,” in _ICANN_. Springer, 2018, pp. 270–279.
  * [7] W. M. Kouw and M. Loog, “An introduction to domain adaptation and transfer learning,” _arXiv:1812.11806_ , 2018.
  * [8] H. Hu, M. Kantardzic, and T. S. Sethi, “No free lunch theorem for concept drift detection in streaming data classification: A review,” _WIREs DMKD_ , vol. 10, no. 2, p. e1327, 2020.
  * [9] S. Wares, J. Isaacs, and E. Elyan, “Data stream mining: methods and challenges for handling concept drift,” _SN Applied Sciences_ , vol. 1, pp. 1–19, 2019\.
  * [10] A. S. Iwashita and J. P. Papa, “An overview on concept drift learning,” _IEEE Access_ , vol. 7, pp. 1532–1547, 2019.
  * [11] J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang, “Learning under concept drift: A review,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 31, no. 12, pp. 2346–2363, 2019.
  * [12] P. A. Nayak, P. Sriganesh, K. Rakshitha, M. Manoj Kumar, B. S. Prashanth, and H. R. Sneha, “Literature review on phenomenon of concept drift and its handling approaches,” in _ASIANCON_ , 2021, pp. 1–7.
  * [13] M. A. Hashmani, S. M. Jameel, M. Rehman, and A. Inoue, “Concept drift evolution in machine learning approaches: a systematic literature review,” _International Journal on Smart Sensing and Intelligent Systems_ , vol. 13, no. 1, pp. 1–16, 2020.
  * [14] A. Farahani, S. Voghoei, K. Rasheed, and H. R. Arabnia, “A brief review of domain adaptation,” _ICDATA and IKE_ , pp. 877–894, 2021.
  * [15] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain _et al._ , “Domain-adversarial training of neural networks,” _The journal of machine learning research_ , vol. 17, no. 1, pp. 2096–2030, 2016.
  * [16] M. Xu, J. Zhang, B. Ni, T. Li, C. Wang, Q. Tian, and W. Zhang, “Adversarial domain adaptation with domain mixup,” _AAAI Conference Proceedings_ , vol. 34, no. 04, pp. 6502–6509, 2020.
  * [17] X. Yu, T. Liu, M. Gong, K. Zhang, K. Batmanghelich, and D. Tao, “Label-noise robust domain adaptation,” in _ICML_. PMLR, 2020, pp. 10 913–10 924.
  * [18] T. Lesort, M. Caccia, and I. Rish, “Understanding continual learning settings with data distribution drift analysis,” _arXiv:2104.01678_ , 2021.
  * [19] P. Ceravolo, G. M. Tavares, S. B. Junior, and E. Damiani, “Evaluation goals for online process mining: A concept drift perspective,” _IEEE Transactions on Services Computing_ , vol. 15, no. 4, pp. 2473–2489, 2022.
  * [20] W. M. Kouw and M. Loog, “A review of domain adaptation without target labels,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 43, no. 3, pp. 766–785, 2021.
  * [21] X. Liu, C. Yoo, F. Xing, H. Oh, G. E. Fakhri, J.-W. Kang, and J. Woo, “Deep unsupervised domain adaptation: A review of recent advances and perspectives,” _APSIPA Transactions on Signal and Information Processing_ , vol. 11, no. 1, pp. –, 2022.
  * [22] S. Zhao, X. Yue, S. Zhang, B. Li, H. Zhao _et al._ , “A review of single-source deep unsupervised visual domain adaptation,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 33, no. 2, pp. 473–493, 2022.
  * [23] N. L. A. Ghani, I. A. Aziz, and M. Mehat, “Concept drift detection on unlabeled data streams: A systematic literature review,” in _ICBDA_ , 2020, pp. 61–65.
  * [24] C. Fahy, S. Yang, and M. Gongora, “Scarcity of labels in non-stationary data streams: A survey,” _ACM Comput. Surv._ , vol. 55, no. 2, 2022.
  * [25] S. Agrahari and A. K. Singh, “Concept drift detection in data stream mining : A literature review,” _Journal of King Saud University - Computer and Information Sciences_ , vol. 34, no. 10, Part B, pp. 9523–9540, 2022.
  * [26] Z. Yu, J. Li, Z. Du, L. Zhu, and H. T. Shen, “A comprehensive survey on source-free domain adaptation,” _arXiv:2302.11803_ , 2023.
  * [27] A. L. Suárez-Cetrulo, D. Quintana, and A. Cervantes, “A survey on machine learning for recurring concept drifting data streams,” _Expert Systems with Applications_ , vol. 213, p. 118934, 2023.
  * [28] S. M. Jameel, M. A. Hashmani, H. Alhussain, M. Rehman, and A. Budiman, “A critical review on adverse effects of concept drift over machine learning classification models,” _IJACSA_ , vol. 11, no. 1, 2020.
  * [29] K. Zhou, Z. Liu, Y. Qiao, T. Xiang, and C. C. Loy, “Domain generalization: A survey,” _IEEE PAMI_ , no. 01, pp. 1–20, 2022.
  * [30] M. Lima, M. Neto, T. S. Filho, and R. A. de A. Fagundes, “Learning under concept drift for regression—a systematic literature review,” _IEEE Access_ , vol. 10, pp. 45 410–45 429, 2022.
  * [31] M. Fan, Z. Cai, T. Zhang, and B. Wang, “A survey of deep domain adaptation based on label set classification,” _Multimedia Tools Appl._ , vol. 81, no. 27, p. 39545–39576, 2022.
  * [32] S. Wang, L. L. Minku, and X. Yao, “A systematic study of online class imbalance learning with concept drift,” _IEEE TNNLS_ , vol. 29, no. 10, pp. 4802–4821, 2018.
  * [33] D. Saunders, “Domain adaptation and multi-domain adaptation for neural machine translation: A survey,” _Journal of Artificial Intelligence Research_ , vol. 75, pp. 351–424, 2022.
  * [34] X. Guo and H. Yu, “On the domain adaptation and generalization of pretrained language models: A survey,” _arXiv:2211.03154_ , 2022.
  * [35] J. Y.-L. Chan, K. T. Bea, S. M. H. Leow, S. W. Phoong, and W. K. Cheng, “State of the art: a review of sentiment analysis based on sequential transfer learning,” _Artificial Intelligence Review_ , vol. 56, no. 1, pp. 749–780, 2023.
  * [36] L. Yuan, H. Li, B. Xia, C. Gao, M. Liu, W. Yuan, and X. You, “Recent advances in concept drift adaptation methods for deep learning,” in _IJCAI_ , 2022, pp. 5654–5661.
  * [37] H. Yadav and S. Sitaram, “A survey of multilingual models for automatic speech recognition,” in _LREC_ , 2022, pp. 5071–5079.
  * [38] P. Bell, J. Fainberg, O. Klejch, J. Li, S. Renals, and P. Swietojanski, “Adaptation algorithms for neural network-based speech recognition: An overview,” _IEEE Open Journal of Signal Processing_ , vol. 2, pp. 33–66, 2021.
  * [39] F. Bayram, B. S. Ahmed, and A. Kassler, “From concept drift to model degradation: An overview on performance-aware drift detectors,” _Knowledge-Based Systems_ , vol. 245, p. 108632, 2022.
  * [40] Z. Zhu, K. Lin, and J. Zhou, “Transfer learning in deep reinforcement learning: A survey,” _arXiv:2009.07888_ , 2020.
  * [41] T. Mahmood and T. Fatima, “Concept drift in streaming data: A systematic literature review,” _KIET Journal of Computing and Information Sciences_ , vol. 4, no. 1, pp. 17–17, 2021.
  * [42] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz, “Invariant risk minimization,” _arXiv:1907.02893_ , 2019.
  * [43] M. Ghifary, W. B. Kleijn, M. Zhang, and D. Balduzzi, “Domain generalization for object recognition with multi-task autoencoders,” in _ICCV_ , 2015, pp. 2551–2559.
  * [44] D. Hendrycks and T. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” in _ICLR_ , 2019.
  * [45] J. Sun, Q. Zhang, B. Kailkhura, Z. Yu, C. Xiao, and Z. M. Mao, “Benchmarking robustness of 3d point cloud recognition against common corruptions,” _arXiv:2201.12296_ , 2022.
  * [46] X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and K. Saenko, “Visda: The visual domain adaptation challenge,” _arXiv:1710.06924_ , 2017.
  * [47] H. Song, M. Kim, and J.-G. Lee, “SELFIE: Refurbishing unclean samples for robust deep learning,” in _ICML_ , vol. 97. PMLR, 2019, pp. 5907–5915.
  * [48] J. Wei, Z. Zhu, H. Cheng, T. Liu, G. Niu, and Y. Liu, “Learning with noisy labels revisited: A study using real-world human annotations,” in _ICLR_ , 2022.
  * [49] T. Xiao, T. Xia, Y. Yang, C. Huang, and X. Wang, “Learning from massive noisy labeled data for image classification,” in _CVPR_ , 2015, pp. 2691–2699.
  * [50] W. Li, L. Wang, W. Li, E. Agustsson, and L. Van Gool, “Webvision database: Visual learning and understanding from web data,” _arXiv:1708.02862_ , 2017\.
  * [51] D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Deeper, broader and artier domain generalization,” in _ICCV_ , 2017, pp. 5542–5550.
  * [52] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” in _CVPR_ , 2017, pp. 5018–5027.
  * [53] S. Zhao, B. Li, X. Yue, Y. Gu, P. Xu, R. Hu, H. Chai, and K. Keutzer, “Multi-source domain adaptation for semantic segmentation,” _NeurIPS_ , vol. 32, 2019.
  * [54] D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo _et al._ , “The many faces of robustness: A critical analysis of out-of-distribution generalization,” in _ICCV_ , 2021, pp. 8340–8349.
  * [55] C. Fang, Y. Xu, and D. N. Rockmore, “Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias,” in _ICCV_ , 2013, pp. 1657–1664.
  * [56] P. W. Koh, S. Sagawa, H. Marklund, S. M. Xie, M. Zhang _et al._ , “Wilds: A benchmark of in-the-wild distribution shifts,” in _ICML_. PMLR, 2021, pp. 5637–5664.
  * [57] Y. He, Z. Shen, and P. Cui, “Towards non-iid image classification: A dataset and baselines,” _Pattern Recognition_ , vol. 110, p. 107383, 2021.
  * [58] M. Nǎdejde, A. Currey, B. Hsu, X. Niu, M. Federico, and G. Dinu, “Cocoa-mt: A dataset and benchmark for contrastive controlled mt with application to formality,” in _NAACL_ , 2022, pp. 616–632.
  * [59] Y. Zheng, G. Chen, and M. Huang, “Out-of-domain detection for natural language understanding in dialog systems,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 28, pp. 1198–1209, 2020.
  * [60] A. Coucke, A. Saade, A. Ball, T. Bluche, A. Caulier _et al._ , “Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces,” _arXiv:1805.10190_ , 2018.
  * [61] S. Larson, A. Mahendran, J. J. Peper, C. Clarke, A. Lee _et al._ , “An evaluation dataset for intent classification and out-of-scope prediction,” in _EMNLP-IJCNLP_ , 2019.
  * [62] X. Zang, A. Rastogi, S. Sunkara, R. Gupta, J. Zhang, and J. Chen, “MultiWOZ 2.2 : A dialogue dataset with additional annotation corrections and state tracking baselines,” in _NLP4ConvAI_ , 2020, pp. 109–117.
  * [63] C. Walker, S. Strassel, J. Medero, and K. Maeda, “Ace 2005 multilingual training corpus,” _Linguistic Data Consortium, Philadelphia_ , vol. 57, p. 45, 2006.
  * [64] J. Blitzer, M. Dredze, and F. Pereira, “Biographies, Bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification,” in _ACL_ , 2007, pp. 440–447.
  * [65] A. Mathur, F. Kawsar, N. Berthouze, and N. D. Lane, “Libri-adapt: a new speech dataset for unsupervised domain adaptation,” in _ICASSP_ , 2020, pp. 7439–7443.
  * [66] R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler _et al._ , “Common voice: A massively-multilingual speech corpus,” in _LREC 2020_ , 2020, pp. 4211–4215.
  * [67] C. Karouzos, G. Paraskevopoulos, and A. Potamianos, “UDALM: Unsupervised domain adaptation through language modeling,” in _NAACL_ , 2021, pp. 2579–2590.
  * [68] T. Chu, Y. Liu, J. Deng, W. Li, and L. Duan, “Denoised maximum classifier discrepancy for source-free unsupervised domain adaptation,” in _AAAI_ , vol. 36, no. 1, 2022, pp. 472–480.
  * [69] S. Ao, X. Li, and C. Ling, “Fast generalized distillation for semi-supervised domain adaptation,” in _AAAI_ , vol. 31, no. 1, 2017.
  * [70] C.-M. Chang, B.-H. Su, S.-C. Lin, J.-L. Li, and C.-C. Lee, “A bootstrapped multi-view weighted kernel fusion framework for cross-corpus integration of multimodal emotion recognition,” in _ACII_ , 2017, pp. 377–382.
  * [71] C.-C. Lu, J.-L. Li, and C.-C. Lee, “Learning an arousal-valence speech front-end network using media data in-the-wild for emotion recognition,” in _AVEC_ , 2018, pp. 99–105.
  * [72] R. Elwell and R. Polikar, “Incremental learning of concept drift in nonstationary environments,” _IEEE Transactions on Neural Networks_ , vol. 22, no. 10, pp. 1517–1531, 2011.
  * [73] I. Zliobaite, “How good is the electricity benchmark for evaluating concept drift adaptation,” _arXiv:1301.3524_ , 2013.
  * [74] J. A. Blackard and D. J. Dean, “Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables,” _Computers and electronics in agriculture_ , vol. 24, no. 3, pp. 131–151, 1999.
  * [75] V. Losing, B. Hammer, and H. Wersing, “Interactive online learning for obstacle classification on a mobile robot,” in _IJCNN_ , 2015, pp. 1–8.
  * [76] ——, “Knn classifier with self adjusting memory for heterogeneous concept drift,” in _ICDM_ , 2016, pp. 291–300.
  * [77] I. Žliobaitė, “Combining similarity in time and space for training set formation under concept drift,” _Intelligent Data Analysis_ , vol. 15, no. 4, pp. 589–611, 2011.
  * [78] V. M. Souza, D. M. dos Reis, A. G. Maletzke, and G. E. Batista, “Challenges in benchmarking stream learning algorithms with real-world data,” _Data Mining and Knowledge Discovery_ , vol. 34, pp. 1805–1858, 2020.
  * [79] J. Yang, K. Zhou, Y. Li, and Z. Liu, “Generalized out-of-distribution detection: A survey,” _arXiv:2110.11334_ , 2021.
  * [80] P. Morteza and Y. Li, “Provable guarantees for understanding out-of-distribution detection,” in _AAAI_ , vol. 36, no. 7, 2022, pp. 7831–7840.
  * [81] Y. Sun, C. Guo, and Y. Li, “ReAct: out-of-distribution detection with rectified activations,” in _NeurIPS_ , vol. 34, 2021, pp. 144–157.
  * [82] Y. Sun, Y. Ming, X. Zhu, and Y. Li, “Out-of-distribution detection with deep nearest neighbors,” in _ICML_ , vol. 162. PMLR, 2022, pp. 20 827–20 840.
  * [83] J. Winkens, R. Bunel, A. G. Roy, R. Stanforth, V. Natarajan _et al._ , “Contrastive training for improved out-of-distribution detection,” _arXiv:2007.05566_ , 2020.
  * [84] S. Mohseni, M. Pitale, J. Yadawa, and Z. Wang, “Self-supervised learning for generalizable out-of-distribution detection,” in _AAAI_ , vol. 34, no. 04, 2020, pp. 5216–5223.
  * [85] X. Du, G. Gozum, Y. Ming, and Y. Li, “SIREN: Shaping representations for detecting out-of-distribution objects,” in _NeurIPS_ , 2022.
  * [86] D. Jin, S. Gao, S. Kim, Y. Liu, and D. Hakkani-Tür, “Towards textual out-of-domain detection without in-domain labels,” _IEEE/ACM Transactions on Audio, Speech, and Language Processing_ , vol. 30, pp. 1386–1395, 2022.
  * [87] J. Zhang, J. Yang, P. Wang, H. Wang, Y. Lin _et al._ , “OpenOOD v1.5: Enhanced benchmark for out-of-distribution detection,” in _arXiv preprint arXiv:2306.09301_ , 2023.
  * [88] K. Kirchheim, M. Filax, and F. Ortmeier, “PyTorch-OOD: A library for out-of-distribution detection based on PyTorch,” in _CVPR Workshops_ , 2022, pp. 4351–4360.
  * [89] J. Wang, C. Lan, C. Liu, Y. Ouyang, T. Qin _et al._ , “Generalizing to unseen domains: A survey on domain generalization,” _IEEE Transactions on Knowledge and Data Engineering_ , 2022.
  * [90] F. Qiao, L. Zhao, and X. Peng, “Learning to learn single domain generalization,” in _CVPR_ , 2020.
  * [91] F. Wang, Z. Han, Y. Gong, and Y. Yin, “Exploring domain-invariant parameters for source free domain adaptation,” in _CVPR_ , 2022, pp. 7151–7160.
  * [92] Y. Kim, D. Cho, and S. Hong, “Towards privacy-preserving domain adaptation,” _IEEE Signal Processing Letters_ , vol. 27, pp. 1675–1679, 2020.
  * [93] N. Ding, Y. Xu, Y. Tang, C. Xu, Y. Wang, and D. Tao, “Source-free domain adaptation via distribution estimation,” in _CVPR_ , 2022, pp. 7212–7222.
  * [94] H. Xia, H. Zhao, and Z. Ding, “Adaptive adversarial network for source-free domain adaptation,” in _ICCV_ , 2021, pp. 9010–9019.
  * [95] A. Frikha, H. Chen, D. Krompaß, T. Runkler, and V. Tresp, “Towards data-free domain generalization,” in _NeurIPS Workshop_ , 2021.
  * [96] H. Niu, H. Li, F. Zhao, and B. Li, “Domain-unified prompt representations for source-free domain generalization,” _arXiv:2209.14926_ , 2022.
  * [97] J. Li, M. L. Seltzer, X. Wang, R. Zhao, and Y. Gong, “Large-scale domain adaptation via teacher-student learning,” _Proc. Interspeech_ , pp. 2386–2390, 2017.
  * [98] D. Kim, Y. Yoo, S. Park, J. Kim, and J. Lee, “SelfReg: Self-supervised contrastive regularization for domain generalization,” in _ICCV_ , 2021, pp. 9619–9628.
  * [99] H. Tang and K. Jia, “Discriminative adversarial domain adaptation,” in _AAAI_ , vol. 34, 2020, pp. 5940–5947.
  * [100] J. Yang, J. Liu, N. Xu, and J. Huang, “Tvt: Transferable vision transformer for unsupervised domain adaptation,” in _WACV_ , 2023, pp. 520–530.
  * [101] J. Gideon, M. G. McInnis, and E. M. Provost, “Improving cross-corpus speech emotion recognition with adversarial discriminative domain generalization (ADDoG),” _IEEE Transactions on Affective Computing_ , vol. 12, no. 4, pp. 1055–1068, 2019.
  * [102] A. Rozantsev, M. Salzmann, and P. Fua, “Beyond sharing weights for deep domain adaptation,” _IEEE PAMI_ , vol. 41, no. 4, pp. 801–814, 2018.
  * [103] B. Sun, J. Feng, and K. Saenko, “Return of frustratingly easy domain adaptation,” in _AAAI_ , vol. 30, no. 1, 2016.
  * [104] G. Kang, L. Jiang, Y. Yang, and A. G. Hauptmann, “Contrastive adaptation network for unsupervised domain adaptation,” in _CVPR_ , 2019.
  * [105] J. Shen, Y. Qu, W. Zhang, and Y. Yu, “Wasserstein distance guided representation learning for domain adaptation,” in _AAAI_ , vol. 32, no. 1, 2018.
  * [106] Y. Li, N. Wang, J. Shi, J. Liu, and X. Hou, “Revisiting batch normalization for practical domain adaptation,” _arXiv:1603.04779_ , 2016.
  * [107] Z. Lipton, Y.-X. Wang, and A. Smola, “Detecting and correcting for label shift with black box predictors,” in _ICML_. PMLR, 2018, pp. 3122–3130.
  * [108] K. Azizzadenesheli, A. Liu, F. Yang, and A. Anandkumar, “Regularized learning for domain adaptation under label shifts,” in _ICLR_ , 2019.
  * [109] L. Chen, M. Zaharia, and J. Y. Zou, “Is unsupervised performance estimation impossible when both covariates and labels shift?” in _NeurIPS Workshop_ , 2022.
  * [110] R. Wu, C. Guo, Y. Su, and K. Q. Weinberger, “Online adaptation to label distribution shift,” in _NeurIPS_ , vol. 34, 2021, pp. 11 340–11 351.
  * [111] H. Song, M. Kim, D. Park, Y. Shin, and J.-G. Lee, “Learning from noisy labels with deep neural networks: A survey,” _IEEE TNNLS_ , 2022.
  * [112] A. Kumar, P. K. Sharma, A. Illa, S. Venkatapathy _et al._ , “Learning under label noise for robust spoken language understanding systems,” in _Proc. Interspeech 2022_ , 2022, pp. 3463–3467.
  * [113] Z. Han, X.-J. Gui, C. Cui, and Y. Yin, “Towards accurate and robust domain adaptation under noisy environments,” in _IJCAI_ , 2021, pp. 2269–2276.
  * [114] Q. Yu, A. Hashimoto, and Y. Ushiku, “Divergence optimization for noisy universal domain adaptation,” in _CVPR_ , 2021, pp. 2515–2524.
  * [115] W. Chen, L. Lin, S. Yang, D. Xie, S. Pu, and Y. Zhuang, “Self-supervised noisy label learning for source-free unsupervised domain adaptation,” in _IROS_ , 2022, pp. 10 185–10 192.
  * [116] L. Yi, G. Xu, P. Xu, J. Li, R. Pu, C. Ling, I. McLeod, and B. Wang, “When source-free domain adaptation meets learning with noisy labels,” in _ICLR_ , 2023.
  * [117] A. Parnami and M. Lee, “Learning from few examples: A summary of approaches to few-shot learning,” _arXiv:2203.04291_ , 2022.
  * [118] Z. Chi, L. Gu, H. Liu, Y. Wang, Y. Yu, and J. Tang, “MetaFSCIL: A meta-learning approach for few-shot class incremental learning,” in _CVPR_ , 2022, pp. 14 166–14 175.
  * [119] Y. Song, T. Wang, P. Cai, S. K. Mondal, and J. P. Sahoo, “A comprehensive survey of few-shot learning: Evolution, applications, challenges, and opportunities,” _ACM Comput. Surv._ , 2023.
  * [120] A. Zhao, M. Ding, Z. Lu, T. Xiang, Y. Niu, J. Guan, and J.-R. Wen, “Domain-adaptive few-shot learning,” in _WACV_ , 2021, pp. 1390–1399.
  * [121] X. Yue, Z. Zheng, S. Zhang, Y. Gao, T. Darrell _et al._ , “Prototypical cross-domain self-supervised learning for few-shot unsupervised domain adaptation,” in _CVPR_ , 2021, pp. 13 834–13 844.
  * [122] Z. Hu, Y. Sun, and Y. Yang, “Switch to generalize: Domain-switch learning for cross-domain few-shot classification,” in _ICLR_ , 2022.
  * [123] F. Pourpanah, M. Abdar, Y. Luo, X. Zhou, R. Wang, C. P. Lim, X.-Z. Wang, and Q. J. Wu, “A review of generalized zero-shot learning methods,” _IEEE PAMI_ , 2022.
  * [124] C. Zhou, Q. Li, C. Li, J. Yu, Y. Liu, G. Wang _et al._ , “A comprehensive survey on pretrained foundation models: A history from BERT to ChatGPT,” _arXiv:2302.09419_ , 2023.
  * [125] Y. Du, Z. Liu, J. Li, and W. X. Zhao, “A survey of vision-language pre-trained models,” in _IJCAI_ , 2022, pp. 5436–5443.
  * [126] W. Wang, J. Dai, Z. Chen, Z. Huang, Z. Li _et al._ , “Internimage: Exploring large-scale vision foundation models with deformable convolutions,” in _CVPR_ , 2023, pp. 14 408–14 419.
  * [127] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” _NeurIPS_ , vol. 33, pp. 12 449–12 460, 2020.
  * [128] Y. Tian, X. Chen, and S. Ganguli, “Understanding self-supervised learning dynamics without contrastive pairs,” in _ICML_. PMLR, 2021, pp. 10 268–10 278.
  * [129] Q. Garrido, Y. Chen, A. Bardes, L. Najman, and Y. LeCun, “On the duality between contrastive and non-contrastive self-supervised learning,” in _ICLR_ , 2023.
  * [130] A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, “Data2vec: A general framework for self-supervised learning in speech, vision and language,” in _ICML_. PMLR, 2022, pp. 1298–1312.
  * [131] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh _et al._ , “Learning transferable visual models from natural language supervision,” in _ICML_. PMLR, 2021, pp. 8748–8763.
  * [132] A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu _et al._ , “Xls-r: Self-supervised cross-lingual speech representation learning at scale,” _arXiv_ , vol. abs/2111.09296, 2021.
  * [133] T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ilić _et al._ , “Bloom: A 176b-parameter open-access multilingual language model,” _arXiv:2211.05100_ , 2022.
  * [134] N. Ding, Y. Qin, G. Yang, F. Wei, Z. Yang _et al._ , “Delta tuning: A comprehensive study of parameter efficient methods for pre-trained language models,” _arXiv:2203.06904_ , 2022.
  * [135] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe _et al._ , “Parameter-efficient transfer learning for NLP,” in _ICML_ , vol. 97. PMLR, 2019, pp. 2790–2799.
  * [136] X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in _IJCNLP-AACL_ , 2021, pp. 4582–4597.
  * [137] B. Lester, R. Al-Rfou, and N. Constant, “The power of scale for parameter-efficient prompt tuning,” in _Proceedings of EMNLP_ , 2021, pp. 3045–3059.
  * [138] E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in _ICLR_ , 2022.
  * [139] J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig, “Towards a unified view of parameter-efficient transfer learning,” in _ICLR_ , 2022.
  * [140] Y. Sun, X. Wang, L. Zhuang, J. Miller, M. Hardt, and A. A. Efros, “Test-time training with self-supervision for generalization under distribution shifts,” in _ICML_ , 2020.
  * [141] Z. Chi, Y. Wang, Y. Yu, and J. Tang, “Test-time fast adaptation for dynamic scene deblurring via meta-auxiliary learning,” in _CVPR_ , 2021, pp. 9137–9146.
  * [142] S. Goyal, M. Sun, A. Raghunathan, and J. Z. Kolter, “Test time adaptation via conjugate pseudo-labels,” in _NeurIPS_ , 2022.
  * [143] D. Chen, D. Wang, T. Darrell, and S. Ebrahimi, “Contrastive test-time adaptation,” in _CVPR_ , 2022.
  * [144] Y. Liu, P. Kothari, B. van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi, “Ttt++: When does self-supervised test-time training fail or thrive?” in _NeurIPS_ , vol. 34. Curran Associates, Inc., 2021, pp. 21 808–21 820.
  * [145] Y. Gandelsman, Y. Sun, X. Chen, and A. A. Efros, “Test-time training with masked autoencoders,” in _NeurIPS_ , 2022.
  * [146] Y. Su, X. Xu, and K. Jia, “Revisiting realistic test-time training: Sequential inference and adaptation by anchored clustering,” in _NeurIPS_ , 2022.
  * [147] Y. Iwasawa and Y. Matsuo, “Test-time classifier adjustment module for model-agnostic domain generalization,” _NeurIPS_ , vol. 34, pp. 2427–2440, 2021.
  * [148] S. Niu, J. Wu, Y. Zhang, Y. Chen, S. Zheng, P. Zhao, and M. Tan, “Efficient test-time model adaptation without forgetting,” in _ICML_. PMLR, 2022, pp. 16 888–16 905.
  * [149] Q. Wang, O. Fink, L. Van Gool, and D. Dai, “Continual test-time domain adaptation,” in _CVPR_ , 2022, pp. 7201–7211.
  * [150] A. Søgaard, S. Ebert, J. Bastings, and K. Filippova, “We need to talk about random splits,” in _EACL_ , 2021, pp. 1823–1832.
  * [151] J. Gama, P. Medas, G. Castillo, and P. Rodrigues, “Learning with drift detection,” in _Advances in Artificial Intelligence–SBIA_. Springer, 2004, pp. 286–295.
  * [152] A. Bifet and R. Gavalda, “Learning from time-changing data with adaptive windowing,” in _SDM_. SIAM, 2007, pp. 443–448.
  * [153] A. Liu, J. Lu, Y. Song, J. Xuan, and G. Zhang, “Concept drift detection delay index,” _IEEE Transactions on Knowledge and Data Engineering_ , pp. 1–1, 2022.
  * [154] S. Yu, X. Wang, and J. C. Príncipe, “Request-and-reverify: hierarchical hypothesis testing for concept drift detection with expensive labels,” in _IJCAI_ , 2018, pp. 3033–3039.
  * [155] P. Sidhu and M. Bhatia, “An online ensembles approach for handling concept drift in data streams: diversified online ensembles detection,” _International Journal of Machine Learning and Cybernetics_ , vol. 6, no. 6, pp. 883–909, 2015.
  * [156] I. Khamassi, M. Sayed-Mouchaweh, M. Hammami, and K. Ghédira, “A new combination of diversity techniques in ensemble classifiers for handling complex concept drift,” _Learning from Data Streams in Evolving Environments: Methods and Applications_ , pp. 39–61, 2019.
  * [157] R. Anderson, Y. S. Koh, G. Dobbie, and A. Bifet, “Recurring concept meta-learning for evolving data streams,” _Expert Systems with Applications_ , vol. 138, p. 112832, 2019.
  * [158] H. Yu, Q. Zhang, T. Liu, J. Lu, Y. Wen, and G. Zhang, “Meta-add: A meta-learning based pre-trained model for concept drift active detection,” _Information Sciences_ , vol. 608, pp. 996–1009, 2022.
  * [159] A. Zubaroğlu and V. Atalay, “Data stream clustering: a review,” _Artificial Intelligence Review_ , vol. 54, no. 2, pp. 1201–1236, 2021.
  * [160] G. Oliveira, L. L. Minku, and A. L. Oliveira, “Tackling virtual and real concept drifts: An adaptive gaussian mixture model approach,” _IEEE Transactions on Knowledge and Data Engineering_ , 2021.
  * [161] A. Liu, J. Lu, F. Liu, and G. Zhang, “Accumulating regional density dissimilarity for concept drift detection in data streams,” _Pattern Recognition_ , vol. 76, pp. 256–272, 2018.
  * [162] A. Castellani, S. Schmitt, and B. Hammer, “Task-sensitive concept drift detector with constraint embedding,” in _SSCI_. IEEE, 2021, pp. 01–08.
  * [163] W. Li, X. Yang, W. Liu, Y. Xia, and J. Bian, “DDG-DA: Data distribution generation for predictable concept drift adaptation,” in _AAAI_ , vol. 36, no. 4, 2022, pp. 4092–4100.
  * [164] C. Raab, M. Heusinger, and F.-M. Schleif, “Reactive soft prototype computing for concept drift streams,” _Neurocomputing_ , vol. 416, pp. 340–351, 2020\.
  * [165] P. Li, Y. Liu, Y. Hu, Y. Zhang, X. Hu, and K. Yu, “A drift-sensitive distributed lstm method for short text stream classification,” _IEEE Transactions on Big Data_ , vol. 9, no. 1, pp. 341–357, 2023.
  * [166] A. Liu, J. Lu, and G. Zhang, “Diverse instance-weighting ensemble based on region drift disagreement for concept drift adaptation,” _IEEE Trans. NNLS_ , vol. 32, no. 1, pp. 293–307, 2021.
  * [167] H. Ghomeshi, M. M. Gaber, and Y. Kovalchuk, “Eacd: evolutionary adaptation to concept drifts in data streams,” _Data Mining and Knowledge Discovery_ , vol. 33, pp. 663–694, 2019.
  * [168] M. Heusinger, C. Raab, and F.-M. Schleif, “Passive concept drift handling via variations of learning vector quantization,” _Neural Computing and Applications_ , vol. 34, no. 1, pp. 89–100, 2022.
  * [169] W. Zheng, P. Zhao, G. Chen, H. Zhou, and Y. Tian, “A hybrid spiking neurons embedded lstm network for multivariate time series learning under concept-drift environment,” _IEEE Transactions on Knowledge and Data Engineering_ , pp. 1–1, 2022.
  * [170] I. Chalkidis and A. Søgaard, “Improved multi-label classification under temporal concept drift: Rethinking group-robust algorithms in a label-wise setting,” in _ACL_ , 2022, pp. 2441–2454.
  * [171] Z. Li, W. Huang, Y. Xiong, S. Ren, and T. Zhu, “Incremental learning imbalanced data streams with concept drift: The dynamic updated ensemble algorithm,” _Knowledge-Based Systems_ , vol. 195, p. 105694, 2020.
  * [172] H. J. Jung and M. Lease, “Modeling temporal crowd work quality with limited supervision,” in _AAAI_ , vol. 3, 2015, pp. 83–91.
  * [173] I. Abraham, O. Alonso, V. Kandylas, R. Patel, S. Shelford, and A. Slivkins, “How many workers to ask? adaptive exploration for collecting high quality labels,” in _SIGIR_. Association for Computing Machinery, 2016, p. 473–482.
  * [174] T. Goyal, T. McDonnell, M. Kutlu, T. Elsayed, and M. Lease, “Your behavior signals your reliability: Modeling crowd behavioral traces to ensure quality relevance annotations,” _AAAI_ , vol. 6, no. 1, pp. 41–49, 2018.
  * [175] F. Zafari, I. Moser, and T. Baarslag, “Modelling and analysis of temporal preference drifts using a component-based factorised latent approach,” _Expert Systems with Applications_ , vol. 116, pp. 186–208, 2019.
  * [176] Y. Xie, J. Li, T. Zhu, and C. Liu, “Continuous-valued annotations aggregation for heart rate detection,” _IEEE Access_ , vol. 7, pp. 37 664–37 671, 2019\.
  * [177] S. Liu, S. Xue, J. Wu, C. Zhou, J. Yang, Z. Li, and J. Cao, “Online active learning for drifting data streams,” _IEEE Trans. NNLS_ , vol. 34, no. 1, pp. 186–200, 2023.
  * [178] X. Zhan, Q. Wang, K.-h. Huang, H. Xiong, D. Dou, and A. B. Chan, “A comparative survey of deep active learning,” _arXiv:2203.13450_ , 2022.
  * [179] Y. Yu, R. Zhang, R. Xu, J. Zhang, J. Shen, and C. Zhang, “Cold-start data selection for few-shot language model fine-tuning: A prompt-based uncertainty propagation approach,” _arXiv:2209.06995_ , 2022.
  * [180] G. Hacohen, A. Dekel, and D. Weinshall, “Active learning on a budget: Opposite strategies suit high and low budgets,” in _ICML_. PMLR, 2022, pp. 8175–8195.
  * [181] Ł. Korycki and B. Krawczyk, “Streaming decision trees for lifelong learning,” in _ECML PKDD_. Springer, 2021, pp. 502–518.
  * [182] M. Pratama, C. Za’in, A. Ashfahani, Y. S. Ong, and W. Ding, “Automatic construction of multi-layer perceptron network from streaming examples,” in _CIKM_ , 2019, pp. 1171–1180.
  * [183] D. Liu, Y. Wu, and H. Jiang, “Fp-elm: An online sequential learning algorithm for dealing with concept drift,” _Neurocomputing_ , vol. 207, pp. 322–334, 2016.
  * [184] G. M. van de Ven, T. Tuytelaars, and A. S. Tolias, “Three types of incremental learning,” _Nature Machine Intelligence_ , pp. 1–13, 2022.
  * [185] L. Korycki and B. Krawczyk, “Class-incremental experience replay for continual learning under concept drift,” in _CVPR Workshops_ , June 2021, pp. 3649–3658.
  * [186] A. A. Darem, F. A. Ghaleb, A. A. Al-Hashmi, J. H. Abawajy, S. M. Alanazi, and A. Y. Al-Rezami, “An adaptive behavioral-based incremental batch learning malware variants detection model using concept drift detection and sequential deep learning,” _IEEE Access_ , vol. 9, pp. 97 180–97 196, 2021.
  * [187] A. M. N. Taufique, C. S. Jahan, and A. Savakis, “Unsupervised continual learning for gradually varying domains,” in _CVPR Workshops_ , 2022, pp. 3740–3750.
  * [188] J. Li, M. Jing, H. Su, K. Lu, L. Zhu, and H. T. Shen, “Faster domain adaptation networks,” _IEEE Transactions on Knowledge and Data Engineering_ , vol. 34, no. 12, pp. 5770–5783, 2022.
  * [189] B. Jiao, Y. Guo, D. Gong, and Q. Chen, “Dynamic ensemble selection for imbalanced data streams with concept drift,” _IEEE Trans. NNLS_ , pp. 1–14, 2022.
  * [190] B. Jiao, Y. Guo, C. Yang, J. Pu, Z. Zheng, and D. Gong, “Incremental weighted ensemble for data streams with concept drift,” _IEEE Transactions on Artificial Intelligence_ , pp. 1–12, 2022.
  * [191] C.-C. J. Kuo and A. M. Madni, “Green learning: Introduction, examples and outlook,” _Journal of Visual Communication and Image Representation_ , p. 103685, 2022.
  * [192] X. Huang, D. Kroening, W. Ruan, J. Sharp _et al._ , “A survey of safety and trustworthiness of deep neural networks: Verification, testing, adversarial attack and defence, and interpretability,” _Computer Science Review_ , vol. 37, p. 100270, 2020.
  * [193] T. Bai, J. Luo, J. Zhao, B. Wen, and Q. Wang, “Recent advances in adversarial training for adversarial robustness,” in _IJCAI_ , 2021, pp. 4312–4321.
  * [194] J. Wang, H. Xixu, W. Hou, H. Chen, R. Zheng, Y. Wang _et al._ , “On the robustness of chatgpt: An adversarial and out-of-distribution perspective,” in _ICLR Workshop_ , 2023.
  * [195] K. Jin, X. Cheng, J. Yang, and K. Shen, “Differentially private correlation alignment for domain adaptation.” in _IJCAI_ , vol. 21, 2021, pp. 3649–3655.
  * [196] D. Peterson, P. Kanani, and V. J. Marathe, “Private federated learning with domain adaptation,” _arXiv:1912.06733_ , 2019.
  * [197] D. M. Manias, I. Shaer, L. Yang, and A. Shami, “Concept drift detection in federated networked systems,” in _GLOBECOM_ , 2021, pp. 1–6.
  * [198] F. E. Casado, D. Lema, M. F. Criado, R. Iglesias, C. V. Regueiro, and S. Barro, “Concept drift detection and adaptation for federated and continual learning,” _Multimedia Tools and Applications_ , pp. 1–23, 2022.
  * [199] D. Mukherjee, F. Petersen, M. Yurochkin, and Y. Sun, “Domain adaptation meets individual fairness. and they get along.” in _NeurIPS_ , 2022.
  * [200] H. Wang, J. Hong, J. Zhou, and Z. Wang, “How robust is your fairness? evaluating and sustaining fairness under unseen distribution shifts,” _arXiv:2207.01168_ , 2022.
  * [201] D. Madras, E. Creager, T. Pitassi, and R. Zemel, “Learning adversarially fair and transferable representations,” in _ICML_. PMLR, 2018, pp. 3384–3393.
  * [202] J. Zenisek, N. Wild, and J. Wolfartsberger, “Investigating the potential of smart manufacturing technologies,” _Procedia Computer Science_ , vol. 180, pp. 507–516, 2021.
  * [203] J. a. Gama, I. Žliobaitundefined, A. Bifet, M. Pechenizkiy, and A. Bouchachia, “A survey on concept drift adaptation,” _ACM Comput. Surv._ , vol. 46, no. 4, 2014.
  * [204] C.-C. Lin, D.-J. Deng, C.-H. Kuo, and L. Chen, “Concept drift detection and adaption in big imbalance industrial iot data using an ensemble learning method of offline classifiers,” _IEEE Access_ , vol. 7, pp. 56 198–56 207, 2019.
  * [205] J. J. Thiagarajan, D. Rajan, and P. Sattigeri, “Understanding behavior of clinical models under domain shifts,” in _ACM SIGKDD_ , 2019.
  * [206] L. Ju, X. Wang, L. Wang, D. Mahapatra, X. Zhao, Q. Zhou, T. Liu, and Z. Ge, “Improving medical images classification with label noise using dual-uncertainty estimation,” _IEEE Transactions on Medical Imaging_ , vol. 41, no. 6, pp. 1533–1546, 2022.

|  Jeng-Lin Li is currently a Senior Data Scientist at Inventec Corporation
and an Adjunct Assistant Professor in the Biomed AI PhD program at National
Tsing Hua University (NTHU). He received his B.S. (2016) and Ph.D. (2022)
degrees from the Department of Electrical Engineering at NTHU. He was awarded
ACLCLP dissertation award (2022), NTHU Principal Outstanding Student
Scholarship (2017-2020), industrial scholarships from Garmin (2018), Yahoo
(2019), Novatek (2020, 2021), and travel grants from EMBC (2019), Interspeech
(2019). His research interests are machine learning, health analytics,
behavior signal processing, and affective computing.
---|---
|  Chih-Fan Hsu Chih-Fan Hsu is a Senior Data Scientist at Inventec
Corporation. His research interests include computer vision, machine learning,
virtual reality, and multimedia systems. Dr. Hsu was a Postdoc at the
University of California, Davis, National Tsing Hua University, and National
Yang Ming Chiao Tung University (2020-2021), and a research assistant at the