# Adaptive Domain Generalization via Online Disagreement Minimization

Xin Zhang and Ying-Cong Chen Corresponding author: Ying-Cong Chen.Xin Zhang is
with the AI Thrust, Information Hub, Hong Kong University of Science and
Technology (Guangzhou), Guangzhou, China (e-mail: devinxzhang@ust.hk).Ying-
Cong Chen is with the AI Thrust, Information Hub, Hong Kong University of
Science and Technology (Guangzhou), Guangzhou, China, also with the Department
of Computer Science and Engineering, Hong Kong University of Science and
Technology, and also with the HKUST (Guangzhou) - SmartMore Joint Lab,
Guangzhou, China (e-mail: yingcongchen@ust.hk).This work is supported by
National Natural Science Foundation of China (No. 62206068).

###### Abstract

Deep neural networks suffer from significant performance deterioration when
there exists distribution shift between deployment and training. Domain
Generalization (DG) aims to safely transfer a model to unseen target domains
by only relying on a set of source domains. Although various DG approaches
have been proposed, a recent study named DomainBed [1], reveals that most of
them do not beat simple empirical risk minimization (ERM). To this end, we
propose a general framework that is orthogonal to existing DG algorithms and
could improve their performance consistently. Unlike previous DG works that
stake on a static source model to be hopefully a universal one, our proposed
AdaODM adaptively modifies the source model at test time for different target
domains. Specifically, we create multiple domain-specific classifiers upon a
shared domain-generic feature extractor. The feature extractor and classifiers
are trained in an adversarial way, where the feature extractor embeds the
input samples into a domain-invariant space, and the multiple classifiers
capture the distinct decision boundaries that each of them relates to a
specific source domain. During testing, distribution differences between
target and source domains could be effectively measured by leveraging
prediction disagreement among source classifiers. By fine-tuning source models
to minimize the disagreement at test time, target-domain features are well
aligned to the invariant feature space. We verify AdaODM on two popular DG
methods, namely ERM and CORAL, and four DG benchmarks, namely VLCS, PACS,
OfficeHome, and TerraIncognita. The results show AdaODM stably improves the
generalization capacity on unseen domains and achieves state-of-the-art
performance.

###### Index Terms:

Domain shift, domain generalization, online adaptation, consistency
regularization.

##  I Introduction

Deep neural networks work best when training and testing data are sampled from
the same distribution [2]. However, this is not always achievable in most
real-world scenarios. For instance, medical images acquired by different
scanners, vendors, and operators may show visible differences. Autonomous
driving system must handle variations in light and weather conditions.

Unfortunately, the distribution shift is notorious to harm the model’s
generalization ability on unseen data. A general solution to tackle this
problem is Domain Adaptation (DA) [3, 4]. DA assumes (unlabeled) samples of
the target domain are available during training and alleviates domain shift by
minimizing the discrepancy between source and target domain. Despite its
promising performance, domain adaptation is not applicable when target domain
is inaccessible during training. Besides, it also requires retraining the
models when target domain changes (e.g., the system is moved to a new
environment), which may cause large training costs. Domain Generalization (DG)
[5, 6, 7] relieves the constraint. It aims to learn a model that is
generalizable to unseen target domains solely on multiple source domains,
without access to any samples from target domain. Despite the ambitious goal,
a recent study on large-scale benchmarks [1] shows that most existing DG
methods actually do not bring significant benefits compared with the naive
Empirical Risk Minimization (ERM). This might imply that it is difficult to
handle various unknown domain samples merely with a generalized source model
trained via the DG protocol.

Figure 1: Comparison of UDA, DG, TTA, and Adaptive DG settings. UDA assumes
access to unlabeled target data during training. DG learns from multiple
source domains to improve models’ generalization ability. TTA adapts a source
model to target domains via test-time online adaptation. Adaptive DG takes
advantages of both DG and TTA to better leverage domain information for
adaptively adjusting source models for new domains.

Therefore, we are particularly interested in how to facilitate DG to further
utilize target domain information. Recently, the Test-Time Adaptation (TTA)
paradigm has become popular in resolving the domain shift problem which
updates the source model to mitigate the domain gap at test time. TTA is based
on the fact that during testing, the model can always access query samples of
the target domain which provides essential target domain information. We
introduce the idea of TTA to the DG field to adaptively adjust the DG model
based on the target domain samples during testing (See Fig. 1 for comparison).

Existing TTA works include updating the batch-norm statistics based on testing
data [8, 9], designing unsupervised loss functions (e.g., entropy
minimization) to update the feature extractor [10, 11], or adjusting the
classifier based on testing samples [12]. Although effective, these methods do
not take domain differences into consideration. Domain differences provide
valuable information to find the invariance shared by all domains. Without
leveraging this information, the model may fail in handling target domains
with large domain shift [13, 5]. Moreover, losses like entropy minimization
can only enforce target features away from decision boundaries whose
optimization direction might be less reliable (see Fig. 2(b)). Our adaptive
domain generalization aims to better leverage both source and target domains
information to measure the distribution differences, and models are optimized
towards aligning target domain features to be invariant to domain changes.

Motivated by the above considerations, we propose a novel test-time domain
generalization framework named AdaODM which Adaptively adjusts the source DG
model based on the target domain data via Online Disagreement Minimization.
Our idea is to introduce a set of domain-aware classifiers, each of which is
trained with data from one single domain. As such, these classifiers may
leverage both domain-generic and domain-specific features. During testing,
these classifiers may generate different prediction scores because of the
interference of domain-specific features. Therefore, by minimizing the
prediction disagreement, we can reduce the influence of fragile features that
are specific for each domain, leading to better performance of out-of-domain
samples. Note that such optimization is model-agnostic and can be performed
during testing. This allows us to facilitate existing DG methods to adapt
source models for different target domains.

Figure 2: Comparison of ERM, Entropy Minimization (EM)-based Test-Time
Adaptation (TTA) method, and the proposed AdaODM. Here we take for instance a
two-class classification task with three source domains. Solid and dashed
circles represent two classes. Colors represent various domains. (a) shows the
vanilla ERM which performs well on three training sources while predicting
wrongly on some target samples (dashed points above the line and solid points
below the line). In (b), the EM-based TTA aims to make more confident
predictions on target domains by pushing target features away from the
decision boundary. This could reduce domain gaps to some extent. However,
those falsely predicted samples are pushed in wrong directions, which harms
adaptation performance. AdaODM constructs domain-specific classifiers as (d).
During training, AdaODM promotes each classifier’s generalization capacity by
encouraging prediction consistency on unseen domains (“meta-target” domains in
(c)). This is done through adversarial training where the feature extractor
minimizes the prediction disagreement while classifiers maximize it.
Consequently, we achieve domain-invariant feature space encompassed by class
boundaries. At test time, classifiers are frozen. We optimize the feature
extractor to minimize the prediction disagreement on target samples, which
aligns target features to domain-invariant regions.

Although the idea looks neat, training these classifiers is challenging. This
is because each classifier is trained only on a subset of total source
domains, and thus they are more likely to exhibit higher variance. As such,
simply minimizing the prediction disagreement may not always lead to accurate
predictions, as it can cause the classifiers to agree on incorrect classes. To
solve this problem, we propose a novel adversarial training scheme to improve
each classifier’s generalization ability on other domains during training.
Specifically, we optimize the feature extractor to minimize the prediction
disagreement among source classifiers, and optimize the classifiers to
maximize it. Consequently, the feature extractor becomes more invariant to
domain changes, while the classifiers are more sensitive to distribution
differences among different domains. Such a process can also be seen as
achieving domain-invariant feature space encompassed by domain-specific class
decision boundaries (see Fig. 2(d)). During inference, source classifiers are
frozen and the feature extractor is adjusted to reduce the prediction
disagreement among classifiers. The main contributions of this paper can be
summarized as follows.

  * •

We propose a novel adaptive domain generalization framework, which could
dynamically adjust source DG models to target domains at test time via online
prediction disagreement minimization.

  * •

We capture domain information with domain-specific classifiers and present a
novel adversarial training scheme to learn domain-invariant features.

  * •

We evaluate our framework on two DG methods and four DG benchmarks and achieve
state-of-the-art performance.

##  II Related Work

###  II-A Domain Adaptation and Generalization

Unsupervised Domain Adaptation (UDA) [3, 4] jointly optimizes the model on
labeled source domain images and unlabeled target domain images to mitigate
the domain gap. Feature alignment-based methods focus on matching the
distributions of feature activations between source and target data. This is
done typically in two manners: (1) moment matching schemes [14, 15, 16] aim to
explicitly minimize some specific statistics distances between the two
domains; (2) adversarial learning-based approaches [17, 18, 19] leverage a
domain discriminator to encourage the domain confusion. In particular, MCD
[18] utilizes classifier discrepancy as a measure to optimize both the feature
extractor and the classifier via adversarial training. Other works also
explore the distribution alignment in data space [20]. UDA has to access
unlabeled target domain data and retrain the model for new target domains,
which makes it impractical for real-world applications. This is improved by
recent “source-free” DA methods [10, 21, 22] which adapt the source model
solely utilizing unlabeled target data. However, these methods generally
require access to all target data during adaptation and optimize source models
for multiple epochs, making them impractical for online adaptation. Instead,
AdaODM aims to bridge the domain gap while performing inference, in which case
the test samples pass through the model only once without being revisited.

The goal of domain generalization is to learn a generalized model on multiple
source domains without relying on any target domain information. Like UDA, a
common pervasive theme in domain generalization aims to capture not only the
task-discriminative but the domain-invariant representations [23, 24, 25, 26].
The underlying principle is that the mapping relationship from input to output
is consistent and invariant to environment changes, DG solutions aim to
estimate various types of invariance from data [1]. Meta-learning-based DG
methods [27, 28, 29] simulate domain shift by splitting the source domains
into meta-train and meta-test and learn transferable weight representations
via episodic training. Invariant risks minimization [30] regularizes ERM to
explore invariant predictors across all domains. Other approaches consider
various domain augmentation strategies [31, 32, 33, 34, 35, 36] to facilitate
the inter-domain constraint. Although a large number of studies have been
developed to address the difficult domain generalization issue, a
comprehensive benchmark comparison conducted by Gulrajani and Lopez-Paz [1]
shows that ERM is still a strong baseline method compared with most state-of-
the-art approaches.

###  II-B Test-Time Training/Adaptation

Recently, adapting models solely on target data at test time has become
appealing since test-time adaptation can leverage target data to adapt the
source model online [11, 10, 37, 38]. The distribution difference is estimated
and adapted while a model makes the inference. Batch normalization statistics
have shown to be essential to model robustness [9]. Some studies propose to
replace the batch-norm statistics estimated on the training set with those on
the test samples [9, 8]. Tent [11] further adapts affine parameters of batch-
norm layers by minimizing the prediction entropy. Self-supervision is also
widely used for test-time feature alignment [37, 10, 39]. Test-time training
[37, 39] jointly trains an auxiliary self-supervised task along with the main
task during training, and optimizes the model with only the self-supervised
task at test time. SHOT [10] updates the feature extractor by both information
maximization and self-supervised pseudo-labeling [40]. MEMO [41] extends Tent
to enhance TTA’s reliability by estimating prediction entropy over multiple
augmented views of test examples. However, these methods do not consider the
domain information among source domains for test-time adaptation.

###  II-C Consistency Regularization

Consistency regularization is widely used for semi-supervised [42, 43] and
unsupervised learning [44, 45]. Consistency regularization for semi-supervised
learning regularizes model predictions to be invariant to semantics-preserving
changes to images, such as rotation, crop, Gaussian noise, or adversarial
attacking, so as to improve classification of unlabeled images. Mean-teacher
[46] designs a consistency cost between predictions of the student and the
teacher models, resulting in a better learning of the student model. Studies
also exploit consistency regularizers to stabilize training of Generative
Adversarial Networks (GANs) [44] and solve the mismatch of latent
representations learned by Variational Auto-Encoders (VAEs) [45]. Recently,
consistency regularization is also utilized in UDA or DG for feature alignment
[47, 48, 49, 18] or style transfer [20]. Inspired by these works, we design a
disagreement score among predictions of source classifiers to further minimize
distribution discrepancy at test time.

##  III Method

Our method allows for adjusting the model parameters to adaptively alleviate
the domain gaps during testing time. Specifically, our model consists of a
domain-generic feature extractor and multiple domain-specific classifiers.
During training, the feature extractor is trained to minimize the prediction
disagreement, while the classifiers are trained to maximize it. This draws
source domains closer and encourages a domain-invariant feature space revealed
by the common nature of distinct classifiers. At test time, the classifiers
are frozen and we optimize the feature extractor to reduce the prediction
disagreement. This aligns query samples from target domains to the invariant
feature space. We illustrate the proposed approach in Fig. 2. In the
following, we first give the basic formulation of domain generalization, then
detail the prediction disagreement-based adversarial training. Finally, we
will introduce how to adapt the trained source model to target domains via
online disagreement minimization. It is worth noting that our algorithm is a
general framework which could be implemented on most existing DG methods. We
show case on vanilla Empirical Risk Minimization (ERM).

###  III-A Problem Setup

Without loss of generality, we focus on the image classification task which
maps inputs $x\in\mathcal{X}$ to the corresponding output label
$y\in\mathcal{Y}$ with model $f$, $f:\mathcal{X}\xrightarrow{}\mathcal{Y}$.
The model $f$ can be further decomposed as a feature extractor $g$ followed by
a classifier $h$, $f=h\circ g$. Consider the problem of domain generalization,
we assume the accessibility to training datasets
$D^{d}=\\{{(x^{d}_{i},y^{d}_{i})}\\}^{n_{d}}_{i=1}$ collected from multiple
source domains $d\in\\{1,...,d_{tr}\\}$, where $d$, $n_{d}$ and $d_{tr}$
denote domain index, number of samples in domain $d$, and number of domains,
respectively. Within each dataset $D^{d}$, samples are identically and
independently distributed (i.i.d.) from the same joint probability density
function $P(X^{d},Y^{d})$, where $X^{d}\in\mathcal{X}$ and
$Y^{d}\in\mathcal{Y}$. Domain generalization aims to learn a generalized model
$f$ on all source domain datasets $D^{d}$ to perform well on a new target
domain $D^{d_{te}}$ where $P(X^{d_{te}},Y^{d_{te}})\neq P(X^{d},Y^{d})$ for
all $d\in\\{1,...,d_{tr}\\}$. Note that no prior information of target
distribution $P(X^{d_{te}},Y^{d_{te}})$ is available during training, previous
studies generally try to learn a generalized feature extractor $g$ to map
samples from all domains into a domain-invariant feature space
$\hat{\mathcal{S}}$, $g:x^{d}\xrightarrow{}\hat{S}$ for all
$d\in\\{1,...,d_{tr},d_{te}\\}$. The adaptive domain generalization further
adapts the source model $f^{tr}$ into a target one $f^{te}$ with bunches of
non-revisitable target samples, during which source data are not accessible.

###  III-B Disagreement Score

The goal of domain generalization is to achieve consistent mapping from the
input $\mathcal{X}$ to the output label $\mathcal{Y}$ regardless of the
distribution shift. An essential route is to learn a shared feature space
$\hat{\mathcal{S}}$ for data from all domains. We consider the role of class
boundaries, which are neglected by most existing DG methods, to capture source
domain characteristics and achieve the invariance of feature space
$\hat{\mathcal{S}}$ for both source and target domains. Specifically, we
propose to learn a set of classifiers $\\{h^{1},...,h^{d_{tr}}\\}$ specific to
each source domain $\\{1,...,d_{tr}\\}$ on top of a shared feature extractor,
respectively. Suppose each domain-specific classifier $h^{d}$ could predict
in-domain samples $x^{d}$ (we omit class notion for clarity) into correct
categories well. With information from different domains, these domain-
specific classifiers will be semantically related but show various decision
boundaries. Intuitively, when an incoming target sample $x^{d_{te}}$ is
classified consistently by all source classifiers, this indicates that the
features of the sample are likely to be located in the shared region of
$\hat{\mathcal{S}}$. If the sample is far from the support of
$\hat{\mathcal{S}}$, classifiers are likely to make inconsistent predictions
because $x$ can be out of some classifiers’ decision boundaries. As shown in
the left of Fig. 2(d), each domain-specific classifier does not perfectly
classify samples in the other two source domains.

Thus, we define a Disagreement Score ($\mathcal{DS}$) as the distance of
prediction probabilities of any two source classifiers on target samples.
Theoretically, there are many choices for computing the distance. In section
IV-G, we show that $L_{1}$-norm works better than $L_{2}$-norm and $KL$
Divergence. We calculate $\mathcal{DS}$ as the sum of the $L_{1}$-norm of the
difference of softmax outputs of any two source classifiers on predicting
target samples:

$\mathcal{DS}=\sum_{\begin{subarray}{c}j,k\in\\{1,...,d_{tr}\\}\\\ j\neq
k\end{subarray}}\frac{1}{n_{te}}\sum_{i=1}^{n_{te}}||s(h^{j}(g(x^{te}_{i})))-s(h^{k}(g(x^{te}_{i})))||_{1}.$
(1)

Here, $h^{j},h^{k}\in\\{h^{1},...,h^{d_{tr}}\\},j\neq k$, $x^{te}_{i}$ is the
target domain sample and $s$ is the softmax function. A larger $\mathcal{DS}$
generally means a more dramatic distribution shift. By minimizing
$\mathcal{DS}$ we hope to narrow the domain gap and achieve better
generalization performance. In the following, we will describe how to leverage
$\mathcal{DS}$ for better feature alignment at training and testing stages.

###  III-C Adversarial Training

Given source data from $d_{tr}$ domains, we learn $d_{tr}$ classifiers
$\\{h^{1},...,h^{d_{tr}}\\}$ on a shared feature extractor $g$. Each domain-
specific classifier $h^{d}$ is first empowered to be discriminative to the
corresponding domain $d$ by the simple Empirical Risk Minimization (ERM):

$\mathcal{ERM}=\frac{1}{n_{d}}\sum_{i=1}^{n_{d}}\ell(h^{d}(g(x_{i}^{d})),y_{i}^{d}),$
(2)

where $\ell$ is the cross-entropy loss.

With the above notions, a straightforward idea to adapt target domains would
be directly applying the trained source classifiers to minimize $\mathcal{DS}$
at test time. However, it is generally hard to achieve desired generalization
performance, especially when large distribution differences exist among source
domains. That is because the features of each source domain show large
variations in the feature space. With unaligned feature distribution, the
generalization capability of a specific classifier $h^{d}$ to unseen domain
$\\{1,...,d_{tr}\\}\backslash\\{d\\}$ is not tightly guaranteed. Besides,
optimizing classifiers by directly minimizing $\mathcal{DS}$ at training time
may lead classifiers to lose domain-specific characteristics and become the
same, making minimizing $\mathcal{DS}$ at test time less effective.

Therefore, we propose an adversarial training strategy which encourages the
feature extractor to generate domain-invariant features, and meanwhile, learns
distinct domain-specific classifiers. Basically, a specific source domain $d$
can be considered as a target domain for classifiers of other source domains
$\\{h^{1},...,h^{d_{tr}}\\}\backslash\\{h^{d}\\}$. Following the terminology
of meta-learning [50, 29], domain $d$ is called the meta-source domain for
$h^{d}$, and the meta-target domain for the remaining classifiers. Likewise,
we define $h^{d}$ as the meta-source classifier for domain $d$ and the meta-
target classifier for the other domains (as shown in Fig. 2(c)).

Then for a meta-source sample, we minimize ERM loss on both the feature
extractor and the meta-source classifier. $\mathcal{DS}$ is calculated between
any two meta-target classifiers. The feature extractor is updated to minimize
$\mathcal{DS}$, resulting in the generated features close to the shared
feature region, while meta-target classifiers are optimized to maximize
$\mathcal{DS}$, avoiding meta-target classifiers become too similar. Ideally,
after the model converges, we will have the generated features located in the
feature regions shared by all involved source domains, and distinct domain-
specific classifiers each adapted well to meta-target domains. Compared with
the naive ERM training which only learns a common classifier on all source
domains, multiple domain-specific classifiers will learn various decision
boundaries. They together describe a more concrete and invariant distribution
for each label class.

The adversarial training of the feature extractor and classifiers is performed
using the gradient reversal layer (GRL) [6] which reverses the gradient
direction to optimize classifiers. As such, we could train all objectives end-
to-end. $\mathcal{DS}$ at training time will be rephrased as:

$\mathcal{DS}_{tr}=\sum_{\begin{subarray}{c}d,j,k\in\\{1,...,d_{tr}\\}\\\
d\neq j\neq
k\end{subarray}}\frac{1}{n_{d}}\sum_{i=1}^{n_{d}}||s(h^{j}(g(x^{d}_{i})))-s(h^{k}(g(x^{d}_{i})))||_{1}.$
(3)

Note that we exclude the meta-source classifier when computing $\mathcal{DS}$,
as it tends to produce overconfident predictions on meta-source data. This
would bias the direction to the invariant feature space and make the training
hard to optimize. We also add an entropy loss ($\mathcal{ET}$) for
regularization which is calculated as:

$\mathcal{ET}=-\frac{1}{C}\sum_{c=1}^{C}log(\frac{1}{n_{d}}\sum_{i=1}^{n_{d}}s(h^{d}(g(x^{d}_{i})))),$
(4)

where $C$ is the number of classes.

The overall losses are as follows:

$\mathop{\arg\min}_{\Phi_{g,\\{h^{1},...,h^{d_{tr}}\\}}}\sum_{d=1}^{d_{tr}}(\mathcal{ERM}+\gamma\mathcal{ET})+\lambda\mathop{\arg\min_{\Phi_{g}}\max_{\Phi_{\\{h^{1},...,h^{d_{tr}}\\}}}}\mathcal{DS}_{tr},$
(5)

where $\gamma$ and $\lambda$ are the weights for the entropy and the
prediction disagreement-based regularization,
$\Phi_{g,\\{h^{1},...,h^{d_{tr}}\\}}$ are parameters of the feature extractor
and classifiers. $\gamma$ is set to 0.01 for all experiments. See Algorithm 1
for details.

Algorithm 1 AdaODM Training Algorithm

0: num of source domains $d_{tr}$, training steps $N$, batch size $B$,
gradient ratio $\eta$, weight for the entropy regularization $\gamma$, weight
for the disagreement score $\lambda$, the feature extractor $g$, domain-
specific classifiers $\\{h^{d}\\}_{d=1}^{d_{tr}}$, training sets
$D^{d}=\\{(x^{d},y^{d})\\}_{d=1}^{d_{tr}},d\in\\{1,2,...,d_{tr}\\}$

1: for $step=1,2,...,N$ do

2: Draw random data batch $\\{(x_{i}^{d},y_{i}^{d})\\}_{i=1}^{B}$ from $D^{d}$

3: Extract features $\\{s^{d}_{i}\\}_{i=1}^{B}$ by $s_{i}^{d}=g(x_{i}^{d})$

4: $\mathcal{L}\\_ERM=\mathcal{L}\\_DS=\mathcal{L}\\_ET=0$

5: for $d=1,2,...,d_{tr}$ do

6:
$\mathcal{L}\\_ERM\mathrel{+}=cross\\_entropy((h^{d}(s_{i}^{d}),y_{i}^{d}))$

7: $\mathcal{L}\\_Entropy\mathrel{+}=\mathcal{ET}$

8: Reverse gradient directions for
$\\{h^{1},h^{2},...,h^{d_{tr}}\\}\backslash\\{h^{d}\\}:h.grad=-\eta h.grad$

9: for $(j=1,2,...,d_{tr})\&j\neq d$ do

10: for $(k=1,2,...,d_{tr})\&(k\neq d)\&(k\neq j)$ do

11:
$\mathcal{L}\\_DS\mathrel{+}=||softmax(h^{j}(s_{i}^{d}))-softmax(h^{k}(s_{i}^{d}))||_{1}$

12: end for

13: end for

14: end for

15:
$\mathcal{L}=\mathcal{L}\\_ERM+\gamma\mathcal{L}\\_ET+\lambda\mathcal{L}\\_DS$

16:
$\Phi_{g},\Phi_{\\{h^{1},h^{2},...,h^{d_{tr}}\\}}=SGD(\mathcal{L},\Phi_{g},\Phi_{\\{h^{1},h^{2},...,h^{d_{tr}}\\}})$

17: end for

Algorithm 2 AdaODM Testing Algorithm

0: num of source domains $d_{tr}$, training steps $N$, batch size $B$, trained
feature extractor $g$, trained domain-specific classifiers
$\\{h^{d}\\}_{d=1}^{d_{tr}}$, testing sets $D^{d_{te}}=\\{x^{d_{te}}\\}$,
class $c\in\\{1,2,...,C\\}$

1: Freeze $\\{h^{d}\\}_{d=1}^{d_{tr}}$

2: Receive data batch $\\{x_{i}^{d_{te}}\\}_{i=1}^{B}$ from $D^{d_{te}}$

3: for $step=1,2,...,N$ do

4: Extract features $\\{s^{d_{te}}_{i}\\}_{i=1}^{B}$ by
$s_{i}^{d_{te}}=g(x_{i}^{d_{te}})$

5: $\mathcal{L}\\_DS=0$

6: for $j=1,2,...,d_{tr}$ do

7: for $(k=1,2,...,d_{tr})\&(k\neq j)$ do

8:
$\mathcal{L}\\_DS\mathrel{+}=||softmax(h^{j}(s_{i}^{d_{te}}))-softmax(h^{k}(s_{i}^{d_{te}}))||_{1}$

9: end for

10: end for

11: $\Phi_{g}=SGD(\mathcal{L}\\_DS,\Phi_{g})$

12: end for

13: Prediction
$\hat{y}^{d_{te}}_{i}=\mathop{\arg\max}_{c}(\sum_{d=1}^{d_{tr}}(h^{d}(g(x_{i}^{d_{te}}))))$

###  III-D Online Disagreement Minimization

Note that the adversarial training above has enabled a domain-invariant
feature extractor and multiple different classifiers. This could improve the
generalization capacity to an extent. However, when faced with query samples
that are far from source domains, domain shift still exists. Fortunately, the
classifiers could effectively capture domain shifts with prediction
disagreement. Based on this, we propose to adjust the feature extractor with
$\mathcal{DS}$ during testing, which allows the model to be optimized for
target domains. Specifically, we fix domain-specific classifiers and finetune
the feature extractor with Eq. (6).

$\mathop{\arg\min}_{\Phi}\mathcal{DS},$ (6)

where $\Phi$ represents the parameters of the feature extractor to be updated.
Following [11], we update the scale and shifting parameters of batch-norm
layers only. This avoids forgetting knowledge learned at the training stage.
As shown in the right of Fig. 2(d), by minimizing prediction disagreement, we
move features of target samples to the domain-invariant feature region, which
largely alleviates domain shift in the latent space. See Algorithm 2 for the
testing stage of AdaODM. Note that AdaODM does not require knowing all test
data ahead. Test samples arrive batch by batch and are processed only once.
Inference and adaptation are performed simultaneously.

##  IV Experiments

###  IV-A Experimental Settings

####  IV-A1 Datasets

We extensively evaluate AdaODM on four standard domain generalization
benchmarks, namely VLCS [51], PACS [52], OfficeHome [53], and TerraIncognita
[54] using the protocol of DomainBed
[1]111https://github.com/facebookresearch/DomainBed. VLCS comprises four
photographic domains $d\in$ {Caltech101, LabelMe, SUN09, VOC2007$\\}$ with
10,729 examples of 5 classes. PACS comprises four domains $d\in$ {art,
cartoons, photos, sketches} with 9,991 examples of 7 classes. OfficeHome
contains four domains $d\in$ {art, clipart, product, real} with 15,588
examples of 65 classes. TerraIncognita contains photographs of wild animals
taken by camera traps at different locations. Following [1], we use four
domains $d\in$ {L100, L38, L43, L46} with 24,788 examples of 10 classes.

####  IV-A2 Baselines

Since AdaODM follows the adaptive domain generalization setting, we compare
with both domain generalization and test-time adaptation algorithms. Domain
generalization algorithms involve Empirical Risk Minimization (ERM) [2],
Invariant Risk Minimization (IRM) [30], Group Distributionally Robust
Optimization (GroupDRO) [55], Inter-domain Mixup (Mixup) [32, 33, 31], Meta-
Learning for Domain Generalization (MLDG) [27], Deep CORrelation ALignment
(CORAL) [16], Maximum Mean Discrepancy (MMD) [24], Domain Adversarial Neural
Networks (DANN) [17], Class-conditional DANN (CDANN) [23], Marginal Transfer
Learning (MTL) [56], Style-Agnostic Networks (SagNet) [57], Adaptive Risk
Minimization (ARM) [58], Variance Risk Extrapolation (VREx) [59],
Representation Self Challenging (RSC) [60], Mixstyle [34], mDSDI [61], and
Adaptive Domain Generalization (DA-ERM and DA-CORAL) [62]. Note that MixStyle
is reproduced number and the others are directly reported from the original
paper and DomainBed [1].

We also compare AdaODM with existing test-time adaptation approaches including
T3A [12], Tent [11], SHOT [10], Pseudo Labeling (PL) [40]. Note that DomainBed
[1] freezes all batch-norm layers before fine-tuning on source domains, batch-
norm statistics keep unchanged throughout the training. Thus, T-BN [9] which
replaces the batch-norm activation statistics estimated on the training sets
with those on the test set, is not suitable for comparison. For the same
reason, we compare with a modified version of Tent (Tent-) which only
optimizes the batch-norm affine parameters by entropy minimization on
predictions of target data.

####  IV-A3 Implementation Details

AdaODM is model-agnostic and can be applied to any domain-invariant-based
domain generalization algorithms. Here we mainly consider ERM and CORAL.
Following DomainBed framework [1], for all algorithms on all datasets, we
choose ImageNet pre-trained ResNet-50 backbone as the feature extractor and
utilize a fully-connected layer for each domain-specific classifier. All
configurations of a dataset are considered where one domain is used for test
and the others serve as training domains. We split each domain of a dataset
into 80% and 20% splits. The larger parts are utilized for training and final
evaluation, and the smaller ones are used for validation. To achieve a fair
comparison, for each algorithm and test domain, we run 20 trials over randomly
searched hyperparameters sampled from distributions predefined in [1].
Learning rate is selected from $10^{Uniform(-5,-3.5)}$, batch size is selected
from $2^{Uniform(3,5.5)}$, dropout rate is selected from [0, 0.1, 0.5], and
weight decay is selected from $10^{Uniform(-6,-2)}$. For introduced $\lambda$
and $\eta$ in our proposed adversarial training procedure, we use
$\lambda\sim\mathcal{U}(0,1)$ and $\eta\sim\mathcal{U}(0,2)$, respectively.
The above process is repeated for three rounds with different hyperparameters,
weight initializations, and dataset splits. The three rounds use seeds
$\\{0,1,2\\}$. For example, there will be totally 240 experiments ($4\
domains\ \times\ 20\ trials\ \times\ 3\ rounds$) for implementing ERM-based
AdaODM on OfficeHome. Every result is reported as a mean and standard error
over three repetitions. The standard training-domain validation is applied to
select the best set of hyperparameters, i.e. smaller splits of each training
domain constitute an overall validation set to choose a model.

To implement TTA baselines, we train source models with ERM and CORAL
following DomainBed, respectively. During inference, we fix batch size to 64
for all experiments of both AdaODM and other TTA approaches. The other
hyperparameters are set to the same as that during training. For AdaODM,
Tent-, SHOT, and PL, two test-time hyperparameters are involved: (1) learning
rate, we choose {0.1, 1.0, 10.0} times of the learning rate used for training
the source model, and (2) steps each test sample is iterated, we choose {1, 3,
5, 7}. T3A has one hyperparameter to decide how many supports to store. We
test {1, 5, 20, 50, 100, ALL} samples. Results are selected similarly as
training.

TABLE I: Domain generalization accuracy for all datasets and algorithms.
Values in red and blue indicates optimal and suboptimal performances in TTA or
Adaptive DG.

Type Algorithm VLCS PACS OfficeHome TerraInc Avg DG ERM [2] 77.5 $\pm$ 0.4
85.5 $\pm$ 0.2 66.5 $\pm$ 0.3 46.1 $\pm$ 1.8 69.0 IRM [30] 78.5 $\pm$ 0.5 83.5
$\pm$ 0.8 64.3 $\pm$ 2.2 47.6 $\pm$ 0.8 68.5 GroupDRO [55] 76.7 $\pm$ 0.6 84.4
$\pm$ 0.8 66.0 $\pm$ 0.7 43.2 $\pm$ 1.1 67.6 Mixup [32, 33, 31] 77.4 $\pm$ 0.6
84.6 $\pm$ 0.6 68.1 $\pm$ 0.3 47.9 $\pm$ 0.8 69.5 MLDG [27] 77.2 $\pm$ 0.4
84.9 $\pm$ 1.0 66.8 $\pm$ 0.6 47.7 $\pm$ 0.9 69.2 CORAL [16] 78.8 $\pm$ 0.6
86.2 $\pm$ 0.3 68.7 $\pm$ 0.3 47.6 $\pm$ 1.0 70.3 MMD [24] 77.5 $\pm$ 0.9 84.6
$\pm$ 0.5 66.3 $\pm$ 0.1 42.2 $\pm$ 1.6 67.7 DANN [17] 78.6 $\pm$ 0.4 83.6
$\pm$ 0.4 65.9 $\pm$ 0.6 46.7 $\pm$ 0.5 68.7 CDANN [23] 77.5 $\pm$ 0.1 82.6
$\pm$ 0.9 65.8 $\pm$ 1.3 45.8 $\pm$ 1.6 67.9 MTL [56] 77.2 $\pm$ 0.4 84.6
$\pm$ 0.5 66.4 $\pm$ 0.5 45.6 $\pm$ 1.2 68.5 SagNet [57] 77.8 $\pm$ 0.5 86.3
$\pm$ 0.2 68.1 $\pm$ 0.1 48.6 $\pm$ 1.0 70.2 ARM [58] 77.6 $\pm$ 0.3 85.1
$\pm$ 0.4 64.8 $\pm$ 0.3 45.5 $\pm$ 0.3 68.3 VREx [59] 78.3 $\pm$ 0.2 84.9
$\pm$ 0.6 66.4 $\pm$ 0.6 46.4 $\pm$ 0.6 69.0 RSC [60] 77.1 $\pm$ 0.5 85.2
$\pm$ 0.9 65.5 $\pm$ 0.9 46.6 $\pm$ 1.0 68.6 MixStyle [34] 77.8 $\pm$ 0.3 85.3
$\pm$ 0.4 61.3 $\pm$ 0.7 44.5 $\pm$ 1.1 67.2 mDSDI [61] 79.0 $\pm$ 0.3 86.2
$\pm$ 0.2 69.2 $\pm$ 0.4 48.1 $\pm$ 1.4 70.6 DA-ERM [62] 78.0 $\pm$ 0.2 84.1
$\pm$ 0.5 67.9 $\pm$ 0.4 47.3 $\pm$ 0.5 69.3 DA-CORAL [62] 78.5 $\pm$ 0.4 84.5
$\pm$ 0.7 68.9 $\pm$ 0.4 48.1 $\pm$ 0.3 70.0 TTA ERM (Vanilla Training) 78.1
$\pm$ 0.6 84.7 $\pm$ 0.9 66.8 $\pm$ 0.4 46.9 $\pm$ 1.1 69.1 +T3A [12] 79.9
$\pm$ 1.6 85.5 $\pm$ 0.9 68.5 $\pm$ 0.5 47.2 $\pm$ 0.7 70.3 +Tent- [11] 77.1
$\pm$ 1.5 85.4 $\pm$ 0.9 65.6 $\pm$ 0.3 44.4 $\pm$ 1.1 68.1 +SHOT [10] 72.7
$\pm$ 0.9 85.5 $\pm$ 0.9 66.9 $\pm$ 0.2 37.0 $\pm$ 1.0 65.5 +PL [40] 73.5
$\pm$ 3.6 84.7 $\pm$ 0.5 61.5 $\pm$ 3.5 40.6 $\pm$ 0.6 65.1 TTA CORAL (Vanilla
Training) 77.8 $\pm$ 0.7 85.2 $\pm$ 0.3 68.9 $\pm$ 0.2 46.3 $\pm$ 0.5 69.6
+T3A [12] 78.4 $\pm$ 0.9 86.2 $\pm$ 0.8 69.4 $\pm$ 0.5 46.8 $\pm$ 2.0 70.2
+Tent- [11] 76.1 $\pm$1.0 85.4 $\pm$ 0.8 68.6 $\pm$ 1.4 47.5 $\pm$ 2.3 69.4
+SHOT [10] 74.4 $\pm$ 3.2 86.3 $\pm$ 0.8 69.6 $\pm$ 0.5 39.0 $\pm$ 1.1 67.3
+PL [40] 72.6 $\pm$ 3.4 84.0 $\pm$ 0.4 66.1 $\pm$ 3.7 47.0 $\pm$ 1.4 67.4
Adaptive DG (Ours) ERM (AdaODM Training) 77.7 $\pm$ 0.5 84.1 $\pm$ 0.2 67.2
$\pm$ 0.3 46.8 $\pm$ 1.1 69.0 +AdaODM Testing 79.7 $\pm$ 0.8 87.8 $\pm$ 0.5
68.8 $\pm$ 0.2 47.7 $\pm$ 1.2 71.0 Adaptive DG (Ours) CORAL (AdaODM Training)
77.1 $\pm$ 0.5 85.4 $\pm$ 0.6 68.4 $\pm$ 0.0 46.2 $\pm$ 3.7 69.3 +AdaODM
Testing 79.1 $\pm$ 1.1 86.6 $\pm$ 1.4 69.9 $\pm$ 0.2 47.5 $\pm$ 1.8 70.8

###  IV-B Comparison with Existing DG Methods

We first compare with existing domain generalization methods. Test accuracies
are summarized in Table I. Table I is split into five blocks. The first block
reports the performance of the existing domain generalization algorithms (the
first block from ERM to DA-CORAL). The fourth and the fifth blocks are results
of AdaODM, built on ERM and CORAL, respectively. Within each of these two
blocks, performance of both base models (directly inference with source
models) and adapted models (via test-time online disagreement minimization)
are posted. Notably, our model involves multiple classifiers, we aggregate
multiple domain-specific classifiers to obtain the final predictions.

Firstly, by comparing the performances of base models trained with Prediction
Disagreement-based Adversarial Training (PD-AT) and those trained in a vanilla
manner, both ERM and CORAL show no significant differences. It is reasonable
since PD-AT applied in our training process builds a feature space invariant
only to source domains. The feature extractor cannot guarantee that target
features are mapped into the shared feature clusters. Therefore, inference
directly using base models gets limited benefits. However, significant
performance gains are achieved after conducting test-time adaptive DG. AdaODM
at test time stably improves the performances of ERM by 2.0%, 3.7%, 1.6%, and
0.9% on VLCS, PACS, OfficeHome, and TerraIncognita, respectively. Similarly,
we consistently improve the performance of CORAL on all four datasets, by
2.0%, 1.2%, 1.5%, and 1.3%, respectively. Notably, although the average
performance of our reproduced CORAL (69.3%) is a little lower than the
reported score (70.3%), we still achieve a higher performance of 70.8% after
test-time adaptive DG. In particular, the proposed method achieves 87.8% for
PACS and 69.9% for OfficeHome, which both surpass the best-reported scores.
Overall, we achieve state-of-the-art performance of 71.0% with ERM in terms of
the average test accuracy. This indicates that during inference, by
encouraging prediction consistency, the feature extractor effectively aligns
target features to the domain-invariant feature space.

From another perspective, we take OfficeHome as an example to visualize the
effect of the adversarial training in Fig. 3. Specifically, we train vanilla
ERM models (without prediction disagreement-based adversarial training) and
ERM-based AdaODM models (with prediction disagreement-based adversarial
training). Then we visualize the learned features using t-SNE [63]. The left
pair shows an example that trains the source model on “art”, “product”, and
“real” domains, and generalizes to “clipart” domain, while the right pair
tests on “art” domain and trains on the other domains. Within each pair, the
two figures illustrate the source features that are generated without and with
adversarial training, respectively. It could be easily observed that the
adversarial training procedure successfully clusters features from various
domains. To evaluate whether the test-time online disagreement minimization
benefits the target feature alignment, we apply a CORAL-like metric as an
indicator to measure the distance between the learned source and target
feature distributions. The metric is calculated as the second-order difference
of mean and covariance values between source and target features. For the
above two trials, the test-time optimization process of AdaODM improves the
test accuracy from 54.1% to 59.4%, and from 61.7% to 64.0%, respectively. And
the metric decreases from 9.07 to 5.30, and from 1.43 to 1.39, respectively.
This implies our test-time online disagreement minimization indeed mitigates
the distribution gap between source and target domains.

###  IV-C Comparison with Existing TTA Methods

We further compare AdaODM with test-time adaptation methods, including T3A,
Tent-, SHOT, and PL. AdaODM is built on source models with multiple domain-
specific classifiers, which is different from baseline TTA methods. To make a
fair comparison, we train source models separately for AdaODM and existing TTA
methods, i.e., we train ERM and CORAL on vanilla architectures with one
feature extractor and one classifier for baseline TTA methods, and train ERM
and CORAL on architectures with multiple domain-specific classifiers via PD-AT
for our test-time adaptive DG. T3A adjusts a pseudo-prototypical classifier
with target features to adapt new domains. Tent- optimizes batch-norm
transformations with prediction entropy minimization. SHOT freezes the
classifier and updates the feature extractor with prediction entropy
minimization, diversity regularization, and cross-entropy minimization between
prediction and the pseudo label. PL optimizes the whole network with the
pseudo label. The pseudo labels in SHOT and PL are both determined by
thresholding the prediction probabilities with 0.9. T3A, Tent-, SHOT, and PL
are conducted based on the same base models. All base models are selected
based on the training-domain validation standard from 20 $\times$ 3 training
models for each test environment.

[w/o AT]  [w/ AT]

(a) Target domain: Clipart

[w/o AT]  [w/ AT]

(a) Target domain: Art

Figure 3: Two examples to illustrate the effectiveness of the prediction
disagreement-based adversarial training. The left and the right pairs are
tested on “clipart” and “art” domains of the OfficeHome dataset, respectively,
and trained on the other three domains. Within each pair, the two figures
visualize generated source domain features trained without and with the
Adversarial Training (AT). Three colors represent three source domains. We can
see that AT effectively encourages domain-invariant feature clusters.

The second and the third blocks in Table I summarize performances of baseline
TTA methods with ERM and CORAL, respectively. It can be observed that among
all baselines, only T3A gets improved performance over all datasets. SHOT
performs better than base models on PACS and OfficeHome. Tent- receives better
performance on PACS. PL obtains poor performance on most datasets. However,
our method consistently outperforms T3A or achieves comparable performances.
In particular, our method surpasses T3A by 0.7% with ERM and 0.5% with CORAL.
Entropy minimization used in SHOT and Tent- benefits adaptation on several
datasets, but fails on the others. Minimizing prediction entropy prompts more
confident supervision, and might be effective in some circumstances. However,
the optimization direction cannot be guaranteed to the right class. Compared
to existing TTA methods, our method first builds a domain-invariant feature
space by imposing consistency regularization. With the same procedure during
inference, target features can also be aligned to the shared feature space.
Meanwhile, domain-specific classifiers could better measure the distribution
difference between the target domain and source domains. Both factors
contribute to the reliability of the optimization direction at test time.

###  IV-D Quantitative analysis of adaptive DG results

We evaluate how much the adaptive DG procedure (test-time online disagreement
minimization) of AdaODM could boost the target domain performance. Since we
have run 60 experiments ($20\ trials\ \times\ 3\ rounds$) for each target
domain with different hyperparameters and dataset splits, performance gain
brought by the test-time online disagreement minimization for each experiment
could be calculated. Fig. 4 summarizes performance gains of each domain of
OfficeHome, PACS, and TerraIncognita with CORAL-based AdaODM using box plots.
We observe that for OfficeHome, the third-quarter quantile of the performance
gain is larger than 0 for all target domains, which means AdaODM is robust to
the hyperparameters of the base model. On PACS and TerraIncognita, performance
gains exhibit more volatile fluctuations. AdaODM could improve the performance
of base models by more than 5% with suitable hyperparameters. AdaODM does not
work quite well when generalizing to the “art” domain of PACS, “L43”, and
“L46” domains of TerraIncognita.

(a) CORAL-based AdaODM on OfficeHome

(b) CORAL-based AdaODM on PACS

(c) CORAL-based AdaODM on TerraIncognita

Figure 4: Distribution of performance gains by CORAL-based AdaODM with
different hyperparameters ($20\ trials\times 3\ rounds$ for each target
domain) on OfficeHome, PACS, and TerraIncognita using box plot.

###  IV-E AdaODM with single source domain

Since domain-specific classifiers in AdaODM are built based on the number of
source domains, we are also interested in how AdaODM behaves when there is
only a single source domain. Under such a circumstance, we instead construct
classifiers with different initialized weights to improve the diversity. Thus,
the number of classifiers is independent of the number of source domains.
Specifically, we learn 3 different classifiers on a single source domain and
implement AdaODM based on ERM. We consider all generalization pairs of each
dataset. Every pair is conducted once with default hyperparameters. Table II
describes the detailed results. We could observe that even with only one
source domain, our method still improves the performance at testing stage by
1.4%, 1.0%, 1.5%, and 1.6% in average for VLCS, PACS, OfficeHome, and
TerraIncognita, respectively.

TABLE II: Performance of AdaODM for single source domain generalization. Bold
numbers indicate performance improvements from base models. Note that
experiments are conducted only on the default hyperparameters of ERM-based
AdaODM.

VLCS C$\xrightarrow{}$L C$\xrightarrow{}$S C$\xrightarrow{}$V
L$\xrightarrow{}$C L$\xrightarrow{}$S L$\xrightarrow{}$V S$\xrightarrow{}$C
S$\xrightarrow{}$L S$\xrightarrow{}$V V$\xrightarrow{}$C V$\xrightarrow{}$L
V$\xrightarrow{}$S Avg AdaODM Training 48.0 50.0 57.8 31.4 59.2 62.9 81.0 64.3
70.4 93.2 62.7 75.1 63.0 +AdaODM Testing 48.4 50.6 57.5 32.6 58.6 63.5 89.3
67.1 72.1 94.3 62.2 76.2 64.4 PACS A$\xrightarrow{}$C A$\xrightarrow{}$P
A$\xrightarrow{}$S C$\xrightarrow{}$A C$\xrightarrow{}$P C$\xrightarrow{}$S
P$\xrightarrow{}$A P$\xrightarrow{}$C P$\xrightarrow{}$S S$\xrightarrow{}$A
S$\xrightarrow{}$C S$\xrightarrow{}$P Avg AdaODM Training 74.6 94.9 70.4 75.0
90.6 71.4 51.2 49.1 56.6 64.3 69.4 62.6 69.2 +AdaODM Testing 76.0 96.8 74.4
76.8 91.5 73.4 51.5 48.9 56.4 64.9 69.4 61.8 70.2 OfficeHome
A$\xrightarrow{}$C A$\xrightarrow{}$P A$\xrightarrow{}$R C$\xrightarrow{}$A
C$\xrightarrow{}$P C$\xrightarrow{}$R P$\xrightarrow{}$A P$\xrightarrow{}$C
P$\xrightarrow{}$R R$\xrightarrow{}$A R$\xrightarrow{}$C R$\xrightarrow{}$P
Avg AdaODM Training 44.2 57.2 68.2 44.2 54.9 56.7 40.3 38.3 64.2 57.8 46.1
70.9 53.6 +AdaODM Testing 46.9 58.2 69.3 45.7 55.5 58.2 42.5 39.1 66.1 59.4
46.0 74.2 55.1 TerraInc L100$\xrightarrow{}$L38 L100$\xrightarrow{}$L43
L100$\xrightarrow{}$L46 L38$\xrightarrow{}$L100 L38$\xrightarrow{}$L43
L38$\xrightarrow{}$L46 L43$\xrightarrow{}$L100 L43$\xrightarrow{}$L38
L43$\xrightarrow{}$L46 L46$\xrightarrow{}$L100 L46$\xrightarrow{}$L38
L46$\xrightarrow{}$P Avg AdaODM Training 16.2 29.4 38.8 34.2 21.9 21.8 41.7
44.7 39.2 40.2 45.3 54.8 35.7 +AdaODM Testing 17.0 29.1 39.2 34.6 22.3 24.1
48.4 46.0 39.5 38.9 53.8 55.0 37.3

TABLE III: Ablation studies to prove the effectiveness of domain-specific
classifiers and adversarial training. Bold numbers indicate best results in
each setting.

Base DG Method Algorithm VLCS PACS OfficeHome TerraInc Avg ERM full 79.7 $\pm$
0.8 87.8 $\pm$ 0.5 68.8 $\pm$ 0.2 47.7 $\pm$ 1.2 71.0 w/o domain-specific
classifiers 77.7 $\pm$ 1.0 84.7 $\pm$ 1.4 66.5 $\pm$ 1.0 46.2 $\pm$ 0.6 68.8
w/o adversarial training 77.6 $\pm$ 2.0 85.9 $\pm$ 0.6 67.1 $\pm$ 0.7 45.6
$\pm$ 6.4 69.1 CORAL full 79.1 $\pm$ 1.1 86.6 $\pm$ 1.4 69.9 $\pm$ 0.2 47.5
$\pm$ 1.8 70.8 w/o domain-specific classifiers 78.6 $\pm$ 0.3 86.6 $\pm$ 0.5
69.4 $\pm$ 0.7 47.1 $\pm$ 1.4 70.4 w/o adversarial training 77.7 $\pm$ 0.3
86.3 $\pm$ 0.8 68.7 $\pm$ 0.2 47.3 $\pm$ 3.6 70.0

TABLE IV: The effect of the number of source domains.

Num. of VLCS PACS OfficeHome TerraInc Avg src. domains 2 AdaODM Training 64.5
71.3 58.3 35.5 57.4 +AdaODM Testing 64.8 71.6 58.3 36.4 57.8 3 AdaODM Training
75.0 83.5 65.6 41.9 66.5 +AdaODM Testing 75.6 84.0 66.1 42.3 67.0

###  IV-F The effect of the number of source domains

We evaluate how the number of source domains affects the generalization
performance. Since VLCS, PACS, OfficeHome, and TerraIncognita each contain
four domains, we consider two or three source domains used for training and
the remaining one domain serves as the target domain. To make a fair
comparison, 60% samples of each source domain are included for training when
using three source domains. When the number of source domains is two, the
ratio of included images for each source domain is determined accordingly to
roughly ensure the total number of training samples is the same. For the case
where the number of source domains is two, we report the average performance
of sampling any two from the three source domains. When there are two source
domains present for training, the number of “meta-target” classifiers is only
1 for each source domain. Therefore, we build two differently initialized
domain-specific classifiers for each source domain to calculate the prediction
disagreement score.

For each dataset, the final results are obtained by averaging over the four
scenarios where each domain serves as the target domain. All experiments are
conducted on the default hyperparameters of ERM-based AdaODM. From Table IV we
can see that when decreasing the number of source domains from 3 to 2,
performance drops heavily: the results of vanilla inference drop from 66.5% to
57.4%. However, we still observe the performance gain brought by test-time
training. We also notice that the gains on each dataset become more fluctuated
when decreasing the number of source domains, i.e., results on VLCS, PACS, and
OfficeHome are less boosted while TerraIncognita gets higher improvement by
test-time optimization.

###  IV-G Choice to calculate $\mathcal{DS}$

Table V compares test accuracies of computing $\mathcal{DS}$ by $L_{1}$\- and
$L_{2}$-norm, and $KL$ Divergence. Experiments are conducted once on each
target domain of each dataset with default hyperparameters. ERM-based AdaODM
is implemented. Results empirically show that the adaptive DG benefits much
more from utilizing $L_{1}$-norm to calculate $\mathcal{DS}$. This is
understandable since $L_{1}$-norm encourages prediction differences for each
class to be more sparse. Thus, models focus more on certain classes. Since
classifiers are prone to be less reliable at the early training stage, both
$L_{2}$-norm and $KL$ Divergence may lead meta-target classifiers to agree on
wrong predicted distributions.

TABLE V: Comparison of choices for computing $\mathcal{DS}$. Bold numbers
indicate performance improvements from base models. Note that experiments are
conducted only on the default hyperparameters of ERM-based AdaODM.

Choice for $\mathcal{DS}$ VLCS PACS OfficeHome TerraInc Avg $L_{1}$ AdaODM
Training 75.7 84.4 65.6 41.9 66.9 +AdaODM Testing 75.7 85.1 66.9 42.8 67.6
$L_{2}$ AdaODM Training 77.2 67.6 64.0 42.9 62.9 +AdaODM Testing 76.9 68.8
63.6 42.8 63.0 $KL$ AdaODM Training 73.2 75.1 64.0 41.3 63.4 +AdaODM Testing
72.7 75.5 63.9 41.0 63.3

(a) VLCS

(b) PACS

(c) OfficeHome

(d) TerraIncognita

Figure 5: Effect of the loss weight term $\lambda$ for minimizing prediction
disagreement score. Experiments are conducted on ERM-based AdaODM. $\lambda$
is set to $\\{0,0.3,0.6,0.9,1.2,1.5\\}$. The other hyperparameters are set by
default.

###  IV-H Ablation Studies

We further verify the effectiveness of the adversarial training. To do so, we
mainly implement two variants of our methods. (1) w/o domain-specific
classifiers: we replace domain-specific classifiers with regular classifiers,
i.e. each classifier is trained on all source domains. Thus classifiers will
not measure domain differences effectively. Adversarial training is also not
considered for this setting. (2) w/o the prediction disagreement-based
adversarial training: each classifier still receives information from one
separate domain but adversarial training is not applied. We conduct the two
kinds of variants on all datasets with both ERM and CORAL. Extensive
experiments are performed for each target domain ($20\ trials\ \times\ 3\
rounds$) in this section. Table III compares the two variants with the full
version of our method. Note that all numbers are performances after conducting
test-time online disagreement minimization.

We can make the following observations. (1) For both ERM- and CORAL-based
AdaODM, performances drop when domain-specific classifiers or adversarial
training strategy is removed. Although classifiers are initialized with
different weights, they show similar behavior when receiving the same
knowledge instead of domain-specific one. Thus it is difficult for the
adversarial training to capture the prediction inconsistency among
classifiers. Likewise, without adversarial training, each domain-specific
classifier is updated only based on one source domain. The amount of training
samples is too small and thus they are more likely to overfit to the
corresponding source domains. Minimizing prediction disagreement for the
target samples will be less reliable. Compared with the two variants, the full
version of our method improves target performance consistently and
significantly. This proves the necessity to maximize the discrepancy of source
classifiers and generalize domain-specific classifiers to unseen source
domains. (2) Comparing the performance gaps between the full AdaODM and the
two variants, ERM-based AdaODM is more influenced with either module removed.
Since CORAL encourages feature alignment, domain differences in ERM-based
experiments are much larger than those in CORAL-based experiments. This again
emphasizes the critical roles of our adversarial training strategy for
handling large distribution shifts.

Since we set the test-time batch size to 64 for both AdaODM and baselines. We
are also interested in how the performance will be when test samples come one
by one, i.e., the test-time batch size equals 1. We evaluate this on ERM-based
AdaODM. Results shown in Table VI indicate that when test samples arrive one
by one, the test results become slightly lower and more fluctuated. This is
reasonable since the gradient direction is less stable when the batch size is
small.

TABLE VI: Ablation studies to show the effect of test-time batch size on the
performance. Bold numbers indicate the best results in each setting.

Test-time batch size VLCS PACS OfficeHome TerraInc Avg 1 79.2 $\pm$ 1.3 86.8
$\pm$ 1.1 68.3 $\pm$ 0.9 47.1 $\pm$ 1.6 70.5 64 (default) 79.7 $\pm$ 0.8 87.8
$\pm$ 0.5 68.8 $\pm$ 0.2 47.7 $\pm$ 1.2 71.0

We provide an additional hyper-parameter sensitivity study for the
disagreement score $\lambda$. The experiments are conducted on ERM-based
AdaODM with default hyperparameters except that the loss weight term $\lambda$
is set in the range of {0, 0.3, 0.6, 0.9, 1.2, 1.5}. Fig. 5 illustrates the
performance change with different $\lambda$. We can observe that the
prediction disagreement minimization loss consistently improves the
generalization performance.

##  V Conclusion

We have proposed an adaptive domain generalization approach AdaODM, which aims
to adapt a source model to new domains at test time. The main idea of AdaODM
is to: (1) build domain-invariant feature space during training and (2) align
target domain features to the invariant feature region at test time, both
through promoting prediction consistency. To achieve this, AdaODM learns a
domain-generic feature extractor and multiple domain-specific classifiers via
a novel prediction disagreement-based adversarial training strategy. Extensive
experimental results on four public benchmarks demonstrate that AdaODM is able
to align target features effectively, which leads to state-of-the-art
performance for domain generalization.

## References

  * [1] I. Gulrajani and D. Lopez-Paz, “In search of lost domain generalization,” in _International Conference on Learning Representations_ , 2020.
  * [2] V. N. Vapnik, “An overview of statistical learning theory,” _IEEE transactions on neural networks_ , vol. 10, no. 5, pp. 988–999, 1999.
  * [3] G. Wilson and D. J. Cook, “A survey of unsupervised deep domain adaptation,” _ACM Transactions on Intelligent Systems and Technology (TIST)_ , vol. 11, no. 5, pp. 1–46, 2020.
  * [4] G. Csurka, “Domain adaptation for visual applications: A comprehensive survey,” _arXiv preprint arXiv:1702.05374_ , 2017.
  * [5] X. Peng, Q. Bai, X. Xia, Z. Huang, K. Saenko, and B. Wang, “Moment matching for multi-source domain adaptation,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 1406–1415.
  * [6] Y. Ganin and V. Lempitsky, “Unsupervised domain adaptation by backpropagation,” in _International conference on machine learning_. PMLR, 2015, pp. 1180–1189.
  * [7] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell, “Adversarial discriminative domain adaptation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 7167–7176.
  * [8] X. Hu, G. Uzunbas, S. Chen, R. Wang, A. Shah, R. Nevatia, and S.-N. Lim, “Mixnorm: Test-time adaptation through online normalization estimation,” _arXiv preprint arXiv:2110.11478_ , 2021.
  * [9] S. Schneider, E. Rusak, L. Eck, O. Bringmann, W. Brendel, and M. Bethge, “Removing covariate shift improves robustness against common corruptions,” in _Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS)_ , 2020.
  * [10] J. Liang, D. Hu, and J. Feng, “Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 6028–6039.
  * [11] D. Wang, E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell, “Tent: Fully test-time adaptation by entropy minimization,” in _International Conference on Learning Representations_ , 2020.
  * [12] Y. Iwasawa and Y. Matsuo, “Test-time classifier adjustment module for model-agnostic domain generalization,” _Advances in Neural Information Processing Systems_ , vol. 34, 2021.
  * [13] P. Luo, F. Zhuang, H. Xiong, Y. Xiong, and Q. He, “Transfer learning from multiple source domains via consensus regularization,” in _Proceedings of the 17th ACM conference on Information and knowledge management_ , 2008, pp. 103–112.
  * [14] M. Long, H. Zhu, J. Wang, and M. I. Jordan, “Deep transfer learning with joint adaptation networks,” in _International conference on machine learning_. PMLR, 2017, pp. 2208–2217.
  * [15] A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola, “A kernel two-sample test,” _The Journal of Machine Learning Research_ , vol. 13, no. 1, pp. 723–773, 2012.
  * [16] B. Sun and K. Saenko, “Deep coral: Correlation alignment for deep domain adaptation,” in _European conference on computer vision_. Springer, 2016, pp. 443–450.
  * [17] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” _The journal of machine learning research_ , vol. 17, no. 1, pp. 2096–2030, 2016.
  * [18] K. Saito, K. Watanabe, Y. Ushiku, and T. Harada, “Maximum classifier discrepancy for unsupervised domain adaptation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 3723–3732.
  * [19] M. Long, Z. Cao, J. Wang, and M. I. Jordan, “Conditional adversarial domain adaptation,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [20] J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell, “Cycada: Cycle-consistent adversarial domain adaptation,” in _International conference on machine learning_. PMLR, 2018, pp. 1989–1998.
  * [21] R. Li, Q. Jiao, W. Cao, H.-S. Wong, and S. Wu, “Model adaptation: Unsupervised domain adaptation without source data,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 9641–9650.
  * [22] J. N. Kundu, N. Venkat, R. V. Babu _et al._ , “Universal source-free domain adaptation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 4544–4553.
  * [23] Y. Li, X. Tian, M. Gong, Y. Liu, T. Liu, K. Zhang, and D. Tao, “Deep domain generalization via conditional invariant adversarial networks,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 624–639.
  * [24] H. Li, S. J. Pan, S. Wang, and A. C. Kot, “Domain generalization with adversarial feature learning,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 5400–5409.
  * [25] O. Sener and V. Koltun, “Domain generalization without excess empirical risk,” in _Advances in Neural Information Processing Systems_.
  * [26] Y. Ding, L. Wang, B. Liang, S. Liang, Y. Wang, and F. Chen, “Domain generalization by learning and removing domain-specific features,” in _Advances in Neural Information Processing Systems_.
  * [27] D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Learning to generalize: Meta-learning for domain generalization,” in _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018.
  * [28] Q. Dou, D. Coelho de Castro, K. Kamnitsas, and B. Glocker, “Domain generalization via model-agnostic learning of semantic features,” _Advances in Neural Information Processing Systems_ , vol. 32, 2019.
  * [29] Y. Balaji, S. Sankaranarayanan, and R. Chellappa, “Metareg: Towards domain generalization using meta-regularization,” _Advances in neural information processing systems_ , vol. 31, 2018.
  * [30] M. Arjovsky, L. Bottou, I. Gulrajani, and D. Lopez-Paz, “Invariant risk minimization,” _arXiv preprint arXiv:1907.02893_ , 2019.
  * [31] Y. Wang, H. Li, and A. C. Kot, “Heterogeneous domain generalization via domain mixup,” in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 3622–3626.
  * [32] M. Xu, J. Zhang, B. Ni, T. Li, C. Wang, Q. Tian, and W. Zhang, “Adversarial domain adaptation with domain mixup,” in _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 04, 2020, pp. 6502–6509.
  * [33] S. Yan, H. Song, N. Li, L. Zou, and L. Ren, “Improve unsupervised domain adaptation with mixup training,” _arXiv preprint arXiv:2001.00677_ , 2020\.
  * [34] K. Zhou, Y. Yang, Y. Qiao, and T. Xiang, “Domain generalization with mixstyle,” in _International Conference on Learning Representations_ , 2020\.
  * [35] S. Jeon, K. Hong, P. Lee, J. Lee, and H. Byun, “Feature stylization and domain-aware contrastive learning for domain generalization,” in _Proceedings of the 29th ACM International Conference on Multimedia_ , 2021, pp. 22–31.
  * [36] Y. Wang, L. Qi, Y. Shi, and Y. Gao, “Feature-based style randomization for domain generalization,” _arXiv preprint arXiv:2106.03171_ , 2021.
  * [37] Y. Sun, X. Wang, Z. Liu, J. Miller, A. Efros, and M. Hardt, “Test-time training with self-supervision for generalization under distribution shifts,” in _International Conference on Machine Learning_. PMLR, 2020, pp. 9229–9248.
  * [38] S. Yang, D. Das, J. Cho, H. Park, and S. Yun, “Domain agnostic few-shot learning for speaker verification,” _arXiv preprint arXiv:2206.13700_ , 2022\.
  * [39] Y. Liu, P. Kothari, B. van Delft, B. Bellot-Gurlet, T. Mordan, and A. Alahi, “Ttt++: When does self-supervised test-time training fail or thrive?” _Advances in Neural Information Processing Systems_ , vol. 34, 2021.
  * [40] D.-H. Lee _et al._ , “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in _Workshop on challenges in representation learning, ICML_ , vol. 3, no. 2, 2013, p. 896.
  * [41] M. Zhang, S. Levine, and C. Finn, “Memo: Test time robustness via adaptation and augmentation,” _arXiv preprint arXiv:2110.09506_ , 2021.
  * [42] T. Miyato, S.-i. Maeda, M. Koyama, and S. Ishii, “Virtual adversarial training: a regularization method for supervised and semi-supervised learning,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 41, no. 8, pp. 1979–1993, 2018.
  * [43] Q. Xie, Z. Dai, E. Hovy, T. Luong, and Q. Le, “Unsupervised data augmentation for consistency training,” _Advances in Neural Information Processing Systems_ , vol. 33, pp. 6256–6268, 2020.
  * [44] H. Zhang, Z. Zhang, A. Odena, and H. Lee, “Consistency regularization for generative adversarial networks,” in _International Conference on Learning Representations_ , 2019.
  * [45] S. Sinha and A. B. Dieng, “Consistency regularization for variational auto-encoders,” _Advances in Neural Information Processing Systems_ , vol. 34, 2021.
  * [46] A. Tarvainen and H. Valpola, “Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” _Advances in neural information processing systems_ , vol. 30, 2017\.
  * [47] G. French, M. Mackiewicz, and M. Fisher, “Self-ensembling for visual domain adaptation,” _arXiv preprint arXiv:1706.05208_ , 2017.
  * [48] Y. Wu, D. Inkpen, and A. El-Roby, “Dual mixup regularized learning for adversarial domain adaptation,” in _European Conference on Computer Vision_. Springer, 2020, pp. 540–555.
  * [49] Q. Xu, R. Zhang, Y. Zhang, Y. Wang, and Q. Tian, “A fourier-based framework for domain generalization,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 14 383–14 392.
  * [50] J. Vanschoren, “Meta-learning,” in _Automated Machine Learning_. Springer, Cham, 2019, pp. 35–61.
  * [51] C. Fang, Y. Xu, and D. N. Rockmore, “Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2013, pp. 1657–1664.
  * [52] D. Li, Y. Yang, Y.-Z. Song, and T. M. Hospedales, “Deeper, broader and artier domain generalization,” in _Proceedings of the IEEE international conference on computer vision_ , 2017, pp. 5542–5550.
  * [53] H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 5018–5027.
  * [54] S. Beery, G. Van Horn, and P. Perona, “Recognition in terra incognita,” in _Proceedings of the European conference on computer vision (ECCV)_ , 2018, pp. 456–473.
  * [55] S. Sagawa, P. W. Koh, T. B. Hashimoto, and P. Liang, “Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization,” _arXiv preprint arXiv:1911.08731_ , 2019.
  * [56] G. Blanchard, G. Lee, and C. Scott, “Generalizing from several related classification tasks to a new unlabeled sample,” _Advances in neural information processing systems_ , vol. 24, 2011.
  * [57] H. Nam, H. Lee, J. Park, W. Yoon, and D. Yoo, “Reducing domain gap via style-agnostic networks,” _arXiv preprint arXiv:1910.11645_ , vol. 2, no. 7, p. 8, 2019.
  * [58] M. Zhang, H. Marklund, N. Dhawan, A. Gupta, S. Levine, and C. Finn, “Adaptive risk minimization: Learning to adapt to domain shift,” _Advances in Neural Information Processing Systems_ , vol. 34, 2021.
  * [59] D. Krueger, E. Caballero, J.-H. Jacobsen, A. Zhang, J. Binas, D. Zhang, R. Le Priol, and A. Courville, “Out-of-distribution generalization via risk extrapolation (rex),” in _International Conference on Machine Learning_. PMLR, 2021, pp. 5815–5826.
  * [60] Z. Huang, H. Wang, E. P. Xing, and D. Huang, “Self-challenging improves cross-domain generalization,” in _European Conference on Computer Vision_. Springer, 2020, pp. 124–140.
  * [61] M.-H. Bui, T. Tran, A. Tran, and D. Phung, “Exploiting domain-specific features to enhance domain generalization,” _Advances in Neural Information Processing Systems_ , vol. 34, 2021.
  * [62] A. Dubey, V. Ramanathan, A. Pentland, and D. Mahajan, “Adaptive methods for real-world domain generalization,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 14 340–14 349.
  * [63] L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” _Journal of machine learning research_ , vol. 9, no. 11, 2008.