11institutetext: National University of Singapore 22institutetext: Zhejiang
University 33institutetext: University of Science and Technology of China
33email<EMAIL_ADDRESS>33email<EMAIL_ADDRESS>
33email<EMAIL_ADDRESS>33email<EMAIL_ADDRESS>
# Disentangling Masked Autoencoders for Unsupervised Domain Generalization

An Zhang These authors contribute equally to this work.An Zhang is the
corresponding author.  11 0000-0003-1367-711X Han Wang∗ 22 0009-0001-1108-8349
Xiang Wang  33 0000-0002-6148-6329 Tat-Seng Chua  11 0000-0001-6097-7807

###### Abstract

Domain Generalization (DG), designed to enhance out-of-distribution (OOD)
generalization, is all about learning invariance against domain shifts
utilizing sufficient supervision signals. Yet, the scarcity of such labeled
data has led to the rise of unsupervised domain generalization (UDG) — a more
important yet challenging task in that models are trained across diverse
domains in an unsupervised manner and eventually tested on unseen domains. UDG
is fast gaining attention but is still far from well-studied.

To close the research gap, we propose a novel learning framework designed for
UDG, termed the Disentangled Masked AutoEncoder (DisMAE), aiming to discover
the disentangled representations that faithfully reveal the intrinsic features
and superficial variations without access to the class label. At its core is
the distillation of domain-invariant semantic features, which can not be
distinguished by domain classifier, while filtering out the domain-specific
variations (for example, color schemes and texture patterns) that are unstable
and redundant. Notably, DisMAE co-trains the asymmetric dual-branch
architecture with semantic and lightweight variation encoders, offering
dynamic data manipulation and representation level augmentation capabilities.
Extensive experiments on four benchmark datasets (_i.e_. DomainNet, PACS,
VLCS, Colored MNIST) with both DG and UDG tasks demonstrate that DisMAE can
achieve competitive OOD performance compared with the state-of-the-art DG and
UDG baselines, which shed light on potential research line in improving the
generalization ability with large-scale unlabeled data. Our codes are
available at https://github.com/rookiehb/DisMAE.

###### Keywords:

Disentanglement Domain Invariance Unsupervised Domain Generalization

##  1 Introduction

Domain Generalization (DG) strives to achieve robustness against domain shifts
by leveraging high-quality supervision signals [56, 46, 55]. However, data is
never sufficient for today’s large foundation models, and the demands for
large-scale labeled source data considerably hinder the breakthrough toward
truly generalized models [47]. Thus, we delve into a more practical yet
challenging task - Unsupervised Domain Generalization (UDG) [60]. Distinct
from DG, UDG focuses on learning representations that generalize well from
source domains to unseen domains without relying on class labels.

Figure 1: Illustrative reconstructed images generated by DisMAE. Rows 1 and 3
present inputs retaining either semantic or variation attributes, sourced from
the ColorMNIST and three distinct domains of the DomainNet for a comprehensive
comparison. Rows 2 and 4 display images crafted by augmenting either original
or alternate image variation representations in feature space. The evident
integration of colors, textures, and backgrounds in these reconstructed images
highlights the disentangling capability of DisMAE. More examples can be found
in Appendix 0.C.6.

We argue that the transition from DG to UDG is non-trivial. Cutting-edge DG
methods, which have shown great success in supervised data, are often
ineffective in the UDG task [21]. These methods largely depend on inductive
biases derived from available labels [7], leading to suboptimal performance
compared to self-supervised learning frameworks such as MoCo [14], MAE [23],
and SimCLR [12] without supervision. Notably, models trained under UDG using
unlabeled heterogeneous data can be easily repurposed for DG tasks by
assembling classification loss (as elaborated in the methodology section).
Given these observations, our study emphasizes UDG’s potential, a research
direction that remains largely under-explored.

In light of these challenges and opportunities of UDG, we aim to distill
domain-invariant semantic features that faithfully reflect intrinsic
properties of the data, ensuring generalization to unseen distributions in an
unsupervised manner. A notable challenge is the entanglement of these features
with domain-specific variations such as color schemes and texture patterns
[50]. Recent advances in DG research hint at a promising direction: the
decoupling of domain-invariant attributes from these peripheral variation
factors [6, 8, 49]. However, with no access to the class label, ensuring
domain-invariant features truly encapsulate inherent data attributes poses
great challenges [58, 3]. Furthermore, in some real-world scenarios,
distribution shifts among source domains overshadow intra-domain class
differences [59]. For instance, the transformation from a photo to a sketch of
a Teddy manifests more significant changes than transitioning to a Bichon.
This strong heterogeneity often results in representations emphasizing broad
distributional shifts over nuanced domain-specific variations.

Toward this end, we propose a new learning framework designed for UDG,
Disentangled Masked AutoEncoder (DisMAE), that integrates invariance and
disentanglement principles. By the disentanglement principle, we mean that the
representations are decomposed into two components - semantic features and
variations. To force the semantic representations to capture less redundant
information and thus preferably disentangle the variations, we pull the
reconstructed samples with their original semantics and variations closer to
the input and push the reconstructed samples with different in-domain
variations apart. By invariance principle, we mean that the semantic
representations are invariant throughout a variety of domain changes,
therefore their domain category is undistinguished. Put simply, variations
should encapsulate domain-specific features, leaving the disentangled semantic
representations to exclusively preserve intrinsic information.

Guided by these two principles, our DisMAE strategy incorporates four modules:
a transformer-based semantic encoder, a lightweight variation encoder, a
transformer-based decoder, and a domain label-enhanced invariance classifier.
For the reconstruction part, the asymmetric dual-branch architecture with two
separate encoders and one single decoder is implemented together to avoid
information collapse. We additionally introduce an adaptive contrastive loss
to keep two branches capturing information in a principled disentangled
manner. Specifically, towards the disentanglement principle, target semantic
representations are concatenated with other intra-domain samples’ variations,
ensuring the reconstructed samples diverge from the original input. Towards
the invariance principle, the domain label-enhanced invariance classifier
predicts the probability of correctly identifying the domain category, and
these probabilities are then inversely applied as the adaptive weights to re-
weight the contrastive loss. Jointly training under these two principles
enables the DisMAE models to disentangle the domain-invariant semantic
features and domain-specific variations, and further boosts the model’s
capability in enriching the variation through latent space augmentation.

##  2 Preliminary

The goal of unsupervised domain generalization (UDG) is to learn a domain-
invariant feature extractor from diverse domains in a self-supervised manner,
thereby enabling robust generalization to the target unseen domain [54]. Let
$\mathcal{D}_{\text{train}}=\\{x\in\mathcal{I}_{d}|d\in\\{1,\cdots,M-1\\}\\}$
be the training set that involves $M-1$ source domains. Each training domain
$\mathcal{I}_{d}$ comprises a collection of unlabeled images $\\{x_{i}\\}$.
Let $\mathcal{D}_{\text{test}}=\\{x\in\mathcal{I}_{M}\\}$ represent the test
set containing previously unseen images from the target domain
$\mathcal{I}_{M}$.

In order to learn a domain-invariant feature extractor $\phi_{s}$ in self-
supervised learning, there is a foundation assumption [48]: any given sample
$x$ is generated from two disentangled features,
$x=g(\mathbf{s}_{x},\mathbf{v}_{x})$, where $\mathbf{s}_{x}$ denotes the
semantic representation that invariant to domain shifts, $\mathbf{v}_{x}$
signifies the superficial variations that change across domains. The
generative function $g(\cdot,\cdot)$ maps these two latent space
representations back into the sample space. In particular, this assumption
naturally encodes two principles: disentanglement and invariance principles.

Disentanglement Principle. There exists a direct product decomposition of
attributes as well as feature representations in real-world visual generation
scenarios.

Figure 2: t-SNE visualization of MAE representations on DomainNet. Points are
color-coded based on their domain labels. MAE demonstrates inconsistent
feature spaces across different domains, stemming from capturing entangled
features of both semantics and variations.

Let $s_{x}$ be the semantic attribute of sample $x$, and $v_{x}$ denote its
variations, _e.g_. for an image in the Colored MNIST dataset [2], its semantic
attribute $s_{x}$ signifies “digit”, while the variation attributes $v_{x}$ is
“color”. There exists a direct product decomposition of attributes that could
control each sample $x$’s generation: $s_{x}\times v_{x}$, _e.g_. generating a
digit “5” with “green” color (see examples in Figure 1). Thus, this
decomposition implies a disentangled representation learning,
$\phi_{s}\times\phi_{v}$, that maps image pixels to a representation space.
Here, $\mathbf{s}_{x}:=\phi_{s}(x)$ captures the semantic aspects, and
$\mathbf{v}_{x}:=\phi_{v}(x)$ encapsulates the variations of sample $x$.

The disentanglement principle is inherent in a desirable property. Modifying
attributes is essentially akin to directly altering representations in the
feature space. For instance, consider the scenario illustrated in Figure 1,
where transforming a digit “5” from being “green” to “purple” in the
attributes space corresponds to the alteration of “5” $\times$ “green” to “5”
$\times$ “purple”. This change can be readily accomplished by substituting the
variation representation $\phi_{v}(x)$. Thus, through the lens of the
disentanglement principle, we highlight the potential to diversify variations
using representation-level data augmentation.

Invariance Principle. The ideal UDG model refines domain-invariant
representations that causally determine the intrinsic attributes, regardless
of changes in domains.

In other words, the domain-invariant feature extractor $\phi_{s}$ should
exclude any domain-specific features identifiable by a domain classifier. More
formally, let us consider $x_{i}\in\mathcal{I}_{d_{1}}$ and
$x_{j}\in\mathcal{I}_{d_{2}}$. For these cases, the support of semantic
representations for domain $d_{1}$ should align entirely with that for domain
$d_{2}$, implying $\text{Sup}(\phi_{s}(x_{i}))=\text{Sup}(\phi_{s}(x_{j}))$,
for all $i,j$. Here $\text{Sup}(\cdot)$ denotes the support of distribution.

Failing to adhere to the invariance principle can lead to entangled
representation learning, which ultimately undermines the ability for effective
generalization. As depicted in Figure 2, the widely used self-supervised
learner MAE tends to learn representation spaces that incompletely overlap
across domains. This observation underscores the idea that self-supervised
learners tend to faithfully capture distinctions among various domain data by
indiscriminately extracting both semantic and variation features.

Discussion. These two principles play a crucial role in acquiring a robust
semantic extractor $\phi_{s}$ in UDG. Through the disentangled representations
learning, even when testing a new sample from an unseen domain, the domain-
invariant semantics of the sample have already been learned as features during
training. As a result, the feature extractor trained on known domains remains
applicable. Moreover, the new combination of semantics and variations can
reshape the existing sample distribution, consequently boosting the model’s
generalization capability through representation-level data augmentation.

##  3 Methodology

Figure 3: Framework of our proposed DisMAE. DisMAE develops an asymmetric
dual-branch architecture, with the upper main branch distilling the domain-
invariant semantics, along with the lightweight branch extracting the domain-
specific variations. A domain classifier is integrated to quantify the degree
of domain-specific information embedded within the semantic encoder, thereby
monitoring its acquisition of domain-invariant knowledge. Note that the domain
classifier is updated while freezing backbones and is only used for generating
adaptive weights.

We now introduce our Disentangled Masked Autoencoder (DisMAE), designed to
derive decomposed high-level representations for UDG, building on the scalable
self-supervised MAE framework [23]. Grounded in disentanglement and invariance
principles, our approach integrates reconstruction loss and adaptive
contrastive loss in a collaborative manner to guide the learning process. An
overview of DisMAE’s framework is provided in Figure 3.

###  3.1 DisMAE

Considering a multi-domain reconstruction task with the training set
$\mathcal{D}_{\text{train}}=\\{(x_{i},d_{i})\\}_{i=1}^{N}$, where
$x_{i}\in\mathcal{I}_{d_{i}}$ denotes the input image, $d_{i}$ signifies the
corresponding domain category, and $N$ represents the number of training data.
DisMAE develops an asymmetric dual-branch architecture, visualized in Figure
3, with a main branch mapping the input image into the semantic feature space
_i.e_. semantic encoder $\phi_{s}(\cdot)$, along with a lightweight branch
that extracts the input sample’s variation attributes _i.e_. variations
encoder $\phi_{v}(\cdot)$. In alignment with MAE [23], we denote the masking
mechanism as the function $\mathcal{M}$ and the input visible patches as
$\mathcal{M}(x_{i})$ for the original image $x_{i}$. Our approach utilizes two
transformer-based encoders [16] to intensify the domain-invariant semantics
and domain-specific variations as representations:

$(\mathbf{s}_{i},\mathbf{v}_{i})=(\phi_{s}(\mathcal{M}(x_{i})),\phi_{v}(\mathcal{M}(x_{i}))).$
(1)

Consequently, the domain-invariant semantic representation $\mathbf{s}_{i}$
and domain-specific variation representation $\mathbf{v}_{i}$ are to estimate
the attributes $s_{x_{i}}$ and $v_{x_{i}}$ in feature space, respectively. We
denote the [cls] patch embedding of $\mathbf{s}_{i}$ and $\mathbf{v}_{i}$ as
$\mathbf{s}_{i}^{0}$ and $\mathbf{v}_{i}^{0}$, where
$\mathbf{s}_{i}^{0},\mathbf{v}_{i}^{0}\in\mathbb{R}^{B\times H}$ and $B$ is
the batch size and $H$ is the hidden size. Their combination can parameterize
image $x_{i}$ as:

$\mathbf{x}_{ii}=\mathbf{s}_{i}||\mathbf{v}_{i}^{0},$ (2)

where $||$ symbolizes the concatenation operation. Here, we utilize
$\mathbf{v}_{i}^{0}$ since we only need overall information about variation
features.

In addition to the encoders, we utilize a transformer-based decoder, denoted
as $g(\cdot,\cdot)$, mirroring the architecture of the MAE decoder. This
enables the reconstruction
$x_{ii}^{\prime}=g(\mathbf{s}_{i},\mathbf{v}_{i}^{0})$ of $x_{i}$.
Importantly, the decoder is operational only during training for
reconstruction. During the UDG task’s fine-tuning phase, solely the main
branch, namely the semantic encoder $\phi_{s}$, is deployed.

For the effective optimization of both encoders and the decoder, the prevalent
reconstruction learning strategy is employed. Specifically, the risk function
measures the quality of reconstructed pixel values, which can be formulated as
the mean squared error (MSE) or $\gamma$-constrained reconstruction loss [58]
between the masked patches and the corresponding masked patches for the
original image $x_{i}$. Here we apply $\gamma$-constrained reconstruction
loss, which is defined as:

$\mathbf{\mathop{\mathcal{L}}}_{\text{rec}}=\frac{1}{N}\sum_{i=1}^{N}\max\\{||x_{i}-g(\mathbf{s}_{i},\mathbf{v}_{i}^{0})||_{l_{2}}-\gamma,\,0\\},$
(3)

where $\gamma>0$ serves as a hyperparameter, indicating the bound of the
allowable reconstruction error. However, solely minimizing the risks over the
original data distribution fails to model domain-invariant information and
suffers from poor domain generalization.

###  3.2 Implementation of Two Principles

To bring forth better generalizations w.r.t. domain shift, we advocate for
disentanglement and invariance principles. To parameterize the disentanglement
principle, we devise an augmentation operator on semantic and variation
representations, which preserves the estimated domain-invariant intrinsic
attributes but intervenes in the estimated domain-specific information.
Formally, the operator samples in-batch variation representations
$\mathbf{v}_{j}^{0}$ from distinct sample $x_{j}$ to replace
$\mathbf{v}_{i}^{0}$ of $x_{i}$ and later combine it with $\mathbf{s}_{i}$ to
generate the augmented image $x_{ij}^{\prime}$. The augmentation process is:

$\mathbf{x}_{ij}=\mathbf{s}_{i}||\mathbf{v}_{j}^{0},\quad
x_{ij}^{\prime}=g(\mathbf{s}_{i},\mathbf{v}_{j}^{0}).$ (4)

Having established the augmented representations, we enforce the reconstructed
sample with its original semantics and variations closer to the input sample
while concurrently pushing the reconstructed samples possessing distinct
intra-domain variations away from it. This can be captured by a contrastive
loss between the original image $x_{i}$ and its reconstruction
$x_{ii}^{\prime}$:

$l(x_{i},x_{ii}^{\prime})=-\log\frac{\exp(s(x_{i},x_{ii}^{\prime})/\tau)}{\sum_{j\in\mathcal{I}_{d_{i}}\cup\\{i\\}}\exp(s(x_{i},x_{ij}^{\prime})/\tau)},$
(5)

where $\tau$ is the temperature hyperparameter in the contrastive loss [35],
and $s(\cdot,\cdot)$ is a similarity function quantifying the distance between
the reconstructed and original images on the masked patches. For this, we
adopt the negative $\gamma$-constrained reconstruction loss
$s(x_{i},x_{ii}^{\prime})=-\max\\{||x_{i}-x_{ii}^{\prime}||_{l_{2}}-\gamma,\,0\\}$
as the similarity metric. We emphasize that due to the strong heterogeneity of
different domain samples, intra-domain negatives $j\in\mathcal{I}_{d_{i}}$ are
exclusively selected in equation (5).

To further refine our approach, we instantiate the invariance principle,
enabling DisMAE to learn domain-invariant representations that remain
indistinguishable by the domain classifier $f(\cdot)$. We start by determining
the degree of domain-specific information in the semantic representations
$\mathbf{s}_{i}^{0}$ of image $x_{i}$ by employing a two-layer multilayer
perception (MLP) as the domain classifier, with the cross-entropy loss serving
as the objective function:

$p(x_{i}\in\mathcal{I}_{d_{i}}|\mathbf{s}_{i}^{0})=\text{Softmax}(d_{i},\hat{d_{i}}=f(\mathbf{s}_{i}^{0})).$
(6)

Drawing inspiration from Inverse Propensity Weighting (IPW) in causal
inference [26, 36], we multiply the inverse of this degree to reweight the
loss, leading us to our adaptive contrastive loss formulation:

$\mathbf{\mathop{\mathcal{L}}}_{\text{con}}=\sum_{i=1}^{N}\frac{1}{p(x_{i}\in\mathcal{I}_{d_{i}}|\mathbf{s}_{i}^{0})}\cdot
l(x_{i},x_{ii}^{\prime}).$ (7)

This adaptive contrastive loss compels DisMAE to prioritize the refinement of
domain-invariant semantic encoder learning, simultaneously mitigating the
impact stemming from biased representation learning.

Overall, we can aggregate all the foregoing risks and attain the final
objective of DisMAE in the UDG task:

$\mathbf{\mathop{\mathcal{L}}}_{\text{UDG}}=\mathbb{E}_{(x_{i},d_{i})\in\mathcal{D}_{\text{train}}}(\mathbf{\mathop{\mathcal{L}}}_{\text{rec}}+\lambda_{1}\mathbf{\mathop{\mathcal{L}}}_{\text{con}}),$
(8)

with $\lambda_{1}$ acting as the hyperparameter to control the strength of
disentanglement and invariance principles. In the inference phase, we only use
the domain-invariant semantic representations $\mathbf{s}_{x}^{0}$ to make
predictions, shielding them from the influence of domain shifts.

In the context of domain generalization tasks, the training set is formulated
as $\mathcal{D}_{\text{train}}=\\{(x_{i},y_{i},d_{i})\\}_{i=1}^{N}$, where
$y_{i}$ refers to the class label for image $x_{i}$ within domain $d_{i}$. To
adapt DisMAE for domain generalization tasks, it is a straightforward process
of introducing an additional cross-entropy loss that utilizes the supervision
signal $y_{i}$. The overall objective function for this adaptation is
formulated as:

$\mathbf{\mathop{\mathcal{L}}}_{\text{DG}}=\mathbb{E}_{(x_{i},y_{i},d_{i})\in\mathcal{D}_{\text{train}}}(\mathbf{\mathop{\mathcal{L}}}_{\text{rec}}+\lambda_{1}\mathbf{\mathop{\mathcal{L}}}_{\text{con}}+\lambda_{2}\text{CE}(y_{i},\hat{y_{i}}(\mathbf{s}_{i}^{0}))),$
(9)

where $\lambda_{1}$ and $\lambda_{2}$ are hyperparameters.

##  4 Experiments

We aim to answer the following research questions:

  * $\bullet$

RQ1: How does DisMAE perform compared with prevailing UDG and DG approaches?

  * $\bullet$

RQ2: Does DisMAE successfully derive disentangled representations and domain-
invariant features in UDG scenarios?

  * $\bullet$

RQ3: What are the impacts of the components on our DisMAE?

Datasets. For a comprehensive comparison, we evaluate DisMAE across both
unsupervised domain generalization (UDG) and domain generalization (DG) tasks.
For UDG, our evaluation is on DomainNet [37], a dataset with 586,575 images
spanning 345 object classes across six domains: Real, Painting, Sketch,
Clipart, Infograph, and Quickdraw. Our DG evaluations center on two benchmark
datasets: PACS [28] and VLCS [18]. PACS contains 9,991 images over 7 classes
and 4 domains: Art, Cartoons, Photos, and Sketches. VLCS includes 10,729
images across 5 classes from four domains: Caltech101, LabelMe, SUN09, and
VOC2007.

Implementation Details. Contrary to the standard use of ResNet-18 [24], we
employ the more powerful self-supervised learner ViT-B/16 [23] as our default
backbone (for experiments in section 4.3, ViT-Tiny/16 serves as the backbone).
We adopt a learning rate of 1e-4, a weight decay of 0.05, and a batch size of
$N_{d}\times 96$, where $N_{d}$ denotes the number of domains in the training
dataset. For UDG tasks, in alignment with the all correlated setting from
DARLING [60], our model is pretrained on DomainBed, bypassing the ImageNet
dataset. Conversely, for DG tasks, we initialize our backbone using ImageNet
pre-training, as prescribed by DomainBed [21]. Detailed information on
implementation and hyperparameter settings can be found in Appendix 0.C.7.

Baselines. For UDG tasks, we benchmark DisMAE against notable contrastive
learning approaches (MoCo V2 [14], BYOL [20], DARLING [60]) and generative-
based methods (MAE [23], CycleMAE [53]). For DG tasks, our comparisons
encompass diverse learning strategies: from the vanilla (ERM [44]),
distributional robust optimization (GroupDRO [40]), data augmentation-based
(Mixup [52]), to domain-invariant learning (IRM [2], MMD [29]), variance
optimization (VREx [27], Fishr [39]), and disentangled representation learning
(DDG [58]). Related work and detailed implementation are available in Section
5 and Appendix 0.C.1.

###  4.1 Overall Performance Comparison (RQ1)

####  4.1.1 Evaluations on UDG.

|  |  | Label Fraction 1% (Linear evaluation) | Label Fraction 5% (Linear evaluation)
---|---|---|---|---
Methods | Clip | Info | Quick | Overall | Avg. | Clip | Info | Quick | Overall | Avg.
ERM [44] | 9.38 | 9.70 | 6.64 | 8.04 | 8.57 | 11.27 | 8.58 | 6.90 | 8.25 | 8.92
MoCo V2 [14] | 9.76 | 6.53 | 5.24 | 6.51 | 7.18 | 11.44 | 9.20 | 4.95 | 7.42 | 8.53
BYOL [20] | 17.93 | 8.03 | 5.05 | 8.47 | 10.34 | 24.26 | 8.88 | 4.66 | 9.78 | 12.60
MAE [23] | 19.69 | 8.85 | 12.49 | 12.96 | 13.68 | 29.12 | 8.88 | 13.71 | 15.52 | 17.24
DARLING [60] | 13.77 | 6.33 | 7.24 | 8.41 | 9.18 | 19.02 | 8.49 | 9.59 | 11.19 | 12.36
CycleMAE [53] | 11.68 | 6.82 | 8.06 | 8.46 | 8.85 | 22.06 | 10.46 | 12.07 | 13.66 | 14.86
DisMAE (Ours) | 24.55 | 9.18 | 11.75 | 13.64 | 15.16 | 31.70 | 10.14 | 19.06 | 19.19 | 20.30
|  |  | Label Fraction 10% (Full finetuning) | Label Fraction 100% (Full finetuning)
Methods | Clip | Info | Quick | Overall | Avg. | Clip | Info | Quick | Overall | Avg.
ERM [44] | 27.57 | 13.32 | 8.29 | 13.60 | 16.39 | 48.61 | 18.37 | 20.06 | 25.38 | 29.01
MoCo V2 [14] | 26.51 | 11.68 | 10.62 | 14.12 | 16.27 | 57.16 | 16.44 | 29.40 | 31.50 | 34.33
BYOL [20] | 31.33 | 11.55 | 9.51 | 14.48 | 17.46 | 56.62 | 12.77 | 26.90 | 29.08 | 32.10
MAE [23] | 41.76 | 12.91 | 18.98 | 21.94 | 24.55 | 63.90 | 19.34 | 33.00 | 35.54 | 38.75
DARLING [60] | 30.68 | 11.66 | 13.93 | 16.69 | 18.75 | 55.20 | 17.04 | 33.30 | 33.22 | 35.18
CycleMAE [53] | 33.84 | 12.06 | 15.41 | 18.23 | 20.43 | 56.10 | 19.15 | 28.60 | 31.60 | 34.62
DisMAE (Ours) | 49.06 | 15.82 | 22.68 | 26.16 | 29.19 | 72.99 | 23.40 | 38.20 | 41.22 | 44.86

Table 1: Unsupervised domain generalization results on DomainNet with
Painting, Real, and Sketch serving as training domains, while Clipart,
Infograph, and Quickart for testing. Each model undergoes unsupervised pre-
training prior to fine-tuning on labeled data. Overall and Avg. are the
overall test data accuracy and the arithmetic mean of individual domain
accuracy respectively. Note that these metrics differ, given the varying sizes
of the test domains. Bold=best, underline=second best.

Setting. For a balanced comparison, we strictly follow the all-correlated
setting outlined by DARLING [60]. We select 20 classes from DomainNet’s [37]
345 categories for both training and testing phases, using Painting, Real, and
Sketch as source domains, while Clipart, Infograph, and Quickdraw act as
target domains, and vice versa. Specifically, the UDG task is executed in
three stages: 1\. Unsupervised training on the source domains. 2\. Depending
on the labeled data proportion from the source domain, we either fit a linear
classifier on the frozen semantic encoder or fully fine-tune the network. In
particular, for labeled data proportions under 10% of the source domain’s
total training set, we opt for fitting a linear classifier. If it exceeds 10%,
the entire network undergoes fine-tuning. 3\. Model evaluation on the target
domains.

Results. Tables 1 and 7 showcase a performance comparison for the UDG
evaluation using the ViT-B/16 backbone. The best-performing methods per test
are bold, while the second-best methods are underlined. We observe that:

  * •

DisMAE consistently and significantly outperforms all baseline models in terms
of both average and overall accuracy. In Table 1, DisMAE achieves gains of
1.48%, 3.04%, 4.65%, and 6.11% for average accuracy across the 1%, 5%, 10%,
and 100% fraction settings, respectively. The robustness of DisMAE is
attributed to its proficiency in distilling the invariant semantic attributes,
regardless of variations in domain-specific features.

  * •

Entangled baselines exhibit instability when faced with domain shifts.
Conventional contrastive methods display limited generalization capability,
revealing vulnerability in handling unknown target domains. These methods rely
on data augmentations to form positive and negative pairs, lacking the
explicit ability to bridge domain gaps by effectively pulling positive pairs
together and pushing negative pairs apart. In the case of DARLING, negative
samples are generated for each queue based on the dissimilarity between
diverse domain samples. However, the cross-domain pairs constructed in this
way are noisy in some cases, ultimately leading to suboptimal performance.
Generative-based methods such as CycleMAE employ cycle reconstruction tasks to
construct cross-domain pairs while suffering from an implicit domain-variant
encoder, resulting in performance deterioration. In contrast, leveraging the
strength of a domain-invariant extractor, our DisMAE significantly enhances
performance by a substantial margin.

####  4.1.2 Evaluations on DG.

Methods | C | L | S | V | Avg.
---|---|---|---|---|---
ERM [44] | 95.47 | 65.74 | 76.83 | 77.07 | 78.78
MMD [29] | 94.61 | 65.85 | 73.70 | 77.26 | 77.86
GroupDRO[40] | 95.49 | 65.80 | 76.28 | 77.37 | 78.74
IRM [2] | 96.26 | 65.30 | 73.25 | 76.01 | 77.71
Mixup[52] | 95.38 | 65.74 | 76.83 | 77.07 | 78.76
VREx[27] | 96.94 | 63.67 | 73.34 | 76.10 | 77.51
Fishr[39] | 96.17 | 65.19 | 74.77 | 78.06 | 78.55
DDG[58] | 96.97 | 64.21 | 71.76 | 76.58 | 77.38
RIDG[10] | 96.64 | 66.16 | 77.57 | 76.68 | 79.26
DisMAE (Ours) | 97.35 | 66.96 | 77.10 | 80.12 | 80.38

Table 2: Domain generalization results on VLCS.

Setting. We strictly follow DomainBed [21] evaluation protocols, utilizing
training-domain validation set. One domain serves as the target (test), while
the remaining acts as source (training) domains. Training is done with 80% of
the source data, and validation uses the remaining 20%. We implement a
hyperparameter random search, following the DomainBed [21] guidelines. Optimal
hyperparameters are chosen based on validation set performance for each test,
and the model is subsequently evaluated on the target domain. To mitigate the
influence of randomness, experiments are conducted thrice with distinct seeds,
and we report the average accuracy from these runs.

Results. As illustrated in Tables 2 and 8, surprisingly, DisMAE demonstrates
steady superiority over all state-of-the-art DG baselines concerning the
average accuracy on VLCS and PACS. Specifically, compared to the state-of-the-
art baseline, DisMAE achieves substantial gains of 1.12% and 0.50% in average
accuracy for VLCS and PACS, respectively. In contrast, baselines perform
unstablely across various sub-tasks. Benefiting from disentangled and domain-
invariant representations, DisMAE successfully achieves more robust
performance.

###  4.2 Discussion about Two Principles (RQ2)

(a) DisMAE semantic representations $\mathbf{s}_{x}^{0}$ (b) DisMAE variation
representations $\mathbf{v}_{x}^{0}$

Figure 4: t-SNE visualization of DisMAE semantic and variation representations
on DomainNet, with points color-distinguished by their domain labels. (4(a))
DisMAE semantic representations are mixed and interspersed regardless of
domain labels, showcasing the ability of the semantic encoder to discern
domain-invariant features. (4(b)) Samples from different domains reside on
distinct manifolds, highlighting the variation encoder’s capability to extract
domain-specific features.

To visualize the latent representation space and evaluate the effectiveness of
two principles in DisMAE, we conduct a comprehensive set of experiments in
DomainNet, employing Painting, Real, and Sketch domains as training domains
and Clipart, Infograph, and Quickdraw for testing.

  * •

Reconstruction Visualization. Figures 1 and more examples in Appendix 0.C.6
demonstrate DisMAE’s capability in image-level disentanglement.

  * •

Domain-Invariance Assessment. Figure 5 emphasizes the domain-invariant
proficiency of our semantic encoder, as evidenced by the prediction scores of
$\mathbf{s}_{x}^{0}$.

  * •

Representation-level Visualization. Through t-SNE visualizations in Figures 2
and 4, we validate that DisMAE adheres to invariance and disentanglement
principles at the representation level.

The specific analysis is detailed below:

Disentanglement principle: Semantic and variational features learned by DisMAE
are fully disentangled. From an image perspective, as illustrated in Figure 1,
DisMAE differentiates between the foreground and background of an image,
exemplified by its capability to transform the background to a blue sky or a
red hue without affecting the primary subject, the flower. Remarkably, DisMAE
can discern domain styles and fuse domain-specific elements across them — a
notable instance is superimposing the sun from a sketch onto a painting. Such
disentanglement ability endows DisMAE with the flexibility to generate
controllable images by manipulating semantic and variation factors through
swapping. From a feature space viewpoint, as evidenced in Figure 4, closely
situated semantic representations display shared content features. In
contrast, in the space extracted by the variation encoder, neighboring data
points share analogous attributes like color, texture, and background. It is
worth noting that Figure 4(b) shows some overlap between the painting and real
domains. This phenomenon may arise due to the high similarity of images in
these two domains.

Figure 5: Study of invariance principle. Illustration of prediction scores
$p(x_{i}\in\mathcal{I}_{\text{Sketch}}|\mathbf{s}_{i}^{0})$ estimated by
domain classifier throughout training. The dashed line represents the Oracle
score, illustrating a random guess w.r.t. domain category.

Invariance Principle: Representations learned by semantic encoder are domain-
invariant. As depicted in Figure 5, the initial prediction scores of Sketch
for each domain (opaque lines), denoted as
$p(x_{i}\in\mathcal{I}_{\text{Sketch}}|\mathbf{s}_{i}^{0})$, exhibit
randomness at the beginning. With the advancement of training, these
prediction scores elevate for the Sketch domain (opaque green line),
demonstrating that Sketch-specific features are extracted by the semantic
encoder at first. As training progresses, the prediction scores gradually
converge to 0.33 (domain-agnostic score), indicating domain-variant
information excluded from our main encoder and the efficacy of our invariance
principle. Conversely, in scenarios where the reweighting term is omitted, the
prediction score for the Sketch domain (transparent green line) fails to
converge to 0.33, revealing a domain-variant encoder. Overall, the figure
further validates the impact of our adaptive reweighting term and the goal of
the invariance principle. For more analysis, see Appendix 0.C.3.

###  4.3 Study on DisMAE (RQ3)

| Clip | Info | Quick | Overall | Avg.
---|---|---|---|---|---
MAE | 61.04 | 19.15 | 27.50 | 32.02 | 35.90
Inter-domain neg. | 61.82 | 17.60 | 31.60 | 33.91 | 37.01
w/o weights | 64.16 | 17.60 | 34.40 | 35.86 | 38.72
Random weights | 64.68 | 19.15 | 31.10 | 34.65 | 38.31
Reverse weights | 60.52 | 18.57 | 29.90 | 33.02 | 36.33
DisMAE | 63.12 | 19.16 | 35.90 | 36.89 | 39.39

Table 3: Effect of adaptive contrastive loss
$\mathbf{\mathop{\mathcal{L}}}_{\text{con}}$ on DisMAE.

To further evaluate each component of DisMAE, and its potential to enhance
generalization ability, we conduct a comprehensive analysis using DomainBed in
the UDG setting. This analysis contains three-fold: an ablation study of the
adaptive contrastive loss components, the impact of decoder depth, and the
effect of mask ratios. During the evaluation, we employ the Painting, Real,
and Sketch domains as training domains and evaluate the models’ performance on
Clipart, Infograph, and Cartoon domains. Specifically, training is executed on
ViT-Tiny/16 across 500 epochs, leveraging the full fraction of labeled data
for fine-tuning. For thorough results and analysis, readers are referred to
Appendix 0.C.4 due to space constraints.

Effect of adaptive contrastive loss. Table 3 shows that leveraging intra-
domain negative samples significantly enhances generalization. The
meticulously designed reweighting term aids in achieving the domain-invariance
principle.

##  5 Related work

DisMAE is related to the literature on self-supervised learning (SSL), domain
generalization (DG), and unsupervised domain generalization (UDG).

Self-supervised learning (SSL) employs a range of pretext tasks to derive
meaningful representations from large amounts of unlabeled data, aiming to
enhance performance in downstream tasks. Contemporary SSL methods can roughly
fall into two distinct research lines: contrastive approaches and generative
models. Recent advancements in contrastive methods [14, 12, 20, 9, 15, 57, 5]
have excelled in instance discrimination. They strategically draw two
augmented versions of an image closer in the feature space, recognized as
positives, while simultaneously distancing them from negatives. While
contrastive SSL methods have witnessed significant advancements, their
efficacy often hinges on specific factors like high-quality data augmentation
techniques and complicated strategies for training stabilization, such as
well-designed negative sampling. On the contrary, generative SSL can avoid
these dependencies. These approaches primarily focus on the reconstruction of
intrinsic features and information. In the field of computer vision (CV),
inspired by the introduction of ViT [16], masked image modeling (MIM) has
attracted the huge attention of the research community, such as bidirectional
encoder representation from image transformer (BEiT) [4], masked autoencoder
(MAE) [23], context autoencoder (CAE) [13], a simple frame for MIM (SimMIM)
[51], image generating pretraining (iGPT) [11]. Diverging from the SSL methods
previously discussed, which concentrate on single-domain learning, our study
pivots towards a more novel and pragmatic task: pretraining across multiple
domains.

Domain generalization (DG) aims to learn semantic representations that remain
consistent across diverse domains, enabling models to generalize effectively
to unseen domains using labeled data. Common approaches for solving the DG
task can be broadly categorized as follows: minimizing the difference among
source domains by invariant learning [2, 27, 1], adversarial learning [31, 42,
61], or feature alignment [39, 17, 34, 45]; modifying the inputs to assist in
learning general representations [41, 43, 25, 30]; disentangling the features
into domain-shared or domain-specific parts for better generalization [58, 6,
8, 3, 49].

Unsupervised domain generalization (UDG) has been proposed recently as a more
important and challenging task centered on pretraining with unlabeled source
domains [60]. Research in UDG predominantly bifurcates into two methodologies:
contrastive approaches and generative models. In the realm of contrastive
approaches, DARLING [60] pioneered the UDG framework, focusing on domain-aware
representation learning with a novel contrastive loss. Another notable
contribution is [22], which innovatively conceptualized a bridge domain to
unify all source domains. Additionally, DN2A [33] advanced this line of work
by implementing strong augmentations and reducing intra-domain connectivity.
Turning to generative models, DiMAE [54] was instrumental in introducing
cross-domain reconstruction tasks, where input images are augmented using
style noise from varying domains. Building on this, CycleMAE [53] innovated
with a cycle-structured cross-domain reconstruction task, in the absence of
paired images. A novel attempt in this domain is our proposed DisMAE, which,
to our knowledge, is the first to incorporate a disentangled generative
pretraining approach within UDG.

##  6 Conclusion

Despite the great success of domain generalization tasks, the challenge of
unsupervised domain generalization remains relatively underexplored. In this
work, we devised a principled disentangling approach, the disentangled mask
autoencoder (DisMAE), to learn domain-invariant features in an unsupervised
manner. However, DisMAE does present a limitation meriting further
exploration: its reconstruction accuracy is closely tied to the MAE backbone,
which occasionally struggles to generate high-fidelity images (See failure
examples in Appendix 0.C.5).

## Acknowledgements

This research is supported by the National Natural Science Foundation of China
(92270114) and the advanced computing resources provided by the Supercomputing
Center of the USTC.

## References

  * [1] Ahuja, K., Caballero, E., Zhang, D., Gagnon-Audet, J., Bengio, Y., Mitliagkas, I., Rish, I.: Invariance principle meets information bottleneck for out-of-distribution generalization. In: NeurIPS (2021)
  * [2] Arjovsky, M., Bottou, L., Gulrajani, I., Lopez-Paz, D.: Invariant risk minimization. CoRR abs/1907.02893 (2019)
  * [3] Bai, H., Sun, R., Hong, L., Zhou, F., Ye, N., Ye, H., Chan, S.G., Li, Z.: Decaug: Out-of-distribution generalization via decomposed feature representation and semantic augmentation. In: AAAI (2021)
  * [4] Bao, H., Dong, L., Piao, S., Wei, F.: Beit: BERT pre-training of image transformers. In: ICLR (2022)
  * [5] Bardes, A., Ponce, J., LeCun, Y.: Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In: ICLR (2022)
  * [6] Bui, M., Tran, T., Tran, A., Phung, D.Q.: Exploiting domain-specific features to enhance domain generalization. In: NeurIPS (2021)
  * [7] Cabannes, V., Kiani, B.T., Balestriero, R., LeCun, Y., Bietti, A.: The SSL interplay: Augmentations, inductive bias, and generalization. In: ICML (2023)
  * [8] Cai, R., Li, Z., Wei, P., Qiao, J., Zhang, K., Hao, Z.: Learning disentangled semantic representation for domain adaptation. In: IJCAI (2019)
  * [9] Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. In: NeurIPS (2020)
  * [10] Chen, L., Zhang, Y., Song, Y., van den Hengel, A., Liu, L.: Domain generalization via rationale invariance. In: ICCV. pp. 1751–1760. IEEE (2023)
  * [11] Chen, M., Radford, A., Child, R., Wu, J., Jun, H., Luan, D., Sutskever, I.: Generative pretraining from pixels. In: ICML (2020)
  * [12] Chen, T., Kornblith, S., Norouzi, M., Hinton, G.E.: A simple framework for contrastive learning of visual representations. In: ICML (2020)
  * [13] Chen, X., Ding, M., Wang, X., Xin, Y., Mo, S., Wang, Y., Han, S., Luo, P., Zeng, G., Wang, J.: Context autoencoder for self-supervised representation learning. CoRR abs/2202.03026 (2022)
  * [14] Chen, X., Fan, H., Girshick, R.B., He, K.: Improved baselines with momentum contrastive learning. CoRR abs/2003.04297 (2020)
  * [15] Chen, X., He, K.: Exploring simple siamese representation learning. In: CVPR (2021)
  * [16] Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N.: An image is worth 16x16 words: Transformers for image recognition at scale. In: ICLR (2021)
  * [17] Du, Y., Wang, J., Feng, W., Pan, S.J., Qin, T., Xu, R., Wang, C.: Adarnn: Adaptive learning and forecasting of time series. In: CIKM (2021)
  * [18] Fang, C., Xu, Y., Rockmore, D.N.: Unbiased metric learning: On the utilization of multiple datasets and web images for softening bias. In: ICCV. pp. 1657–1664. IEEE Computer Society (2013)
  * [19] Gholami, B., El-Khamy, M., Song, K.: Latent feature disentanglement for visual domain generalization. IEEE Trans. Image Process. 32, 5751–5763 (2023)
  * [20] Grill, J., Strub, F., Altché, F., Tallec, C., Richemond, P.H., Buchatskaya, E., Doersch, C., Pires, B.Á., Guo, Z., Azar, M.G., Piot, B., Kavukcuoglu, K., Munos, R., Valko, M.: Bootstrap your own latent - A new approach to self-supervised learning. In: NeurIPS (2020)
  * [21] Gulrajani, I., Lopez-Paz, D.: In search of lost domain generalization. In: ICLR. OpenReview.net (2021)
  * [22] Harary, S., Schwartz, E., Arbelle, A., Staar, P.W.J., Hussein, S.A., Amrani, E., Herzig, R., Alfassy, A., Giryes, R., Kuehne, H., Katabi, D., Saenko, K., Feris, R., Karlinsky, L.: Unsupervised domain generalization by learning a bridge across domains. In: CVPR (2022)
  * [23] He, K., Chen, X., Xie, S., Li, Y., Dollár, P., Girshick, R.B.: Masked autoencoders are scalable vision learners. In: CVPR (2022)
  * [24] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR. pp. 770–778. IEEE Computer Society (2016)
  * [25] Huang, J., Guan, D., Xiao, A., Lu, S.: FSDR: frequency space domain randomization for domain generalization. In: CVPR. pp. 6891–6902. Computer Vision Foundation / IEEE (2021)
  * [26] Jung, Y., Tian, J., Bareinboim, E.: Learning causal effects via weighted empirical risk minimization. In: NeurIPS (2020)
  * [27] Krueger, D., Caballero, E., Jacobsen, J., Zhang, A., Binas, J., Zhang, D., Priol, R.L., Courville, A.C.: Out-of-distribution generalization via risk extrapolation (rex). In: ICML (2021)
  * [28] Li, D., Yang, Y., Song, Y., Hospedales, T.M.: Deeper, broader and artier domain generalization. In: ICCV. pp. 5543–5551. IEEE Computer Society (2017)
  * [29] Li, H., Pan, S.J., Wang, S., Kot, A.C.: Domain generalization with adversarial feature learning. In: CVPR (2018)
  * [30] Li, P., Li, D., Li, W., Gong, S., Fu, Y., Hospedales, T.M.: A simple feature augmentation for domain generalization. In: ICCV (2021)
  * [31] Li, Y., Tian, X., Gong, M., Liu, Y., Liu, T., Zhang, K., Tao, D.: Deep domain generalization via conditional invariant adversarial networks. In: ECCV (15) (2018)
  * [32] Lin, C., Yuan, Z., Zhao, S., Sun, P., Wang, C., Cai, J.: Domain-invariant disentangled network for generalizable object detection. In: ICCV. pp. 8751–8760. IEEE (2021)
  * [33] Liu, Y., Wang, Y., Chen, Y., Dai, W., Li, C., Zou, J., Xiong, H.: Promoting semantic connectivity: Dual nearest neighbors contrastive learning for unsupervised domain generalization. In: CVPR (2023)
  * [34] Lu, W., Wang, J., Li, H., Chen, Y., Xie, X.: Domain-invariant feature exploration for domain generalization. Trans. Mach. Learn. Res. 2022 (2022)
  * [35] van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018)
  * [36] Pearl, J., Glymour, M., Jewell, N.P.: Causal inference in statistics: A primer. John Wiley & Sons (2016)
  * [37] Peng, X., Bai, Q., Xia, X., Huang, Z., Saenko, K., Wang, B.: Moment matching for multi-source domain adaptation. In: ICCV (2019)
  * [38] Peng, X., Huang, Z., Sun, X., Saenko, K.: Domain agnostic learning with disentangled representations. In: ICML. Proceedings of Machine Learning Research, vol. 97, pp. 5102–5112. PMLR (2019)
  * [39] Ramé, A., Dancette, C., Cord, M.: Fishr: Invariant gradient variances for out-of-distribution generalization. In: ICML. Proceedings of Machine Learning Research, vol. 162 (2022)
  * [40] Sagawa, S., Koh, P.W., Hashimoto, T.B., Liang, P.: Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. CoRR abs/1911.08731 (2019)
  * [41] Shankar, S., Piratla, V., Chakrabarti, S., Chaudhuri, S., Jyothi, P., Sarawagi, S.: Generalizing across domains via cross-gradient training. In: ICLR (Poster). OpenReview.net (2018)
  * [42] Shao, R., Lan, X., Li, J., Yuen, P.C.: Multi-adversarial discriminative deep domain generalization for face presentation attack detection. In: CVPR (2019)
  * [43] Shu, Y., Cao, Z., Wang, C., Wang, J., Long, M.: Open domain generalization with domain-augmented meta-learning. In: CVPR. pp. 9624–9633. Computer Vision Foundation / IEEE (2021)
  * [44] Vapnik, V.: An overview of statistical learning theory. IEEE Trans. Neural Networks 10(5), 988–999 (1999)
  * [45] Wald, Y., Feder, A., Greenfeld, D., Shalit, U.: On calibration and out-of-domain generalization. In: NeurIPS (2021)
  * [46] Wang, J., Lan, C., Liu, C., Ouyang, Y., Qin, T., Lu, W., Chen, Y., Zeng, W., Yu, P.S.: Generalizing to unseen domains: A survey on domain generalization. IEEE Trans. Knowl. Data Eng. 35(8), 8052–8072 (2023)
  * [47] Wang, T., Sun, Q., Pranata, S., Karlekar, J., Zhang, H.: Equivariance and invariance inductive bias for learning from insufficient data. In: ECCV (2022)
  * [48] Wang, T., Yue, Z., Huang, J., Sun, Q., Zhang, H.: Self-supervised learning disentangled group representation as feature. In: NeurIPS (2021)
  * [49] Wang, Y., Li, H., Cheng, H., Wen, B., Chau, L., Kot, A.C.: Variational disentanglement for domain generalization. Trans. Mach. Learn. Res. 2022 (2022)
  * [50] Wen, Z., Li, Y.: Toward understanding the feature learning process of self-supervised contrastive learning. In: ICML (2021)
  * [51] Xie, Z., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z., Dai, Q., Hu, H.: Simmim: a simple framework for masked image modeling. In: CVPR (2022)
  * [52] Yan, S., Song, H., Li, N., Zou, L., Ren, L.: Improve unsupervised domain adaptation with mixup training. CoRR abs/2001.00677 (2020)
  * [53] Yang, H., Li, X., Tang, S., Zhu, F., Wang, Y., Chen, M., Bai, L., Zhao, R., Ouyang, W.: Cycle-consistent masked autoencoder for unsupervised domain generalization. In: ICLR. OpenReview.net (2023)
  * [54] Yang, H., Tang, S., Chen, M., Wang, Y., Zhu, F., Bai, L., Zhao, R., Ouyang, W.: Domain invariant masked autoencoders for self-supervised learning from multi-domains. In: ECCV (31). Lecture Notes in Computer Science, vol. 13691, pp. 151–168. Springer (2022)
  * [55] Ye, H., Xie, C., Cai, T., Li, R., Li, Z., Wang, L.: Towards a theoretical framework of out-of-distribution generalization. In: NeurIPS (2021)
  * [56] Ye, N., Li, K., Bai, H., Yu, R., Hong, L., Zhou, F., Li, Z., Zhu, J.: Ood-bench: Quantifying and understanding two dimensions of out-of-distribution generalization. In: CVPR (2022)
  * [57] Zbontar, J., Jing, L., Misra, I., LeCun, Y., Deny, S.: Barlow twins: Self-supervised learning via redundancy reduction. In: ICML (2021)
  * [58] Zhang, H., Zhang, Y., Liu, W., Weller, A., Schölkopf, B., Xing, E.P.: Towards principled disentanglement for domain generalization. In: CVPR (2022)
  * [59] Zhang, X., He, Y., Xu, R., Yu, H., Shen, Z., Cui, P.: NICO++: towards better benchmarking for domain generalization. CoRR abs/2204.08040 (2022)
  * [60] Zhang, X., Zhou, L., Xu, R., Cui, P., Shen, Z., Liu, H.: Towards unsupervised domain generalization. In: CVPR. pp. 4900–4910. IEEE (2022)
  * [61] Zhao, S., Gong, M., Liu, T., Fu, H., Tao, D.: Domain generalization via entropy regularization. In: NeurIPS (2020)

##  Appendix 0.A Algorithm

Algorithm 1 depicts the detailed procedure of DisMAE.

Algorithm 1 DisMAE: Disentangling Masked Autoencoder for Unsupervised Domain
Generalization

Input: $\mathcal{D}_{S}=\\{(x_{1},d_{1}),...,(x_{n},d_{n})\\}$, maximum
adaptive training epochs $E_{ad}$, maximum training epoch $E$, adaptive
training intervals $T_{ad}$, batch size $B$, margin $\gamma$, coefficients
$\lambda_{1}$, current training epoch $e$

Initial: Parameters of DisMAE (i.e parameter $\theta_{s}$, $\theta_{v}$,
$\theta_{\mathcal{G}}$, and $\theta_{cls}$ for semantic encoder $\phi_{s}$,
variation encoder $\phi_{v}$, decoder $g$, and domain classifier $f_{cls}$ ),
$e\leftarrow 1$

repeat

Freeze parameters of the domain classifier $\theta_{1}$

Compute $l(x_{i},x_{ii}^{\prime})$ and
$p(x_{i}\in\mathcal{I}_{d_{i}}|\mathbf{s}_{i}^{0})$ with
$\theta_{s},\theta_{v},\theta_{\mathcal{G}},\theta_{1}$

Compute
$\mathbf{\mathop{\mathcal{L}}}_{\text{rec}}=\frac{1}{B}\sum_{i=1}^{B}\max\\{||x_{i}-g(\mathbf{s}_{i},\mathbf{v}_{i}^{0})||_{l_{2}}-\gamma,\,0\\}$

Compute
$\mathbf{\mathop{\mathcal{L}}}_{\text{con}}=\sum_{i=1}^{B}\frac{1}{p(x_{i}\in\mathcal{I}_{d_{i}}|\mathbf{s}_{i}^{0})}\cdot
l(x_{i},x_{ii}^{\prime})$

Compute
$\mathbf{\mathop{\mathcal{L}}}=\mathbf{\mathop{\mathcal{L}}}_{\text{rec}}+\lambda_{1}\mathbf{\mathop{\mathcal{L}}}_{\text{con}}$

Update $\theta_{s},\theta_{v},\theta_{\mathcal{G}}$ by minimizing
$\mathbf{\mathop{\mathcal{L}}}$

if $e\bmod T_{ad}==0$ and $e\leq E_{ad}$ then

Freeze parameters of backbones $\theta_{s}$, $\theta_{v}$ and
$\theta_{\mathcal{G}}$

Compute $\mathbf{\mathop{\mathcal{L}}}_{cls}=\sum_{i=1}^{B}\frac{1}{B}\cdot
CE(d_{i},\hat{d}_{i}=f_{cls}(\mathbf{s}_{i}^{0}))$

Update $\theta_{cls}$ by minimizing $\mathbf{\mathop{\mathcal{L}}}_{cls}$

end if

$e\leftarrow e+1$

until $e==E$

##  Appendix 0.B Discussion About Differences

We argue that DisMAE is novel and significantly different from prior studies
w.r.t. three aspects. 1) Scope. Transitioning to UDG is non-trivial. Previous
disentangled methods like DADA [38], DIDN [32], and DIR [19], while effective
in DG, struggle with unsupervised data due to their high dependence on class
labels to encapsulate semantic attributes. 2) Disentangled Targets. Without
class label guidance, achieving a domain-invariant semantic encoder is
challenging. Many UDG methods, such as DiMAE [54] and CycleMAE [53], can only
separate domain styles using multiple decoders but fall short in disentangling
domain-invariant semantics from variations. 3) Disentangle Strategy. DisMAE is
grounded in disentanglement and invariance principles, uniquely combining
adaptive contrastive loss with reconstruction loss collaboratively. The
adaptive contrastive loss, in particular, is designed by seamlessly leveraging
the domain classifier and intra-domain negative sampling. The differences are
summarized in Table 4.

|  DADA [38] |  DIDN [32] |  DIR [19] |  DiMAE [54] | DisMAE
---|---|---|---|---|---
Class Label | ✓ | ✓ | ✓ | ✗ | ✗
Disentanglement &Invariance Strategy |  Adversarial Training | Adversarial Training | Adversarial Training | Fourier Transform |  Adaptive Contrastive Learning
Disentangled Info | Semantics-Variations | - | Semantics-Variations | Domain Styles | Semantics-Variations
Architecture |  1 Encoder 1 Decoder |  2 Encoders 1 Decoder |  2 Encoders 1 Decoder 1 GAN |  1 Encoder $N_{d}$ Decoders |  2 Encoders 1 Decoder

Table 4: Comparison with previous works

##  Appendix 0.C Experiments

###  0.C.1 Experimental Settings

Baseline Hyperparameter Tuning. For a fair comparison, we uniformly substitute
the backbones of all baselines with the same ViT-B/16 and rerun the experiment
using UDG and DG open-source codebases. And we provide the default
hyperparameters for UDG baselines in Table 5. And the search distribution for
each hyperparameter in each DG baseline is detailed in Table 6.

| Default hyperparameters
---|---
MoCo V2 | lr=5e-4, BS=96, WD=0.05, K=65536, m=0.999, T=0.07
BYOL | lr=5e-4, BS=64, WD=0.05, m=0.996
MAE | lr=1e-3, BS=96, WD=0.00, mask ratio=0.75
DARLING | lr=5e-4, BS=96, WD=0.05, K=65536, m=0.995, T=0.07
CycleMAE | lr=7e-4, BS=96, WD=0.05, m=0.999, $\alpha=2$, $\beta=1$

Table 5: Hyperparameters for baselines in UDG. BS represents the batch size,
and WD denotes weight decay.

Condition | Parameter | Default value | Random distribution
---|---|---|---
ViT | learning rate | 0.00005 | $10^{Uniform(-5,-3.5)}$
batch size | 32 | $2^{Uniform(3,5.5)}$
generator learning rate | 0.00005 | $10^{Uniform(-5,-3.5)}$
discriminator learning rate | 0.00005 | $10^{Uniform(-5,-3.5)}$
MMD | gamma | 1 | $10^{Uniform(-1,1)}$
GroupDRO | eta | 0.01 | $10^{Uniform(-3,-1)}$
IRM | lambda | 100 | $10^{Uniform(-1,5)}$
iterations of penalty annealing | 500 | $10^{Uniform(0,4)}$
Mixup | alpha | 0.2 | $10^{Uniform(0,4)}$
VREx | lambda | 10 | $10^{Uniform(-1,5)}$
penalty anneal iters | 500 | $10^{Uniform(0,4)}$
Fishr | lambda | 1000 | $10^{Uniform(1,4)}$
penalty anneal iters | 1500 | $Uniform(0,5000)$
ema | 0.95 | $Uniform(0.90,0.99)$
DDG | recon w | 0.5 | $Random(0.1,0.2,0.5,1.0)$
recon x w | 0.5 | $Random(1,2,5,10)$
margin | 0.25 | $Random(0.10,0.25,0.5,0.75)$
recon xp w | 0.5 | $Random(1,2,5,10)$
recon xn w | 0.50 | $Random(0.10,0.25,0.5,0.75)$
max cyc w | 2.0 | $Random(1,2,4)$
max w | 2.0 | $Random(0.5,1.0,2.0)$
gan w | 1.0 | $Random(0.5,1.0,2.0)$
eta | 0.01 | $Random(0.01,0.05)$
recon cyc w | 0.0 | $Random(0.1,0.2,0.5,1.0)$
warm iter r | 0.2 | $Random(0.1,0.2,0.3,0.4,0.5)$
warm scale | 0.005 | $10^{Uniform(-5,-3)}$

Table 6: Default hyperparameters and random search distribution for baselines
in DG

###  0.C.2 Overall performance

Unsupervised Domain Generalization. Due to limited space in the paper, we show
the rest UDG results in Table 7. We employ Clipart, Infograph, and Quickdraw
as training domains and Painting, Real, and Sketch as test domains. Following
the same all correlated settings and protocols in DARLING, we find that our
DisMAE could achieve 1.14%, 1.19%, 4.40%, and 5.45% gains for average accuracy
over the second-best baselines across 1%, 5%, 10%, and 100% fraction setting
respectively.

|  |  | Label Fraction 1% (Linear evaluation) | Label Fraction 5% (Linear evaluation)
---|---|---|---|---
Methods | Paint | Real | Sketch | Overall | Avg. | Paint | Real | Sketch | Overall | Avg.
ERM [44] | 7.98 | 9.94 | 5.38 | 8.46 | 7.77 | 6.48 | 8.64 | 9.84 | 8.29 | 8.32
MoCo V2 [14] | 7.34 | 10.19 | 4.46 | 8.22 | 7.33 | 11.15 | 13.28 | 7.09 | 11.41 | 10.51
BYOL [20] | 9.23 | 10.57 | 10.58 | 10.21 | 10.13 | 12.43 | 17.34 | 14.58 | 15.44 | 14.78
MAE [23] | 8.80 | 10.30 | 12.62 | 10.38 | 10.57 | 12.13 | 17.63 | 15.02 | 15.60 | 14.93
DARLING [60] | 8.59 | 9.01 | 11.10 | 9.32 | 9.57 | 9.31 | 12.00 | 13.72 | 11.61 | 11.68
CycleMAE [53] | 9.21 | 9.49 | 6.62 | 8.82 | 8.44 | 11.44 | 14.21 | 10.01 | 12.58 | 11.88
DisMAE (Ours) | 11.04 | 11.64 | 12.46 | 11.65 | 11.71 | 13.69 | 17.74 | 16.92 | 16.47 | 16.12
|  |  | Label Fraction 10% (Full finetuning) | Label Fraction 100% (Full finetuning)
Methods | Paint | Real | Sketch | Overall | Avg. | Paint | Real | Sketch | Overall | Avg.
ERM [44] | 14.14 | 16.76 | 12.63 | 15.19 | 14.51 | 26.25 | 33.29 | 23.25 | 29.28 | 27.59
MoCo V2 [14] | 16.45 | 19.12 | 9.48 | 16.39 | 15.02 | 22.94 | 35.26 | 18.14 | 28.36 | 25.44
BYOL [20] | 19.12 | 20.55 | 18.68 | 19.77 | 19.45 | 24.91 | 35.72 | 24.19 | 30.39 | 28.27
MAE [23] | 19.34 | 23.07 | 24.18 | 22.29 | 22.20 | 31.72 | 43.72 | 36.74 | 39.02 | 37.40
DARLING [60] | 13.72 | 19.76 | 16.43 | 17.40 | 16.64 | 25.87 | 37.60 | 26.67 | 32.11 | 30.05
CycleMAE [53] | 17.80 | 22.93 | 17.15 | 20.23 | 19.29 | 31.72 | 39.44 | 30.00 | 35.39 | 33.72
DisMAE (Ours) | 24.49 | 27.06 | 28.24 | 26.61 | 26.60 | 38.89 | 45.95 | 43.72 | 43.58 | 42.85

Table 7: Unsupervised domain generalization results on DomainNet. We employ
Clipart, Infograph, and Quickdraw as training domains and Painting, Real, and
Sketch as test domains. All the models are unsupervised pre-trained before
fine-tuning on the labeled data. Overall and Avg. are the overall test data
accuracy and the arithmetic mean of individual domain accuracy respectively.
Note that they are different because the size of each test domain isn’t equal.
Bold=best, underline=second best.

Domain Generalization. Aligning with the training-domain validation setup in
DomainBed, we achieve 0.50% gains for the average accuracy in PACS datasets,
as shown in Table 8.

Methods | A | C | P | S | Avg.
---|---|---|---|---|---
ERM [44] | 85.34 | 80.26 | 96.83 | 71.75 | 83.55
MMD [29] | 84.83 | 79.53 | 96.16 | 75.42 | 83.98
GroupDRO [40] | 83.99 | 79.96 | 97.36 | 76.38 | 84.42
IRM [2] | 86.05 | 81.86 | 97.33 | 70.00 | 83.81
Mixup [52] | 85.27 | 80.70 | 97.13 | 72.77 | 83.97
VREx [27] | 83.51 | 82.71 | 97.88 | 76.55 | 85.16
Fishr [39] | 84.60 | 80.06 | 97.68 | 72.40 | 83.69
DDG [58] | 84.23 | 82.29 | 97.63 | 76.41 | 84.69
RIDG [10] | 86.09 | 84.81 | 96.41 | 76.18 | 85.87
DisMAE (Ours) | 85.48 | 83.19 | 97.16 | 79.66 | 86.37

Table 8: Domain generalization results on PACS. Bold=best, underline=second
best.

###  0.C.3 Discussion of the invariance principle

In Figure 2 and 4, we visualize the representation acquired through MAE, our
semantics encoder, and the variational encoder via t-SNE. We find that: (1)
The representations generated by MAE for each domain showcase a degree of
overlap at the center of the picture, accompanied by slight variations within
each distribution. This suggests that MAE captures both semantics and domain-
variant information but fails to disentangle them effectively. (2) Our
semantic representations in each domain distribute uniformly. This justifies
that DisMAE could learn domain-invariant representations from each domain. (3)
The variation representations in each domain has their specific distribution.
Clusters of similar variation data further emphasize domain-specific
characteristics.

###  0.C.4 More ablation study

Effects of decoder depth. The efficacy of the adaptive contrastive loss hinges
on the output of decoders. This prompts the inquiry: how many decoder layers
are optimal for achieving peak performance? As shown in Table 9, a deeper
decoder may lead to overfitting in reconstruction and subsequently diminish
the effect of our contrastive loss. Thus, adopting a lightweight decoder could
both accelerate the training and guarantee robustness.

Decoder Layer | Overall | Avg.
---|---|---
1 | 36.89 | 39.39
2 | 34.70 | 37.99
4 | 34.49 | 37.08
8 | 32.97 | 36.59
Table 9: Hyperparameter Analysis of the decoder layer

Effects of mask ratios. In Table 10, we set different mask ratios to test the
robustness of our model. And we found that the 80 percentile of the mask ratio
reaches the optimal result. We set it as our default protocol.

Mask Ratio | Overall | Avg.
---|---|---
50 | 33.07 | 36.61
60 | 33.23 | 36.29
70 | 34.20 | 37.80
80 | 36.89 | 39.39
90 | 34.91 | 37.20
Table 10: Hyperparameter Analysis of the mask ratio

###  0.C.5 Failure cases

Some failure cases of our proposed DisMAE are in Figure 6. Our approach
struggles with reconstruction containing intricate details and lines. It
frequently fails to generate images that possess sufficient detail while
simultaneously providing clear augmented variations. We attribute these
failures to two primary reasons: 1) The MAE backbone operates on patches,
making pixel-level reconstruction difficult, and our method heavily relies on
the MAE model’s reconstruction outcomes. 2) Our disentanglement lacks
granularity, often capturing broad color regions and background information
rather than nuanced details. In the context of UDG, reconstructing images with
fine granularity, high resolution, and authenticity remains a challenging and
crucial research direction. We are also keenly interested in exploring the
potential integration of the diffusion model within the UDG framework.

Figure 6: Some failure cases of reconstructed images generated by DisMAE.

###  0.C.6 Qualitative reconstructions

Additional visualization results of image reconstruction, spanning both
colored MNIST and DomainNet, can be observed in Figure 7.

DisMAE differentiates between the foreground and background of an image.
Remarkably, DisMAE can discern domain styles and fuse domain-specific elements
across them — a notable instance is superimposing the sun from a sketch onto a
painting. Such disentanglement ability endows DisMAE with the flexibility to
generate controllable images by manipulating semantic and variation factors
through swapping.

Figure 7: Illustrative reconstructed images generated by DisMAE.

###  0.C.7 Detailed implementation of DisMAE

We conduct all the experiments in Pytorch on a cluster of 8 NVIDIA Tesla A100
GPUs with 40GB each. Our default backbone consists of 12 blocks of semantic
encoder, 6 blocks of variation encoder, and a transform-based decoder. We
utilize ViT-B/16 as our default backbone for both visualization and main
experiments. And we use ViT-Tiny/16 in our ablation study. We let margin
$\gamma=0.008$ and $\tau=0.4$. In UDG, we choose the AdamW optimizer for the
main branch and set the learning rate as 1e-4 and betas as (0.9, 0.95) for
pre-training. As for finetuning, we adopt the learning rate as 0.025, 0.05,
5e-5, 5e-5 and batch size as 96, 192, 36, and 36 in the label fraction 1%, 5%,
10%, and 100% experiments respectively. And we finetune all the checkpoints
for 50 epochs. In DG, $\lambda_{1}$ is selected within {5e-4, 1e-3, 5e-3,
1e-2} and $\lambda_{2}$ is selected within {0.1, 0.5, 1.0, 2.0}. The detailed
hyperparameters for UDG and DG are listed in Table 11.

Details about training the domain classifier. As for the domain classifier, we
use the SGD optimizer with a learning rate of 0.0005, momentum of 0.99, and
weight decay of 0.05. We choose adaptive training intervals $T_{ad}$ as 15 and
maximum adaptive training epochs $E_{ad}$ as 100 in the UDG setting. We only
update the domain classifier by minimizing the cross-entropy loss while
freezing backbones when $e\bmod T_{ad}==0$ and $e\leq E_{ad}$, where $e$ is
the current training epoch. The detailed algorithm can be found in Appendix
0.A.

| batch size | weight decay | $\lambda_{1}$ | $\lambda_{2}$ | mask ratio
---|---|---|---|---|---
UDG | DomainBed
Paint, Real, Sketch $\rightarrow$ Clip, Info, Quick | $96\times 3$ | 0.05 | 1e-3 | 0 | 0.80
Clip, Info, Quick $\rightarrow$ Paint, Real, Sketch | $96\times 3$ | 0.05 | 1e-3 | 0 | 0.80
DG | PACS
C, P, S $\rightarrow$ A | $32\times 3$ | 0.05 | 1e-3 | 1.0 | 0.55
A, P, S $\rightarrow$ C | $32\times 3$ | 0.05 | 1e-3 | 1.0 | 0.40
A, C, S $\rightarrow$ P | $32\times 3$ | 0.05 | 5e-3 | 1.0 | 0.30
A, C, P $\rightarrow$ S | $40\times 3$ | 0.05 | 1e-2 | 0.5 | 0.55
DG | VLCS
L, S, V $\rightarrow$ C | $32\times 3$ | 0.05 | 1e-3 | 1.0 | 0.70
C, S, V $\rightarrow$ L | $32\times 3$ | 0.05 | 1e-3 | 1.0 | 0.40
C, L, V $\rightarrow$ S | $32\times 3$ | 0.05 | 5e-3 | 1.0 | 0.40
C, L, S $\rightarrow$ V | $40\times 3$ | 0.05 | 1e-3 | 1.0 | 0.40

Table 11: Hyperparameters Selection of DisMAE.


11institutetext: Department of Theoretical Physics, Aristotle University of
Thessaloniki, 54124 Thessaloniki, Greece

# Twin stars: probe of phase transition from hadronic to quark matter

Themistoklis Deloudis 11<EMAIL_ADDRESS>Polychronis Koliogiannis 11
<EMAIL_ADDRESS>Charalampos Moustakidis 11<EMAIL_ADDRESS>
###### Abstract

In agreement with the gravitational-wave events which are constantly
increasing, new aspects of the internal structure of compact stars have come
to light. A scenario in which a first order transition takes place inside
these stars is of particular interest as it can lead, under conditions, to a
third gravitationally stable branch (besides white dwarfs and neutron stars).
This is known as the twin star scenario. The new branch yields stars with the
same mass as normal compact stars but quite different radii. In the current
work, we focus on hybrid stars undergone a hadron to quark phase transition
near their core and how this new stable configuration arises. Emphasis is to
be given especially in the aspects of the phase transition and its
parametrization in two different ways, namely with Maxwell construction and
with Gibbs construction. Qualitative findings of mass-radius relations of
these stars will also be presented.

##  1 Introduction

Compact stars yield the most prominent natural laboratories for the study of
exotic forms of matter Glendenning-1997 ; Haensel-2007 . Recently discovered
pulsars alongside with gravitational waves detection, such as GW170817, have
revealed new aspects of the internal structure of these stars, mainly in terms
of their composition Christian-2018 ; Montana-2019 . Whilst the equation of
state of nuclear matter is well established up to nuclear saturation density,
one encounters the challenge of describing matter in fairly higher densities
realized in the interior of these stars. At these densities the type of matter
is yet to be determined and in turn, the construction of stellar models to
agree with the aforementioned observations is still an open issue. Possible
candidates are strange quark stars composed of deconfined quarks, pure neutron
stars composed by hadrons, and hybrid stars composed by hadronic outer shells
and cores of deconfined quarks. In the present article the latter case is
claimed.

The idea of a third family of compact stars and in particular, the connection
with the possibility to be a signature of a strong phase transition in the
interior of the star, was introduced first by Gerlach Gerlach-1968 . Later on,
Kämpfer worked also on this issue Kampfer-1981a ; Kamfer-1981b . Glendenning
and Kettner introduced the term “twins" in their paper Glendenning-2000 ,
while at the same time Schertler et al. Schertler-2000  worked out that idea
in detail. However, in all the previous studies, the maximum mass was
approximately at the canonical binary pulsar mass $1.4\leavevmode\nobreak\
M_{\odot}$. The revival of the idea of the twin stars started a few years
later by Blaschke et al. Blaschke-2013 ; Castillo-2013 . Specifically, in the
mentioned papers is suggested that high-mass twin stars, once detected by
simultaneous mass and radius measurements, could provide the evidence for a
strong first-order phase transition in cold matter, which then would imply the
existence of at least one critical endpoint in the quantum chromodynamics
phase diagram. Moreover, in the same work is also presented, for the first
time, examples of equations of state that would not only provide twin
solutions, but also fulfill the constraint on the maximum mass from the
existence of pulsars as heavy as $2\leavevmode\nobreak\ M_{\odot}$
Blaschke-2013 ; Castillo-2013 . The previous idea was elaborated by Benic et
al. Benic-2015  (see also Ref. Castillo-2015 ). A systematic Bayesian analysis
of the new twin star equation of state with observational constraints was
presented in Ref. Castillo-2016 . Finally, the analysis of the robustness of
twin solutions against changing the Maxwell to a mixed phase construction, and
the formation of structures in the mixed phase due to the interplay of the
surface tension and Coulomb interaction effects, have been also considered
respectively in Refs Ayriyan-2018 ; Maslov-2019 .

Stars of this branch are expected to have masses in the same range as normal
neutron stars, yet fairly smaller radii. The existence of such stars is a
strong indication that a HQPT is a physical reality, a result of utmost
importance especially in the study of dense matter physics. A study of the
HQPT is presented here, examining the conditions under which the twin star
configuration arises. The compatibility with the mass and radius constraints
as they are formed through up to date observations is also considered.

##  2 Theoretical Framework

Examining the configuration of a compact star is efficiently achieved by
separating the interior into regions. Generally, it is suitable to claim a
hadronic phase composing the outer shells, a core of deconfined quarks and a
connecting region embodying the phase transition. As it will become clear in
the following, this last region can be omitted yielding a sharp phase
transition or, as it is commonly called, a Maxwell construction (MC).
Oppositely, Gibbs construction (GC) assumes the existence of this transitional
region, which is commonly referred as the mixed phase. In each case, the phase
transition is assumed to be of first order Christian-2018 ; Montana-2019 ;
Glendenning-2000 ; Alvarez-2017 ; Bejger-2017 ; Bhattacharyya-2010 ; Endo-2006
; Christian-2019 ; Christian-2021 ; Alford-2013 ; Han-2019 .

###  2.1 Hadronic phase

In the outer layers of a compact star, matter is expected to be mainly
hadronic. It can either consist solely of nucleons or other hadrons, usually
hyperons. In our study the APR model introduced in Ref. Akmal-1998  combined
with the momentum dependent interactions (MDI) parametrization in the way
presented in Refs. Koliogiannis-2020 ; Koliogiannis-2021  is applied. Hence,
no hadrons other than protons and neutrons are assumed. The selection of this
particular hadronic model is due to the numerous advantages provided, namely
the reproduction of symmetric nuclear matter properties, value and slope of
symmetry energy at the saturation density and the agreement with limits
provided by experimental predictions. Furthermore, state-of-the-art
calculations considering high densities, as well as calculations of the Chiral
model considering pure neutron mater are successfully reproduced. Lastly, as
far as the results are concerned, this model can reproduce maximum masses of
neutron stars even higher than those observed Koliogiannis-2020 ;
Koliogiannis-2021 ; Arzoumanian ; Antoniadis ; Cromartie .

###  2.2 Quark phase

Deconfined quark matter composing the stellar core is in general difficult to
formulate Baym-2018 . One needs sophisticated models to study it in detail.
Nevertheless, one possibility is to mimic these models by assuming constant
value of the speed of sound throughout the quark phase. This is a robust
feature on top of being practical, as it simplifies the parametrization of the
energy density expression. Apparently, the speed of sound $c_{\rm
s}^{2}=\partial p/\partial\mathcal{E}$ is subjected to the constraint $c_{\rm
s}\leq 1$ in all possible cases. Its exact value plays a decisive role on the
final configuration that the star will achieve.

###  2.3 Phase transition

As the density of stellar matter soars above the nuclear saturation density, a
phase transition must take place in order for the quark core to occur. A
common approach to describe the former is with a polytropic equation of state
of the form Christian-2018

$p=K\rho^{\Gamma},$ (1)

where $p$ stands for pressure, $\rho$ for baryon density and $K,\Gamma$ are
constants. Through Eq. (1) it is easy to distinguish between a sharp
transition (MC), by imposing $\Gamma=0$, and a non-sharp transition (GC),
assuming $\Gamma\neq 0$.

A phase transition under constant pressure, as in MC, leads to a specific
structure of the star. Due to the monotonic reduction of the pressure relative
to the radius, points of equal pressure must be mapped onto the same radial
coordinate inside the star Glendenning-1997 . That is, the hadronic phase of
the outer shells and the quark phase of the core are essentially in direct
contact with each other at the point of transition, as a result of the absence
of a finite transitional region. This is not the case in GC where the
configuration includes this region and a regular behaviour of the pressure
inside the star. In the following, these two cases are examined in detail.

####  2.3.1 Maxwell construction

An abrupt phase change yields discontinuity in at least one physical quantity.
In this study the energy density is of primary concern, claiming the form
Christian-2018 ; Montana-2019

$\displaystyle\mathcal{E}(p)$ $\displaystyle=$
$\displaystyle\left\\{\begin{array}[]{ll}\mathcal{E}_{\rm APR+MDI}(p),\quad
p<p_{\rm tr}&\\\ \\\ \mathcal{E}(p_{\rm tr})+\Delta\mathcal{E}+(p-p_{\rm
tr})\leavevmode\nobreak\ c_{\rm s}^{-2},\quad p\geq p_{\rm
tr}&\end{array}\right.$ (5)

where $\mathcal{E}(p)$ denotes the energy density, $p$ the pressure, $c_{\rm
s}$ the speed of sound, and $\Delta\mathcal{E}$ is the magnitude of the energy
density jump at the transition point. Subscript $``{\rm tr}"$ denotes the
corresponding quantity at this point. It should be clear that the first line
in Eq. (5) refers to the hadronic phase while the second one refers to the
quark phase.

By definition, MC is characterized by a precondition which, when satisfied,
leads to an unstable regime essential for the third branch of stable hybrid
stars to arise Christian-2018 ; Montana-2019 ; Alvarez-2017 . This constraint,
first shown by Seidov Seidov-1971 , is expressed as

$\Delta\mathcal{E}\geq\frac{3p_{\rm tr}+\mathcal{E}(p_{\rm tr})}{2}.$ (6)

The star will immediately become unstable after its central pressure reaches
the value $p_{\rm tr}$. Otherwise, a stable state occurs. Therefore, when
seeking twin stars with MC one should always ensure that Eq. (6) is satisfied.

####  2.3.2 Gibbs construction

In a case of a non-sharp HQPT (GC), a finite region to embody the transition
is implied. The mixed phase of this transition is composed, as its name
suggests, by intermittent domains of pure hadronic and quark phases
Bhattacharyya-2010 ; Endo-2006 . Contrary to MC, no discontinuities in energy
density appear, giving rise to the profile

$\displaystyle\mathcal{E}(p)$ $\displaystyle=$
$\displaystyle\left\\{\begin{array}[]{ll}\mathcal{E}_{\rm APR+MDI}(p),\quad
p\leq p_{\rm tr}&\\\ \\\ \Lambda p^{1/\Gamma}+p\leavevmode\nobreak\
(\Gamma-1)^{-1},\quad p_{\rm tr}\leq p\leq p_{\rm css}&\\\ \\\
\mathcal{E}(p_{\rm css})+(p-p_{\rm css})\leavevmode\nobreak\ c_{\rm
s}^{-2},\quad p\geq p_{\rm css}.&\end{array}\right.$ (12)

Again, the energy density is denoted by $\mathcal{E}(p)$, the pressure by $p$,
the speed of sound by $c_{\rm s}$, while $\Lambda$ and the polytropic index
$\Gamma$ are constants. Subscript $``{\rm\rm css}"$ denotes the corresponding
quantity at the start of the quark phase. In addition, the continuity at the
two transition points imposes

$\Lambda=\frac{\mathcal{E}(p_{\rm tr})-p_{\rm tr}\leavevmode\nobreak\
(\Gamma-1)^{-1}}{p_{\rm tr}^{1/\Gamma}},$ (13)

and

$\mathcal{E}(p_{\rm\rm css})=\left(\frac{p_{\rm\rm css}}{p_{\rm
tr}}\right)^{1/\Gamma}\left(\mathcal{E}(p_{\rm tr})-\frac{p_{\rm
tr}}{\Gamma-1}\right)+\frac{p_{\rm\rm css}}{\Gamma-1},$ (14)

Apparently, the first line in Eq. (12) corresponds to the hadronic outer
shells, the second one to the mixed phase, embodying the HQPT, and the third
one to the quark core.

Figure 1: Mass-Radius diagram in the case of a sharp transition at $n_{\rm
tr}=2.1\leavevmode\nobreak\ n_{\rm s}$ (dashed lines) for various values of
the energy density jump $\Delta\mathcal{E}$. The solid line refers to the
absence of a phase transition and is added for comparison. The coloured strips
near the top of the figure denotes the range (uncertainty) around the maximum
observed mass values of three pulsars Arzoumanian ; Antoniadis ; Cromartie .

An immediate consequence of GC is that an energy density jump is not defined
as in MC. Therefore, in comparing the energy density change due to the
transition in the two constructions one needs to use the rise of the energy
density throughout the mixed phase in GC as a counterpart of
$\Delta\mathcal{E}$ of MC. It is immediately seen that

$\displaystyle\Delta\mathcal{E}_{\rm GC}$ $\displaystyle=$
$\displaystyle\mathcal{E}(p_{\rm tr})\left[\left(\frac{p_{\rm\rm css}}{p_{\rm
tr}}\right)^{1/\Gamma}-1\right]$ (15) $\displaystyle+$
$\displaystyle\frac{1}{\Gamma-1}\left[p_{\rm\rm css}-\left(\frac{p_{\rm\rm
css}}{p_{\rm tr}}\right)^{1/\Gamma}p_{\rm tr}\right].$

Furthermore, in GC no constraint similar to Eq. (6) is present. In order to
achieve twin star configuration only the appropriate choice of the free
parameters is needed.

##  3 Results

Applying our aforementioned assertions on a non-rotating stellar model a first
approach of the phenomenon is achieved and the basis is set for its study in
rotating models. Remarkably enough, a HQPT introduces only two free parameters
in each construction. Specifically, these correspond to the critical pressure
$p_{\rm tr}$ of the hadronic phase and the magnitude of the discontinuity
$\Delta\mathcal{E}$ in the case of MC, or the critical pressure $p_{\rm tr}$
and the mixed phase interval, commonly described by the pressure difference of
its boundaries, $\Delta p$, in the case of GC. Other free parameters may exist
overall, though not introduced by the implementation of a HQPT. For example,
the value of the energy density at the transition point depends on the
hadronic equation of state that is used.

Figure 1 is a M-R diagram visualising the case of a sharp transition taking
place at $n_{\rm tr}=2.1n_{\rm s}$, where $n$ is the baryon density and the
subscripts $``{\rm tr}"$ and $``{\rm s}"$ denote its value at the transition
and the saturation point, respectively.

Figure 2: Mass-Radius diagram in the case of a sharp transition at $n_{\rm
tr}=2.2\leavevmode\nobreak\ n_{\rm s}$ (dashed lines) for various values of
the energy density jump $\Delta\mathcal{E}$. The solid line refers to the
absence of a phase transition and is added for comparison. The coloured strips
near the top of the figure denotes the range (uncertainty) around the maximum
observed mass values of three pulsars Arzoumanian ; Antoniadis ; Cromartie .

In examining Figure 1, the branching due to the existence of a HQPT is
immediately evident. The change of $\Delta{\cal E}$ leads to a different
position of the second minimum but the shape of the second branch remains
almost identical. Moreover, it is worth to notice that higher values of
$\Delta{\cal E}$ lead to smaller radii for the corresponding maximum mass
configurations. A possible explanation is that larger $\Delta\mathcal{E}$
yield heavier stellar cores with an accompanying greater gravitational pull on
the outer shells, as suggested in Ref. Montana-2019 . These conclusions are in
agreement with Ref. Christian-2018 . As predicted by theory, twin star
branches yield fairly smaller radii compared with the normal neutron star
branch for each value of stellar mass. It is also clear that the higher the
energy density jump at the transition point, the lower the resulting stellar
mass. Another important remark is that the parametrization used for the HQPT
reproduces the maximum masses imposed by observations Christian-2018 ;
Montana-2019 ; Alvarez-2017 ; Arzoumanian ; Antoniadis ; Cromartie . To
explicitly show that, the masses of three pulsars belonging to different
binary systems are included in Figure 1.

The last point one needs to remark in that figure is the fact that the
transition occurs at baryon density equal to 2.1 times the saturation density.
We display also the Figure 2, in order to study the effect of the location of
the transition density on the various twin stars branches.

We observed that all of the fundamental properties mentioned above are still
valid for $n_{\rm tr}=2.2n_{\rm s}$. The only difference is that in this case
the point of branching is slightly pushed towards higher masses, a fact that
obviously does not affect the agreement with the observations. Lastly, it
should be noted that the results for the MC were derived by assuming $c_{\rm
s}=1$.

It is worth noting that the case of a non-sharp phase transition (GC) should
not be omitted, at least in a form of reference. Gibbs construction yields a
more complicated configuration of the compact star. This difficulty has
resulted in not many publications reviewing this case in detail. We are
currently working on the twin star scenario with GC, with the results yet in
progress. We are confident that the extraction of these results will allow not
only for a comparison and discrimination between the two constructions, but
also to the conclusion on which of the two is the most physically plausible.

##  4 Conclusions and outlook

Analyzing the twin star scenario, it is primarily seen that two ways to
formulate the HQPT, MC and GC, are dominant. These seem to cover every
plausible scenario for the transition. It has to be noted that configurations
achieved with MC are simpler than GC. However, in MC case the condition Eq.
(6) must also be fulfilled. Results posed in Figures 1 and 2 with MC show
clearly the differentiation of cases in a compact star with and without a
phase transition. The formulation achieved for the former case reproduces
masses and radius in agreement with observations in a satisfying interval of
values of the baryon density at the transition point. For a more complete
study, the same results are expected to be deducted for GC.

Future considerations may include: (a) designating the conditions under which
the twin star branch and the normal neutron star branch are separated; (b)
achieving plausible configurations with $c_{\rm s}^{2}=1/3$, a value which is
closer to reality than causality; and (c) examining the phenomenon in the
frame of rotating stellar models.

The authors would like to thank Prof. D. Blaschke for his useful insight and
comments.

## References

  * (1) N. K. Glendenning, Compact stars - Nuclear physics, Particle physics and General Relativity, Springer (1997)
  * (2) P. Haensel, A. Y. Potekhin, and D. G. Yakovlev, Neutron Stars 1: Equation of State and Structure (Springer-Verlag, New York, 2007)
  * (3) J.E. Christian, A. Zacchi, and J. Schaffner-Bielich, Eur. Phys. J. A 54, 28 (2018)
  * (4) G. Montana, L. Tolos, M. Hanauske, L. Rezzolla, Phys. Rev. D 99, 103009 (2019)
  * (5) U.H. Gerlach, Phys. Rev. 172, 1325 (1968)
  * (6) KÃmpfer, J. Phys. A: Math. Gen. 14, L471, (1981)
  * (7) KÃmpfer, Phys. Lett. B 101, 366 (1981)
  * (8) N.K. Glendenning, C. Kettner, Astron. Astrophys. 353, L9âL12 (2000)
  * (9) K. Schertler, C. Greiner, J. Schaffner-Bielich, and M.H. Thoma, Nucl. Phys. A 677, 463 (2000)
  * (10) D. Blaschke, D.E. Alvarez-Castillo, and S. Benic, PoS CPOD2013 063 (2013); [arXiv:1310.3803]
  * (11) D.E. Alvarez-Castillo and D. Blaschke, arXiv:1304.7758
  * (12) S. Benic, D. Blaschke, D.E. Alvarez-Castillo, T. Fischer, and S. Typel, A$\&$A 577, A40 (2015)
  * (13) D.E. Alvarez-Castillo and D. Blaschke, Phys. Part. Nucl. 46, 846 (2015)
  * (14) D.E. Alvarez-Castillo, A. Ayriyan, S. Benic, D. Blaschke. H. Grigorian, and S. Typel, Eur. Phys. J. A 52, 69(2016)
  * (15) A. Ayrinan, N.-U. Vastian, D. Blaschke, H. Grigorian, K. Maslov, and D.V. Voskresensky, Phys. Rev. C 97, 45802 (2018)
  * (16) K. Maslov, N. Yasutake, D. Blaschke, A. Ayriyan, H. Grigorian, and T. Maruyama, Phys. Rev. C 100, 025802 (2019)
  * (17) D.E. Alvarez Castillo and D.B. Blaschke , Phys. Rev. C 96, 045809, (2017)
  * (18) M. Bejger, D. Blaschke, P. Haensel, J.L. Zdunik, and M. Fortin, A&A 600, A39 (2017)
  * (19) A. Bhattacharyya, I. N. Mishustin, W. Greiner, J. Phys. G 37, 025201 (2010)
  * (20) T. Endo, T. Maruyama, S. Chiba, T. Tatsumi, arXiv: hep-ph/0502216
  * (21) J.E. Christian, A. Zacchi, and J. Schaffner-Bielich, Phys. Rev. D 99, 023009 (2019)
  * (22) J.E. Christian, and J. Schaffner-Bielich, Phys. Rev. D 103, 063042 (2021)
  * (23) M.G. Alford, S. Han, Phys. Rev. D 88, 083013 (2013)
  * (24) S. Han, M.A.A. Mamun, S. Lalit, C. Constantinou, M. Prakash, Phys. Rev. D 100, 103022 (2019)
  * (25) A. Akmal, V.R. Pandharipande, and D.G. Ravenhall, Phys. Rev. C 58, 1804 (1998)
  * (26) P. S. Koliogiannis, Ch. C. Moustakidis, Phys. Rev. C 101, 015805 (2020)
  * (27) P.S. Koliogiannis and Ch.C. Moustakidis, Astroph.J. 912, 69 (2021)
  * (28) Z. Arzoumanian, A. Brazier, S. Burke-Spolaor, et al., Astrophys. J. Suppl. Ser. 235, 37 (2018)
  * (29) J. Antoniadis, P. Freire, N. Wex, et al., Sci. 340, 448 (2013)
  * (30) H. Cromartie, E. Fonseca, S. Ransom, et al., Nat. Astron. 4, 72 (2020)
  * (31) G. Baym, T. Hatsuda, Toru Kojo, P. D. Powell, Y. Song, and T. Takatsuka, Rep. Progr. Phys. 81, 056902 (2018)
  * (32) Z.F. Seidov, Soviet Astronomy, 15, 347 (1971)