# RobustSentEmbed: Robust Sentence Embeddings Using Adversarial Self-
Supervised Contrastive Learning

Javad Rafiei Asl,1 Prajwal Panzade,1 Eduardo Blanco,2 Daniel Takabi,3 Zhipeng
Cai1
1Georgia State University, 2University of Arizona, 3Old Dominion University
<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Pre-trained language models (PLMs) have consistently demonstrated outstanding
performance across a diverse spectrum of natural language processing tasks.
Nevertheless, despite their success with unseen data, current PLM-based
representations often exhibit poor robustness in adversarial settings. In this
paper, we introduce RobustSentEmbed, a self-supervised sentence embedding
framework designed to improve both generalization and robustness in diverse
text representation tasks and against a diverse set of adversarial attacks.
Through the generation of high-risk adversarial perturbations and their
utilization in a novel objective function, RobustSentEmbed adeptly learns
high-quality and robust sentence embeddings. Our experiments confirm the
superiority of RobustSentEmbed over state-of-the-art representations.
Specifically, Our framework achieves a significant reduction in the success
rate of various adversarial attacks, notably reducing the BERTAttack success
rate by almost half (from 75.51% to 38.81%). The framework also yields
improvements of 1.59% and 0.23% in semantic textual similarity tasks and
various transfer tasks, respectively.

RobustSentEmbed: Robust Sentence Embeddings Using Adversarial Self-Supervised
Contrastive Learning

Javad Rafiei Asl,1 Prajwal Panzade,1 Eduardo Blanco,2 Daniel Takabi,3 Zhipeng
Cai1 1Georgia State University, 2University of Arizona, 3Old Dominion
University<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>

##  1 Introduction

Pre-trained Language Models (PLMs) have demonstrated state-of-the-art
performance in learning contextual word embeddings Devlin et al. (2019),
contributing to significant advancements in various Natural Language
Processing (NLP) tasks Yang et al. (2019); He et al. (2021); Ding et al.
(2023). PLMs, including prominent models like BERT Devlin et al. (2019) and
GPT-3 Brown et al. (2020), have revolutionized text classification, sentence
representation, and machine translation among a plethora of diverse NLP tasks.
While PLMs have expanded their focus to include universal sentence embeddings,
which effectively capture the semantic representation of input text, PLM-based
sentence representations lack two crucial characteristics: generalization and
robustness.

Extensive research efforts have been dedicated to the development of universal
sentence embeddings employing PLMs Reimers and Gurevych (2019); Zhang et al.
(2020); Neelakantan et al. (2022); Wang et al. (2023). Although these
embeddings have demonstrated proficiency in generalization across various
downstream tasks Sun et al. (2019); Gao et al. (2021), they exhibit
limitations when subjected to adversarial settings and remain vulnerable to
adversarial attacks Nie et al. (2020); Wang et al. (2021). Existing research
has highlighted the limited robustness of PLM-based representations Garg and
Ramakrishnan (2020); Wu et al. (2023); Hauser et al. (2023). The vulnerability
arises when these representations can be easily deceived by making small,
imperceptible modifications to the input text.

To address these limitations, we propose a method to obtain robust sentence
embeddings called RobustSentEmbed. The main idea is to generate small
adversarial perturbations and employ an efficient contrastive objective Chen
et al. (2020). The goal is to enhance the adversarial resilience of the
sentence embeddings. Specifically, our framework involves an iterative
collaboration between an adversarial perturbation generator and the PLM-based
encoder to generate high-risk perturbations in both token-level and sentence-
level embedding spaces. RobustSentEmbed then employs a contrastive learning
objective in conjunction with a token replacement detection objective to
maximize the similarity between the embedding of the original sentence and the
adversarial embedding of a positive pair (the former objective) as well as its
edited sentence (the latter objective).

We have conducted comprehensive experiments to substantiate the efficacy of
the RobustSentEmbed framework. The tasks encompass TextAttack Morris et al.
(2020) assessments, adversarial Semantic Textual Similarity (STS) tasks, Non-
adversarial STS tasks Conneau and Kiela (2018), and transfer tasks Conneau and
Kiela (2018). Two initial series of experiments were designed to evaluate the
robustness of our sentence embeddings against various adversarial attacks and
tasks. Subsequently, we conducted two final series of experiments to assess
the quality of our embeddings in the contexts of semantic similarity and
natural language understanding. RobustSentEmbed demonstrates significant
improvements in robustness, reducing the attack success rate from 75.51% to
38.81% against the BERTAttack attack and from 71.86% to 12.80% on adversarial
STS. Moreover, the framework outperforms existing methods in ten out of
thirteen tasks while obtaining comparable results with the other three,
showcasing improvements of 1.59% and 0.23% on STS tasks and NLP transfer
tasks, respectively.

Contributions. Our main contributions are summarized as follows:

  * •

We introduce RobustSentEmbed, an innovative framework designed for generating
sentence embeddings that are robust against adversarial attacks. Existing
methods are vulnerable to such adversarial challenges. RobustSentEmbed fills
this gap by generating high-risk perturbations and utilizing an efficient
adversarial objective function.111Our code are publicly available at
https://github.com/jasl1/RobustSentEmbed

  * •

We conduct comprehensive experiments to empirically evaluate the effectiveness
of the RobustSentEmbed framework. The empirical findings substantiate the
efficacy of our framework, as demonstrated by its superior performance in both
robustness and generalization benchmarks.

##  2 Related Work

Recently, self-supervised methods using contrastive objectives have become
prominent for learning effective and robust text representations: SimCSE, as
outlined by Gao et al. (2021), introduced a minimal augmentation method
involving the application of two distinct dropout masks to predict the input
sentence. The ConSERT model Yan et al. (2021) employed four unique data
augmentation techniques, namely adversarial attacks, token shuffling, cut-off,
and dropout, to generate a variety of perspectives in order to carry out a
contrastive objective. Miao et al. (2021) utilized adversarial training to
improve the robustness of contrastive learning. They achieved this by
incorporating regularization into their learning objective, combining benign
contrastive learning with an adversarial contrastive scenario. Rima et al.
(2022) proposed a novel method for training language processing models,
combining adversarial training and contrastive learning. Their approach
incorporates linear perturbations to input embeddings and uses contrastive
learning to minimize the distance between the original and perturbed
representations. Pan et al. (2022) introduced a simple technique to improve
the fine-tuning of Transformer-based encoders. Their method involves
regularization by generating adversarial examples through word embedding
perturbations and using contrastive learning to obtain noise-invariant
representations.

Unlike existing approaches for training text representation through
contrastive adversarial learning Yan et al. (2021); Miao et al. (2021); Rima
et al. (2022); Pan et al. (2022), our framework generates more efficient,
high-risk perturbations at both the token-level and sentence-level within the
embedding space. Furthermore, our framework utilizes a robust contrastive
objective and incorporates an adversarial replaced token detection method,
leading to high-quality text representations that yield improved
generalization and robustness characteristics.

Figure 1: The general architecture of the RobustSentEmbed framework.

##  3 The Proposed Framework

We introduce RobustSentEmbed, a straightforward yet highly effective method
for generating robust text representation. Given a PLM $f_{\theta}(.)$ as the
encoder and a raw dataset $\bm{\mathrm{\mathcal{D}}}$, our framework aims to
pre-train $f_{\theta}(\cdot)$ on $\bm{\mathrm{\mathcal{D}}}$ to enhance the
efficacy of sentence embeddings across a wide range of NLP tasks (improved
generalization) and to fortify its resilience against various adversarial
attacks (improved robustness). Figure 1 presents an overview of our framework.
The framework involves an iterative interaction between the perturbation
generator and the $f_{\theta}(.)$ encoder to produce high-risk adversarial
perturbations in both token-level and sentence-level embedding spaces. These
perturbations provide the essential adversarial examples required for
adversarial training by both the $f_{\theta}(.)$ encoder and a PLM-based
discriminator. The subsequent sections will delve into the main components of
our framework.

###  3.1 Perturbation Generator

Adversarial perturbation involves adding maliciously crafted perturbations
into benign data, with the objective of misleading Machine Learning (ML)
models Goodfellow et al. (2015). A highly effective and broadly applicable
method for generating adversarial perturbations is to apply a small noise
$\bm{\delta}$ within a norm-constraint ball, aiming to maximize the
adversarial loss function:

$\arg\max_{{||\bm{\delta}||\leq\epsilon}}L(f_{\theta}(X+\bm{\delta}),y),$ (1)

where $f_{\theta}(.)$ denotes an ML model parameterized with $X$ as the sub-
word embeddings. There are numerous gradient-based algorithms designed to
address this optimization problem. Our framework extends the token-level
perturbation method proposed by Li and Qiu (2021) by complementing the
perturbation with an innovative sentence-level perturbation generator in order
to generate worst-case adversarial examples. The main idea is to train a PLM-
based model to withstand a broad spectrum of adversarial attacks, spanning
both word and instance levels.

Recognizing the different roles that individual tokens play within a sentence,
the RobustSentEmbed framework incorporates a scaling index to allow larger
perturbations for tokens exhibiting larger gradients during the normalization
of token-level perturbations:

$n^{i}=\frac{\lVert\bm{\eta}^{t}_{i}\rVert_{P}}{\max_{j}\lVert\bm{\eta}^{t}_{j}\rVert_{P}},$
(2)

where $\bm{\eta}^{t}_{i}$ represents the token-level perturbation for word $i$
at step $t$ of the gradient ascent, and $P$ denotes the type of norm
constraint. Considering the encoder $f_{\theta}(.)$ and an input sentence $x$,
RobustSentEmbed passes the sentence through $f_{\theta}(.)$ by applying
standard dropout twice. This process yields two different embeddings, denoted
as "positive pairs" and represented as $({X},{X^{+}})$. Finally, the newly
adjusted token-level perturbation is formulated as:

$\bm{\eta}^{t+1}_{i}=n^{i}*(\bm{\eta}^{t}_{i}+\gamma\frac{\bm{g}_{{\eta}_{i}}}{\lVert\bm{g}_{{\eta}_{i}}\rVert_{P})},$
(3)
$\bm{\eta}^{t+1}\leftarrow\Pi_{\lVert\bm{\eta}\rVert_{P}\leq\epsilon}(\bm{\eta}^{t}),$
(4)

where
$\bm{g}_{{\eta}_{i}}=\nabla_{\eta}{\mathcal{L}}_{con,\theta}(\bm{X}+\bm{\delta}^{t-1}+\bm{\eta}^{t-1},\\{\bm{X}^{+}\\})$
is the gradient of the contrastive learning loss with respect to $\bm{\eta}$.
The perturbation is generated by the $\ell_{\infty}$ norm-ball with radius
$\epsilon$, and $\Pi$ projects the perturbation onto the $\epsilon$-ball.

To generate adversarial perturbations at the sentence-level, RobustSentEmbed
employs a combination of the Fast Gradient Sign Method (FGSM) Goodfellow et
al. (2015) and the Projected Gradient Descent (PGD) technique Madry et al.
(2018). The framework iterates using this combination, specifically T-step
FGSM and K-step PGD, to systematically reinforce invariance within the
embedding space. Ultimately, this strategy leads to enhanced generalization
and robustness. It proceeds with the following steps to update the
perturbation for PGD in iteration $k+1$ and FGSM in iteration $t+1$:

$\bm{\delta}^{k+1}_{\mathrm{pgd}}=\Pi_{\lVert\bm{\delta}\rVert_{P}\leq\epsilon}(\bm{\delta}^{k}+\alpha
g(\bm{\delta}^{k})/\lVert g(\bm{\delta}^{k})\rVert_{P}),$ (5)
$\bm{\delta}^{t+1}_{\mathrm{fgsm}}=\Pi_{\lVert\bm{\delta}\rVert_{P}\leq\epsilon}(\bm{\delta}^{t}+\beta\mathrm{sign}(g(\bm{\delta}^{t}))),$
(6)

where
$g(\bm{\delta}^{n})=\nabla_{\bm{\delta}}{\mathcal{L}}_{con,\theta}(\bm{X}+\bm{\delta}^{n},\\{\bm{X}^{+}\\})$
with $n=t\;or\;k$ represents the gradient of the contrastive learning loss
with respect to $\bm{\delta}$. The variables $\alpha$ and $\beta$ denote the
step sizes for the attacks, while $\mathrm{sign}(.)$ yields the vector’s sign.
The final perturbation is obtained by employing a practical combination of
T-step FGSM and K-step PGD:

$\bm{\delta}_{\mathrm{final}}=\rho\bm{\delta}^{K}_{\mathrm{pgd}}+(1-\rho)\bm{\delta}^{T}_{\mathrm{fgsm}},$
(7)

where $0\leq\rho\leq 1$ modulates the relative importance of each separate
perturbation in the formation of the final perturbation.

###  3.2 Robust Contrastive Learning

To achieve robust text representations through adversarial learning, we employ
a straightforward approach that can be described as the combination of a
Replaced Token Detection (RTD) objective (Figure 1, right) with a novel self-
supervised contrastive learning objective (Figure 1, left).

Our framework extends an adversarial version of the RTD task used in ELECTRA
Clark et al. (2020). In this approach, given an input sentence $x$, ELECTRA
utilizes a pre-trained masked language model as the generator $G$ to recover
randomly masked tokens in $x^{{}^{\prime}}=\mathrm{Mask}(x)$, resulting in the
edited sentence $x^{{}^{\prime\prime}}=G(x^{{}^{\prime}})$. Subsequently, a
discriminator $D$ is tasked with predicting whether token replacements have
occurred, which constitutes the RTD task. As illustrated in Figure 1, the
perturbation generator module introduces token-aware perturbations into the
embedding of each individual token, making it more challenging for
discriminator $D$ to perform the RTD task effectively. The gradient of $D$ can
be back-propagated into $f$ through $\bm{h}=f_{\theta}(x)$. This mechanism
encourages $f$ to make vector $\bm{h}$ sufficiently informative, enhancing its
resilience against token-level adversarial attacks. Consequently, our
framework employs the following adversarial objective for a single sentence
$x$:

$\displaystyle{\mathcal{L}}^{x}_{RTD}$ $\displaystyle=\sum_{j=1}^{\lvert
x\rvert}[\mathds{-1}(X^{adv}_{j}={X}_{j})\;\mathrm{log}\>D(X^{adv},\,\bm{h},\,j)$
$\displaystyle\mathds{-1}(X^{adv}_{j}\neq{X}_{j})\;\mathrm{log}\>(1-D(X^{adv},\,\bm{h},\,j))],$
(8)

where $X^{adv}={X}^{{}^{\prime\prime}}+\bm{\eta}^{max(K,\>T)}_{i}$ represent
the $i$th perturbed token in $x$. The training objective for the batch $B$ is
${\mathcal{L}}_{RTD,\,\theta}=\sum_{i=1}^{\lvert
B\rvert}{\mathcal{L}}^{x_{i}}_{RTD}$. Furthermore, we use self-supervised
contrastive learning to acquire effective low-dimensional representations by
bringing semantically similar samples closer and pushing dissimilar ones
further apart. Let $\\{(x_{i},x_{i}^{+})\\}_{i=1}^{N}$ denote a set of $N$
positive pairs, where $x_{i}$ and $x_{i}^{+}$ are semantically correlated and
$(z_{i},z_{i}^{+})$ represents the corresponding embedding vectors for the
positive pair $(x_{i},x_{i}^{+})$. We define $z_{i}$’s positive set as
$z_{i}^{pos}=\\{z_{i}^{+}\\}$, while the negative set
$z_{i}^{neg}=\\{z_{i}^{-}\\}$ is the set of positive pairs from other
sentences in the same batch. Then, the contrastive training objective is
defined as follows:

$\displaystyle{\mathcal{L}}_{con,\theta}(z_{i},z_{i}^{pos},z_{i}^{neg})=$
$\displaystyle-\log(\frac{\sum_{z_{i}^{pos}}\exp({sim(z_{i},z_{i}^{+})}/\tau)}{\sum_{(z_{i}^{pos}\cup\,z_{i}^{neg})}\exp({sim(z_{i},z_{i}^{+\,or\,-})}/\tau)}),$
(9)

where $\tau$ denotes a temperature hyperparameter and
$sim(u,v)=\frac{u^{\top}v}{\Arrowvert{u}\Arrowvert.\Arrowvert{v}\Arrowvert}$
is the cosine similarity between two representations. Our framework utilizes
contrastive learning to maximize the similarity between clean examples and
their adversarial perturbation by incorporating the adversarial example as an
additional element within the positive set:

${\mathcal{L}}_{RobustSentEmbed,\;\theta}:={\mathcal{L}}_{con,\theta}(z,\\{z^{pos},\;z^{adv}\\},\\{z^{neg}\\}).$

$\begin{split}\mathcal{L}_{total}:=&\mathcal{L}_{RobustSentEmbed,\,\theta}+{\lambda_{1}}\cdot{\mathcal{L}}_{con,\,\theta}(z^{adv},\\{z^{pos}\\},\\{z^{neg}\\})\\\
&+{\lambda_{2}}\cdot{\mathcal{L}}_{RTD,\,\theta},\end{split}$

(10)

where $z^{adv}=z+\bm{\delta}_{final}$ represents the adversarial perturbation
of the input sample $x$ in the embedding space, and $\lambda_{1}$,
$\lambda_{2}$ denote weighting coefficients. The first component of the total
contrastive loss (Eq. 10) is designed to optimize the sentence-level
similarity between the input sample $x$, its positive pair, and its
adversarial perturbation, while the second component serves to regularize the
loss by encouraging the convergence of the adversarial perturbation and the
positive pair of $x$. The final component introduces the adversarial Replaced
Token Detection (RTD) objective into the total contrastive loss.

Adversarial Attack | Model | IMDB | MR | SST2 | YELP | MRPC | SNLI | MNLI-Mismatched | Avg.
---|---|---|---|---|---|---|---|---|---
TextFooler | SimCSE-BERTbase | 75.32 | 65.53 | 71.49 | 79.67 | 80.07 | 72.65 | 68.54 | 72.61
USCAL-BERTbase | 61.94 | 48.71 | 55.38 | 62.30 | 60.18 | 54.82 | 53.74 | 56.72
RobustSentEmbed-BERTbase | 40.02 | 31.39 | 35.83 | 43.78 | 37.54 | 36.99 | 34.15 | 37.10
TextBugger | SimCSE-BERTbase | 52.21 | 42.04 | 49.67 | 56.19 | 56.73 | 45.39 | 40.16 | 48.91
USCAL-BERTbase | 39.16 | 27.37 | 31.90 | 41.25 | 37.86 | 30.79 | 25.45 | 33.40
RobustSentEmbed-BERTbase | 23.16 | 17.49 | 19.62 | 27.93 | 19.37 | 18.05 | 15.51 | 20.16
PWWS | SimCSE-BERTbase | 64.41 | 55.73 | 60.48 | 67.54 | 68.15 | 56.09 | 52.58 | 60.71
USCAL-BERTbase | 51.95 | 40.67 | 45.29 | 52.30 | 46.86 | 50.92 | 39.37 | 46.77
RobustSentEmbed-BERTbase | 32.94 | 28.05 | 29.28 | 29.14 | 24.72 | 26.28 | 27.90 | 28.33
BAE | SimCSE-BERTbase | 73.50 | 61.83 | 68.27 | 75.15 | 77.84 | 69.06 | 65.43 | 70.15
USCAL-BERTbase | 58.57 | 46.19 | 51.72 | 59.49 | 58.38 | 50.90 | 51.16 | 53.77
RobustSentEmbed-BERTbase | 37.16 | 29.12 | 31.43 | 40.96 | 35.53 | 33.87 | 31.85 | 34.27
BERTAttack | SimCSE-BERTbase | 78.42 | 66.94 | 73.59 | 80.87 | 82.16 | 74.35 | 72.22 | 75.51
USCAL-BERTbase | 63.23 | 51.08 | 57.73 | 63.96 | 63.05 | 55.41 | 55.86 | 58.62
RobustSentEmbed-BERTbase | 41.51 | 34.19 | 38.16 | 44.96 | 38.26 | 38.60 | 35.98 | 38.81

Table 1: Attack success rates (lower is better) of various adversarial attacks
applied to three sentence embeddings (SimCSE, USCAL, and RobustSentEmbed)
across five text classification and two natural language inference tasks.
RobustSentEmbed reduces the attack success rate to less than half across all
attacks.

##  4 Evaluation and Experimental Results

This section presents a comprehensive set of experiments conducted to validate
the proposed framework’s effectiveness in terms of robustness and
generalization metrics. To evaluate robustness, the experiments include
adversarial attacks and adversarial Semantic Textual Similarity (STS) tasks.
To evaluate generalization, the experiments include non-adversarial STS and
transfer tasks within the SentEval
framework.222https://github.com/facebookresearch/SentEval Appendices A and B
provide training details and ablation studies that illustrate the effects of
hyperparameter tuning.

Adversarial Attack | Model | AdvSTS-B | AdvSICK-R | Avg.
---|---|---|---|---
TextFooler | SimCSE-BERTbase | 21.07 | 24.17 | 22.62
USCAL-BERTbase | 16.52 | 18.71 | 17.62
RobustSentEmbed-BERTbase | 7.18 | 8.53 | 7.86
TextBugger | SimCSE-BERTbase | 27.49 | 28.34 | 27.91
USCAL-BERTbase | 21.52 | 24.88 | 23.20
RobustSentEmbed-BERTbase | 11.32 | 12.94 | 12.13
PWWS | SimCSE-BERTbase | 24.15 | 26.82 | 25.49
USCAL-BERTbase | 21.28 | 23.65 | 22.47
RobustSentEmbed-BERTbase | 12.68 | 13.90 | 13.29
BAE | SimCSE-BERTbase | 26.92 | 28.81 | 27.86
USCAL-BERTbase | 22.92 | 25.48 | 24.20
RobustSentEmbed-BERTbase | 10.53 | 12.09 | 11.31
BERTAttack | SimCSE-BERTbase | 31.60 | 32.85 | 32.23
USCAL-BERTbase | 26.02 | 28.51 | 27.26
RobustSentEmbed-BERTbase | 12.58 | 13.02 | 12.80

Table 2: Attack success rates (lower is better) of five adversarial attack
techniques applied to three sentence embeddings (SimCSE, USCAL, and
RobustSentEmbed) across two Adversarial Semantic Textual Similarity (AdvSTS)
tasks (i.e. AdvSTS-B and AdvSICK-R). RobustSentEmbed reduces the attack
success rate to less than half across all attacks.

###  4.1 Adversarial Attacks

We evaluate the robustness of our framework against various adversarial
attacks, comparing it with two state-of-the-art sentence embedding models:
SimSCE Gao et al. (2021) and USCAL Miao et al. (2021). We fine-tuned the BERT-
based PLM across seven text classification and natural language inference
tasks, specifically MRPC Dolan and Brockett (2005), YELP Zhang et al. (2015),
IMDb Maas et al. (2011), Movie Reviews (MR) Pang and Lee (2005), SST2 Socher
et al. (2013), Stanford NLI (SNLI) Bowman et al. (2015), and Multi-NLI (MNLI)
Williams et al. (2018). To assess the robustness of our fine-tuned model, we
investigated the impact of five popular adversarial attacks: TextBugger Li et
al. (2019), PWWS Ren et al. (2019), TextFooler Jin et al. (2020), BAE Garg and
Ramakrishnan (2020), and BERTAttack Li et al. (2020b). Additional information
of these attacks is provided in Appendix C. To ensure statistical validity, we
conducted each experiment five times, with each iteration comprising 1000
adversarial attack samples.

Table 1 presents the average attack success rates of five adversarial attacks
applied to three sentence embeddings. Notably, our embedding framework
consistently outperforms the other two embedding methods, demonstrating
significantly lower attack success rates (less than half) across all text
classification and natural language inference tasks. Consequently,
RobustSentEmbed achieves the lowest average attack success rate against all
adversarial attack techniques. These findings substantiate the robustness of
our embedding framework and highlight the vulnerabilities of other state-of-
the-art sentence embeddings when confronted with various adversarial attacks.

Figure 2 presents the results of 1000 attacks conducted on two fine-tuned
sentence embeddings, assessing the average number of queries required and the
resulting accuracy reduction. Attacks on the RobustSentEmbed framework are
represented by green data points, while red points denote attacks on the USCAL
approach Miao et al. (2021). Each pair of connected points corresponds to a
specific attack. Ideally, a robust sentence embedding should be positioned in
the top-left region of the graph, indicating that it necessitates a higher
number of queries for an attack to deceive the model while causing minimal
performance degradation. Across all adversarial attacks, RobustSentEmbed
consistently exhibits greater stability compared to the USCAL method. In other
words, a larger number of queries is required for RobustSentEmbed, resulting
in a lower accuracy reduction (i.e., better performance) compared to USCAL.

Figure 2: Average number of queries and the resulting accuracy reduction for
two fine-tuned embeddings.

###  4.2 Robust Embeddings

We introduce a new task named Adversarial Semantic Textual Similarity (AdvSTS)
to assess the robustness of sentence embeddings. AdvSTS leverages an efficient
adversarial technique, like TextFooler, to manipulate an input sentence pair
of a non-adversarial STS task in a manner that leads the target model to
generate a regression score that maximally deviates from the actual score
(truth label). As a result, we generate an adversarial STS dataset by
transforming all benign instances from the original (i.e. non-adversarial)
dataset into adversarial examples. Table 2 presents the attack success rates
of five adversarial attacks applied to three sentence embeddings, including
our framework. These evaluations are conducted for two AdvSTS tasks,
specifically AdvSTS-B (originated from STS Benchmark Cer et al. (2017)) and
AdvSICK-R (originated from SICK-Relatedness Marelli et al. (2014)). Notably,
our framework consistently outperforms the other two sentence embedding
methods, exhibiting significantly lower attack success rates across both
AdvSTS tasks and all employed adversarial attacks. These results provide
additional evidence supporting the notion that RobustSentEmbed generates
robust text representation.

Model | STS12 | STS13 | STS14 | STS15 | STS16 | STS-B | SICK-R | Avg.
---|---|---|---|---|---|---|---|---
GloVe embeddings (avg.) $\heartsuit$ | 55.14 | 70.66 | 59.73 | 68.25 | 63.66 | 58.02 | 53.76 | 61.32
BERTbase (first-last avg.) $\clubsuit$ | 39.70 | 59.38 | 49.67 | 66.03 | 66.19 | 53.87 | 62.06 | 56.70
BERTbase-flow $\clubsuit$ | 58.40 | 67.10 | 60.85 | 75.16 | 71.22 | 68.66 | 64.47 | 66.55
BERTbase-whitening $\clubsuit$ | 57.83 | 66.90 | 60.90 | 75.08 | 71.31 | 68.24 | 63.73 | 66.28
ConSERT-BERTbase | 64.56 | 78.55 | 69.16 | 79.74 | 76.00 | 73.91 | 67.35 | 72.75
ATCL-BERTbase | 67.14 | 80.86 | 71.73 | 79.50 | 76.72 | 79.31 | 70.49 | 75.11
SimCSE-BERTbase | 68.66 | 81.73 | 72.04 | 80.53 | 78.09 | 79.94 | 71.42 | 76.06
USCAL-BERTbase | 69.30 | 80.85 | 72.19 | 81.04 | 77.52 | 81.28 | 71.98 | 76.31
RobustSentEmbed-BERTbase | 71.90 | 81.12 | 74.92 | 82.38 | 79.43 | 82.02 | 73.53 | 77.90
RoBERTabase-whitening | 46.99 | 63.24 | 57.23 | 71.36 | 68.99 | 61.36 | 62.91 | 61.73
ConSERT-RoBERTabase | 66.90 | 79.31 | 70.33 | 80.57 | 77.95 | 81.42 | 68.16 | 74.95
SimCSE-RoBERTabase | 68.75 | 80.81 | 71.19 | 81.79 | 79.35 | 82.62 | 69.56 | 76.30
USCAL-RoBERTabase | 69.28 | 81.15 | 72.81 | 81.47 | 80.55 | 83.34 | 70.94 | 77.08
RobustSentEmbed-RoBERTabase | 70.03 | 82.15 | 73.27 | 82.48 | 79.61 | 83.82 | 71.66 | 77.57
USCAL-RoBERTalarge | 68.70 | 81.84 | 74.26 | 82.52 | 80.01 | 83.14 | 76.30 | 78.11
RobustSentEmbed-RoBERTalarge | 69.30 | 81.76 | 75.14 | 83.57 | 79.74 | 83.90 | 77.08 | 78.64

Table 3: Semantic Similarity performance on STS tasks (Spearman’s correlation,
“all” setting) for sentence embedding models. We emphasize the top-performing
numbers among models that share the same pre-trained encoder. $\heartsuit$:
results from Reimers and Gurevych (2019); $\clubsuit$: results from Gao et al.
(2021); All remaining results have been reproduced and reevaluated by our
team. RobustSentEmbed produces the most effective sentence representations
that are more general in addition to robust representation (section 4.2 and
4.1).

###  4.3 Semantic Textual Similarity (STS) Tasks

In this section, we assess the performance of our framework across seven
Semantic Textual Similarity (STS) tasks encompassing STS datasets from 2012 to
2016 Agirre et al. (2012, 2013, 2014, 2015, 2016), STS Benchmark, and SICK-
Relatedness. To benchmark our framework’s effectiveness, we conducted a
comparative analysis against a range of unsupervised sentence embedding
approaches, including: 1) baseline methods such as GloVe Pennington et al.
(2014) and average BERT embeddings; 2) post-processing methods like BERT-flow
Li et al. (2020a) and BERT-whitening Su et al. (2021); and 3) state-of-the-art
methods such as SimCSE Gao et al. (2021) and USCAL Miao et al. (2021). We
validate the findings of the SimCSE, ConSERT, and USCAL frameworks by
replicating their results. The empirical outcomes, as presented in Table 3,
consistently establish the superior performance of our RobustSentEmbed
framework in contrast to various other sentence embeddings. Our framework
achieves the highest average Spearman’s correlation score when compared to
state-of-the-art approaches. Specifically, utilizing the BERT encoder, our
framework surpasses the second-best embedding method, USCAL, by a margin of
1.59%. Moreover, RobustSentEmbed achieves the highest score in the majority of
individual STS tasks, outperforming other embedding methods in 6 out of 7
tasks. For the RoBERTa encoder, RobustSentEmbed outperforms the state-of-the-
art embeddings in five out of seven STS tasks and attains the highest average
Spearman’s correlation score.

Model | MR | CR | SUBJ | MPQA | SST2 | MRPC | Avg.
---|---|---|---|---|---|---|---
GloVe embeddings (avg.) $\clubsuit$ | 77.25 | 78.30 | 91.17 | 87.85 | 80.18 | 72.87 | 81.27
Skip-thought $\heartsuit$ | 76.50 | 80.10 | 93.60 | 87.10 | 82.00 | 73.00 | 82.05
BERT-[CLS] embedding $\clubsuit$ | 78.68 | 84.85 | 94.21 | 88.23 | 84.13 | 71.13 | 83.54
ConSERT-BERTbase | 79.52 | 87.05 | 94.32 | 88.47 | 85.46 | 72.54 | 84.56
SimCSE-BERTbase | 81.29 | 86.94 | 94.72 | 89.49 | 86.70 | 75.13 | 85.71
USCAL-BERTbase | 81.54 | 87.12 | 95.24 | 89.34 | 85.71 | 75.84 | 85.80
RobustSentEmbed-BERTbase | 82.06 | 86.28 | 95.42 | 89.61 | 86.12 | 76.69 | 86.03
SimCSE-RoBERTabase | 81.15 | 87.15 | 92.38 | 86.79 | 86.24 | 75.49 | 84.87
USCAL-RoBERTabase | 82.15 | 87.22 | 92.76 | 87.74 | 84.39 | 76.20 | 85.08
RobustSentEmbed-RoBERTabase | 81.57 | 87.66 | 93.51 | 87.94 | 85.04 | 76.89 | 85.44
USCAL-RoBERTalarge | 82.84 | 87.97 | 93.12 | 88.48 | 86.28 | 76.41 | 85.85
RobustSentEmbed-RoBERTalarge | 82.56 | 88.51 | 93.84 | 88.65 | 86.18 | 77.01 | 86.13

Table 4: Results of transfer tasks for different sentence embedding models.
$\clubsuit$: results from Reimers and Gurevych (2019); $\heartsuit$: results
from Zhang et al. (2020); We emphasize the top-performing numbers among models
that share the same pre-trained encoder. All remaining results have been
reproduced and reevaluated by our team. RobustSentEmbed outperforms all other
methods, regardless of the pre-trained language model (BERTbase, RoBERTabase,
or RoBERTalarge).

###  4.4 Transfer Tasks

We leveraged transfer tasks to assess the performance of our framework,
RobustSentEmbed, across a diverse range of text classification tasks,
including sentiment analysis and paraphrase identification. Our evaluation
encompassed six transfer tasks: CR Hu and Liu (2004), SUBJ Pang and Lee
(2004), MPQA Wiebe et al. (2005), SST2 Socher et al. (2013), and MRPC Dolan
and Brockett (2005). We trained a logistic regression classifier on top of the
fixed sentence embeddings. To ensure the reliability of our findings, we
replicated the SimCSE, ConSERT, and USCAL frameworks. The outcomes, as
presented in Table 4, demonstrate the superior performance of our framework in
terms of average accuracy when compared to other sentence embeddings.
Specifically, when utilizing the BERT encoder, our framework outperforms the
second-best embedding method by a margin of 0.23%. Furthermore,
RobustSentEmbed achieves the highest score in four out of six text
classification tasks. A similar trend is observed for the RoBERTa encoder.
Overall, based on the results presented in Tables 3 and 4, we conclude that
RobustSentEmbed generates general sentence representation in addition to
robust representation (4.1 and section 4.2 ).

In conclusion, the comprehensive experiments, as indicated by the outcomes in
Tables 1, 2, 3, and 4, along with Figure 2, confirm the exceptional
performance of RobustSentEmbed in text representation and resilience against
adversarial attacks and adversarial tasks. These findings highlight the
framework’s outstanding robustness and generalization capabilities,
underscoring its potential as a versatile method for generating high-quality
sentence embeddings.

###  4.5 Distribution of Sentence Embeddings

We employed two critical metrics, alignment and uniformity Wang and Isola
(2020), for evaluating the quality of our representations. With a distribution
of positive pairs $p_{pos}$, alignment computes the expected distance between
the embeddings of paired instances:

$\ell_{\text{align}}\triangleq\mathop{\mathbb{E}}_{{(x,x^{+})}\thicksim
p_{pos}}\|f(x)-f(x^{+})\|^{2}$ (11)

Uniformity measures how well the embeddings are uniformly distributed in the
representation space:

$\ell_{\text{uniform}}\triangleq\log\mathop{\mathbb{E}}_{{x,y\stackrel{{\scriptstyle
i.i.d.}}{{\thicksim}}p_{\text{data}}}}e^{-2\|f(x)-f(y)\|^{2}}$ (12)

Figure 3 shows the uniformity and alignment of different sentence embedding
models. Smaller values indicate better performance. In comparison to the other
representations, RobustSentEmbed achieves a similar level of uniformity
(-2.295 vs. -2.305) but exhibits superior alignment (0.051 vs. 0.073). This
demonstrates that our framework is more efficient in optimizing the
representation space in two different directions.

Figure 3:  $\ell_{\text{align}}-\ell_{\text{uniform}}$ plot of models based on
BERTbase. Lower uniformity and alignment is better.

##  5 Conclusion and Future Work

This paper introduces RobustSentEmbed, a self-supervised sentence embedding
framework enhancing robustness against adversarial attacks while achieving
state-of-the-art performance in text representation and NLP tasks. Current
sentence embeddings are vulnerable to attacks, and RobustSentEmbed addresses
this by generating high-risk perturbations at token and sentence levels. These
perturbations are incorporated into novel contrastive and difference
prediction objectives. The framework is validated through comprehensive
experiments on semantic textual similarity and transfer learning tasks,
confirming its robustness against adversarial attacks and semantic similarity
tasks. In future research, we aim to investigate the use of hard negative
examples to further enhance the effectiveness of text representations.

##  6 Limitations

Despite the effectiveness of our approach and its notable performance, there
are potential limitations to our framework:

  * •

The framework is primarily tailored for descriptive models like BERT, adept at
language understanding and representation, including tasks such as text
classification. However, its direct application to generative models like GPT,
focused on generating coherent and contextually relevant text, may pose
challenges. Thus, applying our methodology to enhance generalization and
robustness in generative pre-trained models might have limitations.

  * •

Utilizing substantial GPU resources is necessary for pre-training large-scale
models like RoBERTalarge in our framework. Due to limited GPU availability, we
had to use smaller batch sizes during pre-training. Although larger batch
sizes typically result in better performance, our experiments had to
compromise and use smaller batch sizes to efficiently generate sentence
embeddings within GPU constraints.

## References

  * Agirre et al. (2015) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Inigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, et al. 2015.  Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretability.  In _Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015)_ , pages 252–263.
  * Agirre et al. (2014) Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014.  SemEval-2014 task 10: Multilingual semantic textual similarity.  In _Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014)_ , pages 81–91, Dublin, Ireland. Association for Computational Linguistics.
  * Agirre et al. (2016) Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016.  SemEval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation.  In _Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)_ , pages 497–511, San Diego, California. Association for Computational Linguistics.
  * Agirre et al. (2012) Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012.  SemEval-2012 task 6: A pilot on semantic textual similarity.  In _*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)_ , pages 385–393, Montréal, Canada. Association for Computational Linguistics.
  * Agirre et al. (2013) Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013.  *SEM 2013 shared task: Semantic textual similarity.  In _Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task: Semantic Textual Similarity_ , pages 32–43, Atlanta, Georgia, USA. Association for Computational Linguistics.
  * Bowman et al. (2015) Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015.  A large annotated corpus for learning natural language inference.  In _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_ , pages 632–642, Lisbon, Portugal. Association for Computational Linguistics.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901.
  * Cer et al. (2017) Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017.  SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation.  In _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)_ , pages 1–14, Vancouver, Canada. Association for Computational Linguistics.
  * Chen et al. (2020) Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020.  A simple framework for contrastive learning of visual representations.  In _International conference on machine learning_ , pages 1597–1607. PMLR.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020.  ELECTRA: Pre-training text encoders as discriminators rather than generators.  In _ICLR_.
  * Conneau and Kiela (2018) Alexis Conneau and Douwe Kiela. 2018.  SentEval: An evaluation toolkit for universal sentence representations.  In _Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)_ , Miyazaki, Japan. European Language Resources Association (ELRA).
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics.
  * Ding et al. (2023) Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. 2023.  Parameter-efficient fine-tuning of large-scale pre-trained language models.  _Nature Machine Intelligence_ , 5(3):220–235.
  * Dolan and Brockett (2005) William B. Dolan and Chris Brockett. 2005.  Automatically constructing a corpus of sentential paraphrases.  In _Proceedings of the Third International Workshop on Paraphrasing (IWP2005)_.
  * Gao et al. (2021) Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021.  SimCSE: Simple contrastive learning of sentence embeddings.  In _Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing_ , pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
  * Garg and Ramakrishnan (2020) Siddhant Garg and Goutham Ramakrishnan. 2020.  BAE: BERT-based adversarial examples for text classification.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6174–6181, Online. Association for Computational Linguistics.
  * Goodfellow et al. (2015) Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015.  Explaining and harnessing adversarial examples.  In _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_.
  * Hauser et al. (2023) Jens Hauser, Zhao Meng, Damian Pascual, and Roger Wattenhofer. 2023.  Bert is robust! a case against word substitution-based adversarial attacks.  In _ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , pages 1–5. IEEE.
  * He et al. (2021) Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021.  Deberta: Decoding-enhanced bert with disentangled attention.  In _International Conference on Learning Representations_.
  * Hu and Liu (2004) Minqing Hu and Bing Liu. 2004.  Mining and summarizing customer reviews.  In _Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 168–177.
  * Jin et al. (2020) Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020.  Is bert really robust? a strong baseline for natural language attack on text classification and entailment.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 34, pages 8018–8025.
  * Li et al. (2020a) Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020a.  On the sentence embeddings from pre-trained language models.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 9119–9130, Online. Association for Computational Linguistics.
  * Li et al. (2019) Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2019.  TextBugger: Generating adversarial text against real-world applications.  In _Proceedings 2019 Network and Distributed System Security Symposium_. Internet Society.
  * Li et al. (2020b) Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020b.  BERT-ATTACK: Adversarial attack against BERT using BERT.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 6193–6202, Online. Association for Computational Linguistics.
  * Li and Qiu (2021) Linyang Li and Xipeng Qiu. 2021.  Token-aware virtual adversarial training in natural language understanding.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pages 8410–8418.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.
  * Maas et al. (2011) Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011.  Learning word vectors for sentiment analysis.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics.
  * Madry et al. (2018) Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018.  Towards deep learning models resistant to adversarial attacks.  In _6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings_. OpenReview.net.
  * Marelli et al. (2014) Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014.  A sick cure for the evaluation of compositional distributional semantic models.  In _Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14)_ , pages 216–223.
  * Miao et al. (2021) Deshui Miao, Jiaqi Zhang, Wenbo Xie, Jian Song, Xin Li, Lijuan Jia, and Ning Guo. 2021.  Simple contrastive representation adversarial learning for nlp tasks.
  * Morris et al. (2020) John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. 2020.  Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pages 119–126.
  * Neelakantan et al. (2022) Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, Peter Welinder, and Lilian Weng. 2022.  Text and code embeddings by contrastive pre-training.
  * Nie et al. (2020) Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020.  Adversarial NLI: A new benchmark for natural language understanding.  In _Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics_ , pages 4885–4901, Online. Association for Computational Linguistics.
  * Pan et al. (2022) Lin Pan, Chung-Wei Hang, Avirup Sil, and Saloni Potdar. 2022.  Improved text classification via contrastive adversarial training.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 36, pages 11130–11138.
  * Pang and Lee (2004) Bo Pang and Lillian Lee. 2004.  A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts.  In _Annual Meeting of the Association for Computational Linguistics_.
  * Pang and Lee (2005) Bo Pang and Lillian Lee. 2005.  Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales.  In _Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL’05)_ , pages 115–124, Ann Arbor, Michigan. Association for Computational Linguistics.
  * Pennington et al. (2014) Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014.  GloVe: Global vectors for word representation.  In _Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)_ , pages 1532–1543, Doha, Qatar. Association for Computational Linguistics.
  * Reimers and Gurevych (2019) Nils Reimers and Iryna Gurevych. 2019.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In _Conference on Empirical Methods in Natural Language Processing_.
  * Ren et al. (2019) Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019.  Generating natural language adversarial examples through probability weighted word saliency.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 1085–1097, Florence, Italy. Association for Computational Linguistics.
  * Rima et al. (2022) Daniela N. Rima, DongNyeong Heo, and Heeyoul Choi. 2022.  Adversarial training with contrastive learning in nlp.  _Computer Speech & Language_.  Submitted.
  * Socher et al. (2013) Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013.  Recursive deep models for semantic compositionality over a sentiment treebank.  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1631–1642, Seattle, Washington, USA. Association for Computational Linguistics.
  * Su et al. (2021) Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021.  Whitening sentence representations for better semantics and faster retrieval.  _CoRR_ , abs/2103.15316.
  * Sun et al. (2019) Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019.  How to fine-tune bert for text classification?  In _China national conference on Chinese computational linguistics_ , pages 194–206. Springer.
  * Wang et al. (2021) Dong Wang, Ning Ding, Piji Li, and Hai-Tao Zheng. 2021.  Cline: Contrastive learning with semantic negative examples for natural language understanding.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing_ , pages 2332–2342. Association for Computational Linguistics.
  * Wang et al. (2023) Qian Wang, Weiqi Zhang, Tianyi Lei, Yu Cao, Dezhong Peng, and Xu Wang. 2023.  Clsep: Contrastive learning of sentence embedding with prompt.  _Knowledge-Based Systems_ , 266:110381.
  * Wang and Isola (2020) Tongzhou Wang and Phillip Isola. 2020.  Understanding contrastive representation learning through alignment and uniformity on the hypersphere.  In _International Conference on Machine Learning_ , pages 9929–9939. PMLR.
  * Wiebe et al. (2005) Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005.  Annotating expressions of opinions and emotions in language.  _Language resources and evaluation_ , 39(2):165–210.
  * Williams et al. (2018) Adina Williams, Nikita Nangia, and Samuel Bowman. 2018.  A broad-coverage challenge corpus for sentence understanding through inference.  In _Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)_ , pages 1112–1122, New Orleans, Louisiana. Association for Computational Linguistics.
  * Wu et al. (2023) Chen Wu, Ruqing Zhang, Jiafeng Guo, Maarten De Rijke, Yixing Fan, and Xueqi Cheng. 2023.  Prada: Practical black-box adversarial attacks against neural ranking models.  _ACM Transactions on Information Systems_ , 41(4):1–27.
  * Yan et al. (2021) Yuanmeng Yan, Rumei Li, Sirui Wang, Fuzheng Zhang, Wei Wu, and Weiran Xu. 2021.  ConSERT: A contrastive framework for self-supervised sentence representation transfer.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pages 5065–5075, Online. Association for Computational Linguistics.
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ , 32.
  * Zhang et al. (2015) Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015.  Character-level convolutional networks for text classification.  _Advances in neural information processing systems_ , 28:649–657.
  * Zhang et al. (2020) Yan Zhang, Ruidan He, Zuozhu Liu, Kwan Hui Lim, and Lidong Bing. 2020.  An unsupervised sentence embedding method by mutual information maximization.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing_ , pages 1601–1610. Association for Computational Linguistics.

##  Appendix A Training Details

we initialize our sentence encoder using the checkpoints obtained from BERT
Devlin et al. (2019) and RoBERTa Liu et al. (2019). RobustSentEmbed utilizes
the representation of the [CLS] token as the starting point and incorporates a
pooler layer on top of the [CLS] representations to facilitate contrastive
learning objectives. The training process of RobustSentEmbed involves 4
epochs. The best checkpoint, determined by the highest average STS score, is
selected for final evaluation. To train the model, we utilize a dataset
consisting of $10^{6}$ randomly sampled sentences from English Wikipedia, as
provided by the SimCSE framework Gao et al. (2021). The average training time
for RobustSentEmbed is 2-4 hours. As our framework is initialized with pre-
trained checkpoints, it exhibits robustness that is not sensitive to batch
sizes, thus enabling us to employ batch sizes of either 64 or 128.

##  Appendix B Ablation Studies

In this section, we conduct an analysis of the impact of five critical
hyperparameters employed in the RobustSentEmbed framework on its overall
performance. BERTbase is employed as the encoder, and the assessment of
hyperparameters is carried out using the development set of STS tasks.

Figure 4: The impact of step sizes in perturbation generation on the average
performance of STS tasks.

###  B.1 Step Sizes in Perturbation Generator

The RobustSentEmbed framework integrates two step sizes, denoted as $\alpha$
and $\beta$, to conduct iterative updates during the PGD and FGSM perturbation
generation processes, respectively. Figure 4 shows the cooperative impact of
adjusting the ranges for these two step sizes in generating high-risk
perturbations, a crucial aspect for achieving an effective contrastive
learning objective. The outcomes demonstrate more substantial improvements
when $\beta$ is fine-tuned to a lower bound, coupled with $\alpha$ set to an
upper bound. More precisely, enhanced performance is evident when $\alpha$ and
$\beta$ are allocated ranges of [1e-4, 1e-6] and [1e-3, 1e-4], respectively.
Consequently, we employ $\alpha$ = 1e-5 and $\beta$ = 1e-3 for our
experiments, as this configuration yields the optimal results among the
different configurations.

###  B.2 Step Numbers in Perturbation Generator

RobustSentEmbed employs T-step FGSM and K-step PGD iterations to acquire high-
risk adversarial perturbations for the contrastive learning objective. For
simplicity in perturbation generation analysis, we establish K = T. The
influence of varying step numbers (N = K or T) on effectiveness is illustrated
in Figure 5. A gradual improvement is observed as N increases from 1 to 12;
however, beyond N=12, the improvement becomes negligible. Additionally, higher
N results in longer running time and inequitable resource allocation.
Consequently, we opt for N=5 in our experiments.

Figure 5: The impact of the step number (represented by N = K or T) in the
T-step FGSM and K-step PGD methods on the averaged correlation of the STS
tasks.

###  B.3 Norm Constraint

To ensure imperceptibility in the generated adversarial examples,
RobustSentEmbed regulates the magnitude of the perturbation vectors (whether
$\bm{\delta}$ or $\bm{\eta}$). This control is achieved through the
utilization of three commonly employed norm functions: $L_{1}$, $L_{2}$, and
$L_{\infty}$, to restrict the magnitude of the perturbation to small values.
The averaged Spearman’s correlation of these norm functions across different
Semantic Textual Similarity tasks is presented in Table 5. The $L_{\infty}$
norm exhibits superior correlation in comparison to the other two norms, thus
warranting its selection as the norm function for our experimental assessment.

Norm | Correlation
---|---
$L_{\infty}$ | 77.90
$L2$ | 76.84
$L1$ | 76.52
Table 5: The impact of the norm constraint on perturbation generation on the
average performance of various STS tasks.

###  B.4 Contrastive Learning Loss

The first part of the total loss function (Equation 10) is dedicated to
optimizing the similarity between the input instance $x$ and its positive pair
($x^{pos}$), as well as the similarity between $x$ and its adversarial
perturbation ($x^{adv}$). While this indirectly brings $x^{pos}$ and $x^{adv}$
closer, our findings indicate that incorporating direct contrastive learning
between $x^{pos}$ and $x^{adv}$ (the second part of Equation 10) through the
regularization of the objective function in the first part helps us achieve
enhanced clean accuracy and robustness. Additionally, the third part of the
total loss function introduces the adversarial replaced token detection
objective into the loss function, making it more challenging for adversarial
training to converge. Figure 6 illustrates the impact of different values of
the weighting coefficients (i.e., $\lambda_{1}$, $\lambda_{2}$) on the final
performance of our framework. As illustrated, when $\lambda_{1}=1/128$ and
$\lambda_{2}=0.005$, the framework achieves the highest average accuracy for
semantic textual similarity tasks. We utilize $\lambda_{1}=1/128$ and
$\lambda_{2}=0.005$ for all other experiments.

Figure 6: The impact of weighting coefficients in the total loss function on
the average performance of STS tasks.

###  B.5 Modulation Factor

RobustSentEmbed includes a modulation factor, represented as $0\leq\rho\leq
1$, to adjust the relative importance of each individual perturbation (PGD and
FGSM) in the formation of the sentence-level perturbation. The efficacy of
different values of this modulation factor on semantic textual similarity
tasks is detailed in Table 6. The findings reveal that $\rho=0.5$ yields the
highest averaged correlation across the examined magnitudes, underscoring its
capability to generate more powerful perturbations. Consequently, we employ
this configuration in the setup of our framework.

$\bm{\rho}$ | Correlation
---|---
0 | 76.06
0.25 | 76.85
0.5 | 77.90
0.75 | 77.34
1 | 76.34
Table 6: The impact of the modulation factor on the average performance of
different Semantic Textual Similarity (STS) tasks in generating the final
perturbation.

##  Appendix C Adversarial Attack Methods

This section provides additional details regarding the various adversarial
attacks. The TextBugger method Li et al. (2019) identifies crucial words by
analyzing the Jacobian matrix of the target model and selects the optimal
perturbation from a set of five generated perturbations. The PWWS Ren et al.
(2019) employs a synonym-swap technique based on a combination of word
saliency scores and maximum word-swap effectiveness. TextFooler Jin et al.
(2020) identifies significant words, gathers synonyms, and replaces each such
word with the most semantically similar and grammatically correct synonym. The
BAE Garg and Ramakrishnan (2020) employs four adversarial attack strategies
involving word replacement and/or word insertion operations to generate
substitutions. The BERTAttack Li et al. (2020b) comprises two steps: (a)
identifying vulnerable words/sub-words and (b) utilizing BERT MLM to generate
semantic-preserving substitutes for the vulnerable tokens.

##  Appendix D RobustSentEmbed Algorithm

Algorithm 1 illustrates our framework’s approach to generating a norm-bounded
perturbation at both the token-level and sentence-level using an iterative
process. It confuses the $f_{\theta}(\cdot)$ encoder by treating the perturbed
embeddings as different instances. Our framework then utilizes a contrastive
learning objective in conjunction with a replaced token detection objective to
maximize the similarity between the embedding of the input sentence and the
adversarial embedding of its positive pair (former objective), as well as its
edited sentence (latter objective).

Input: Epoch number $E$, PLM Encoder $\bm{\mathbf{f_{\theta}}}$, dataset of
raw sentences $\bm{\mathrm{\mathcal{D}}}$ , embedding perturbation
{$\bm{\delta}$, $\bm{\eta}$}, dropout masks $m_{1}$ and $m_{2}$, perturbation
bound $\epsilon$, adversarial step sizes {$\alpha$, $\beta$, $\gamma$},
learning rate $\xi$, perturbation modulator $\rho$, weighting coefficients
{$\lambda_{1}$, $\lambda_{2}$}, adversarial steps {$K$, $T$}, contrastive
learning objective ${\mathcal{L}}_{con,\theta}$ (eq. 3.2), ELECTRA generator
$G(.)$ and discriminator $D(.)$

Output: Robust Sentence Representation

$\mathcal{V}\in\mathbb{R}^{N*D}\leftarrow\frac{1}{\sqrt{D}}\mathrm{U}(-\sigma,\sigma)$

for  _$\mathrm{epoch}=1,...,E$_ do

for  _minibatch $\mathrm{B}\subset\bm{\mathrm{\mathcal{D}}}$_ do

$\bm{\delta}^{0}\leftarrow\frac{1}{\sqrt{D}}\mathrm{U}(-\sigma,\sigma)\>,\>\bm{\eta}^{0}_{i}\leftarrow\mathcal{V}[w_{i}]$

$\bm{X}\>\>\>=\bm{\mathbf{f_{\theta}}}.$embedding($\mathrm{B},\>m_{1}$)

$\bm{X^{+}}=\bm{\mathbf{f_{\theta}}}.$embedding($\mathrm{B},\>m_{2}$)

for  _t = $1,...,max(K,\;T)$_ do

$\bm{g}_{\delta}=\nabla_{\delta}{\mathcal{L}}_{con,\theta}(\bm{X}+\bm{\delta}^{t-1}+\bm{\eta}^{t-1},\\{\bm{X}^{+}\\})$

if  _$t\leq K$_ then

$\bm{\delta}^{t}_{pgd}=\Pi_{\lVert\bm{\delta}\rVert_{P}\leq\epsilon}(\bm{\delta}^{t-1}+\alpha
g(\bm{\delta}^{t-1})/\lVert g(\bm{\delta}^{t-1})\rVert_{P})$

end if

if  _$t\leq T$_ then

$\bm{\delta}^{t}_{fgsm}=\Pi_{\lVert\bm{\delta}\rVert_{P}\leq\epsilon}(\bm{\delta}^{t-1}+\beta\mathrm{sign}(g(\bm{\delta}^{t-1})))$

end if

$\bm{g}_{{\eta}_{i}}=\nabla_{\eta}{\mathcal{L}}_{con,\theta}(\bm{X}+\bm{\delta}^{t-1}+\bm{\eta}^{t-1},\\{\bm{X}^{+}\\})$

$\bm{\eta}^{t}_{i}=n^{i}*(\bm{\eta}^{t}_{i-1}+\gamma\bm{g}_{{\eta}_{i}}/\lVert\bm{g}_{{\eta}_{i}}\rVert_{P})$

$\bm{\eta}^{t}\leftarrow\Pi_{\lVert\bm{\eta}\rVert_{P}\leq\epsilon}(\bm{\eta}^{t})$

end for

$\mathcal{V}[w_{i}]\leftarrow\bm{\eta}^{max(K,\>T)}_{i}$

$\bm{\delta}_{f}=\rho\bm{\delta}^{K}_{pgd}+(1-\rho)\bm{\delta}^{T}_{fgsm}$

for  _x $\in\mathrm{B}$_ do

$x^{{}^{\prime\prime}}=G(\mathrm{MLM}(x))$

$X^{adv}={X}^{{}^{\prime\prime}}+\bm{\eta}^{max(K,\>T)}_{i}$

${\mathcal{L}}^{x}_{RTD,\,\theta}=\sum_{j=1}^{\lvert
x\rvert}[\mathds{-1}(X^{adv}_{j}={X}_{j})\;\mathrm{log}\>D(X^{adv},\,\bm{\mathbf{f_{\theta}}}(x),\,j)$

$\mathds{-1}(X^{adv}_{j}\neq{X}_{j})\;\mathrm{log}\>(1-D(X^{adv},\,\bm{\mathbf{f_{\theta}}}(x),\,j))]$

end for

${\mathcal{L}}_{RTD,\,\theta}=\sum_{i=1}^{\lvert
B\rvert}{\mathcal{L}}^{x_{i}}_{RTD}$

${\mathcal{L}}_{RobustEmbed,\,\theta}:={\mathcal{L}}_{con,\,\theta}(\bm{X},\\{\bm{X^{+}},\;\bm{X}+\bm{\delta}_{f}\\})$

$\theta=\theta-\xi\nabla_{\bm{\theta}}{\mathcal{L}}_{total}$

end for

end for

Algorithm 1 RobustSentEmbed Algorithm


# Quantum Synchronization of Two Ensembles of Atoms

Minghui Xu, D. A. Tieri, E. C. Fine, James K. Thompson, and M. J. Holland
JILA, National Institute of Standards and Technology and Department of
Physics, University of Colorado, Boulder, Colorado 80309-0440, USA

###### Abstract

We propose a system for observing the correlated phase dynamics of two
mesoscopic ensembles of atoms through their collective coupling to an optical
cavity. We find a dynamical quantum phase transition induced by pump noise and
cavity output-coupling. The spectral properties of the superradiant light
emitted from the cavity show that at a critical pump rate the system undergoes
a transition from the independent behavior of two disparate oscillators to the
phase-locking that is the signature of quantum synchronization.

###### pacs:

05.45.Xt, 42.50.Lc, 37.30.+i, 64.60.Ht

Synchronization is an emergent phenomenon that describes coupled objects
spontaneously phase-locking to a common frequency in spite of differences in
their natural frequencies book1 . It was famously observed by Huygens, the
seventeenth century clock maker, in the antiphase synchronization of two
maritime pendulum clocks Huygens . Dynamical synchronization is now recognized
as ubiquitous behavior occurring in a broad range of physical, chemical,
biological, and mechanical engineering systems book1 ; book2 ; book3 .

Theoretical treatments of this phenomenon are often based on the study of
phase models kuramoto0 ; kuramoto , and as such have been applied to an
abundant variety of classical systems, including the collective blinking of
fireflies, the beating of heart cells, and audience clapping. The concept can
be readily extended to systems with an intrinsic quantum mechanical origin
such as nanomechanical resonators Cross04 ; Milburn12 , optomechanical arrays
Marquardt11 , and Josephson junctions Jain84 ; Wiesenfeld96 . When the number
of coupled oscillators is large, it has been demonstrated that the onset of
classical synchronization is analogous to a thermodynamic phase transition
Winfree67  and exhibits similar scaling behavior Oleg09 .

Recently, there has been increasing interest in exploring manifestations in
the quantum realm. Small systems have been considered, e.g., one qubit
Zhirov06  and two qubits Zhirov09  coupled to a quantum dissipative driven
oscillator, two dissipative spins Orth10 , two coupled cavities Tony12 , and
two micromechanical oscillators mian12 ; Mari12 . Connections between quantum
entanglement and synchronization have been revealed in continuous variable
systems Mari12 . It has been shown that quantum synchronization may be
achieved between two canonically conjugate variables Hriscu13 . Since the
phenomenon is inherently non-equilibrium, all of these systems share the
common property of competition between coherent and incoherent driving and
dissipative forces.

In this paper, we propose a modern-day realization of the original Huygens
experiment Huygens . We consider the synchronization of two active atomic
clocks coupled to a common single-mode optical cavity. It has been predicted
that in the regime of steady-state superradiance Meiser09 ; Meiser101 ;
Thompson12 ; Thompson121  a neutral atom lattice clock could produce an
ultracoherent optical field with a quality factor (ratio of frequency to
linewidth) that approaches $10^{18}$. We show that two such clocks may exhibit
a dynamical phase transition Zoller10 ; Zoller11 ; Cirac12 ; Cirac13  from two
disparate oscillators to quantum phase-locked dynamics. The onset of
synchronization at a critical pump strength is signified by an abruptly
increased relative phase diffusion that diverges in the thermodynamic limit.
Besides being of fundamental importance in nonequilibrium quantum many-body
physics, this work could have broad implications for many practical
applications of ultrastable lasers and precision measurements Meiser09 .

Figure 1: (color online) Two ensembles of driven two-level atoms coupled to a
single-mode cavity field. The atoms in ensemble $A$ are detuned above the
cavity resonance (dashed line). Ensemble $B$ contains atoms detuned below the
cavity resonance by an equivalent amount.

The general setup is shown schematically in Fig. 1. Two ensembles, each
containing $N$ two-level atoms with excited state $|e\rangle$ and ground state
$|g\rangle$, are collectively coupled to a high-quality optical cavity. The
transition frequencies of the atoms in ensembles $A$ and $B$ are detuned from
the cavity resonance by $\delta/2$ and $-\delta/2$ respectively. This could be
achieved by spatially separating the ensembles and applying an inhomogeneous
magnetic field to induce a differential Zeeman shift. The atoms in both
ensembles are pumped incoherently to the excited state, as could be realized
by driving a transition to a third state that rapidly decays to $|e\rangle$
Thompson12 ; Thompson121 .

This system is described by the Hamiltonian in the rotating frame of the
cavity field:

$\hat{H}=\frac{\hbar\delta}{2}(\hat{J}_{A}^{z}-\hat{J}_{B}^{z})+\frac{\hbar\Omega}{2}(\hat{a}^{\dagger}\hat{J}_{A}^{-}+\hat{J}_{A}^{+}\hat{a}+\hat{a}^{\dagger}\hat{J}_{B}^{-}+\hat{J}_{B}^{+}\hat{a})\,,$
(1)

where $\Omega$ is the atom-cavity coupling, and $\hat{a}$ and
$\hat{a}^{\dagger}$ are annihilation and creation operators for cavity
photons. Here
$\hat{J}_{A,B}^{z}=\frac{1}{2}\sum_{j=1}^{N}\hat{\sigma}_{(A,B)j}^{z}$ and
$\hat{J}_{A,B}^{-}=\sum_{j=1}^{N}\hat{\sigma}_{(A,B)j}^{-}$ are the collective
atomic spin operators, written in terms of the Pauli operators for the two-
level system $\hat{\sigma}_{(A,B)j}^{z}$ and
$\hat{\sigma}_{(A,B)j}^{-}=(\hat{\sigma}_{(A,B)j}^{+})^{\dagger}$.

In addition to the coherent atom-cavity coupling, incoherent processes are
critical and include: the cavity intensity decay at rate $\kappa$, the pump
rate $w$, the free-space spontaneous emission rate $\gamma$, and a background
dephasing of the $|e\rangle$–$|g\rangle$ transition at rate $T_{2}^{-1}$. The
total system is then described using a master equation for the reduced density
operator $\rho$:

$\displaystyle\frac{d\rho}{dt}$ $\displaystyle=$
$\displaystyle\frac{1}{i\hbar}[\hat{H},\rho]+\kappa\mathcal{L}[\hat{a}]\,\rho+\sum_{\mathcal{T}=A,B}\sum_{j=1}^{N}\Bigl{(}\gamma_{s}\mathcal{L}[\hat{\sigma}_{\mathcal{T}j}^{-}]$
(2)
$\displaystyle{}+w\mathcal{L}[\hat{\sigma}_{\mathcal{T}j}^{+}]+\frac{1}{2T_{2}}\mathcal{L}[\hat{\sigma}_{\mathcal{T}j}^{z}]\Bigr{)}\,\rho,$

where
$\mathcal{L}[\hat{O}]\,\rho=(2\hat{O}\rho\hat{O}^{\dagger}-\hat{O}^{\dagger}\hat{O}\rho-\rho\hat{O}^{\dagger}\hat{O})/2$
denotes the Lindblad superoperator.

The regime of steady-state superradiance is defined by the cavity decay being
much faster than all other incoherent processes Meiser09 ; Meiser101 ;
Thompson12 ; Thompson121 . In this regime, the cavity can be adiabatically
eliminated Meiser101 , resulting in a field that is slaved to the collective
atomic dipole of the two ensembles of atoms:

$\hat{a}\simeq-\frac{i\Omega}{\kappa+i\delta}\hat{J}_{A}^{-}-\frac{i\Omega}{\kappa-i\delta}\hat{J}_{B}^{-}.$
(3)

For small detuning on the scale of the cavity linewidth, $\delta\ll\kappa$,
Eq. (3) reduces to $\hat{a}\simeq-i\Omega\hat{J}^{-}/\kappa$, where
$\hat{J}^{-}=\hat{J}_{A}^{-}+\hat{J}_{B}^{-}$ is the total collective spin-
lowering operator. In this limit, the net effect of the cavity is to provide a
collective decay channel for the atoms, with rate
$\gamma_{c}=\Omega^{2}/\kappa$. This collective decay should be dominant over
other atomic decay processes Meiser101 , i.e.,
$N\gamma_{c}\gg\gamma_{s},T_{2}^{-1}$, so that the time evolution is
effectively given by a superradiance master equation containing only atoms:

$\frac{d\rho}{dt}=\frac{\delta}{2i\hbar}[J_{A}^{z}-J_{B}^{z},\rho]+\gamma_{c}\mathcal{L}[\hat{J}^{-}]\,\rho+w\sum_{j=1}^{N}(\mathcal{L}[\hat{\sigma}_{Aj}^{+}]+\mathcal{L}[\hat{\sigma}_{Bj}^{+}])\,\rho.$
(4)

With this system we naturally provide the three necessary ingredients for
quantum synchronization; a controllable difference between the oscillation
frequencies of two mesoscopic ensembles, a dissipative coupling generated by
the emission of photons into the same cavity mode, and a driving force
produced by optical pumping.

The photons emitted by the cavity provide directly measurable observables.
Synchronization is evident in the properties of the photon spectra. In the
case of two independent ensembles in the unsynchronized phase, each ensemble
radiates photons at its own distinct transition frequency. This leads to two
Lorentzian peaks that are typically well-separated. In the synchronized phase,
all of the atoms radiate at a common central frequency resulting in a single
peak.

To solve this problem and find the steady state, we use a semiclassical
approximation that is applicable to large atom numbers. Cumulants for the
expectation values of system operators
$\\{\hat{\sigma}_{(A,B)j}^{z},\hat{\sigma}_{(A,B)j}^{\pm}\\}$ are expanded to
second order Meiser09 ; Meiser101 . All expectation values are symmetric with
respect to exchange of atoms within each ensemble, i.e.
$\langle\hat{\sigma}_{Bi}^{+}\hat{\sigma}_{Bj}^{-}\rangle=\langle\hat{\sigma}_{B1}^{+}\hat{\sigma}_{B2}^{-}\rangle$,
for all $i\neq j$. Due to the U(1) symmetry,
$\langle\hat{\sigma}_{(A,B)j}^{\pm}\rangle=0$. Therefore, all nonzero
observables can be expressed in terms of
$\langle\hat{\sigma}_{(A,B)j}^{z}\rangle$,
$\langle\hat{\sigma}_{(A,B)i}^{+}\hat{\sigma}_{(A,B)j}^{-}\rangle$, and
$\langle\hat{\sigma}_{(A,B)i}^{z}\hat{\sigma}_{(A,B)j}^{z}\rangle$.
Expectation values involving only one ensemble are the same for both ensembles
and for these cases we omit the superfluous $A$,$B$ subscripts. The equations
of motion can then be found from Eq. (4):

$\displaystyle\frac{d}{dt}\langle\hat{\sigma}_{1}^{z}\rangle$
$\displaystyle=-\gamma_{c}\left(\langle\hat{\sigma}_{1}^{z}\rangle+1\right)-w\left(\langle\hat{\sigma}_{1}^{z}\rangle-1\right)$
(5)
$\displaystyle{}-2\gamma_{c}(N-1)\langle\hat{\sigma}_{1}^{+}\hat{\sigma}_{2}^{-}\rangle-\gamma_{c}N\left(\langle\hat{\sigma}_{A1}^{+}\hat{\sigma}_{B1}^{-}\rangle+{\rm
c.c}\right),$
$\displaystyle\frac{d}{dt}\langle\hat{\sigma}_{1}^{+}\hat{\sigma}_{2}^{-}\rangle$
$\displaystyle=-(w+\gamma_{c})\langle\hat{\sigma}_{1}^{+}\hat{\sigma}_{2}^{-}\rangle+\frac{\gamma_{c}}{2}\left(\langle\hat{\sigma}_{1}^{z}\hat{\sigma}_{2}^{z}\rangle+\langle\hat{\sigma}_{1}^{z}\rangle\right)$
(6)
$\displaystyle{}+\gamma_{c}(N-2)\langle\hat{\sigma}_{1}^{z}\rangle\langle\hat{\sigma}_{1}^{+}\hat{\sigma}_{2}^{-}\rangle$
$\displaystyle{}+\frac{\gamma_{c}}{2}N\langle\hat{\sigma}_{1}^{z}\rangle\left(\langle\hat{\sigma}_{A1}^{+}\hat{\sigma}_{B1}^{-}\rangle+{\rm
c.c}\right),$
$\displaystyle\frac{d}{dt}\langle\hat{\sigma}_{A1}^{+}\hat{\sigma}_{B1}^{-}\rangle$
$\displaystyle=-(w+\gamma_{c}-i\delta)\langle\hat{\sigma}_{A1}^{+}\hat{\sigma}_{B1}^{-}\rangle+\frac{\gamma_{c}}{2}\left(\langle\hat{\sigma}_{A1}^{z}\hat{\sigma}_{B1}^{z}\rangle+\langle\hat{\sigma}_{1}^{z}\rangle\right)$
(7)
$\displaystyle{}+\gamma_{c}(N-1)\langle\hat{\sigma}_{1}^{z}\rangle\left(\langle\hat{\sigma}_{A1}^{+}\hat{\sigma}_{B1}^{-}\rangle+\langle\hat{\sigma}_{1}^{+}\hat{\sigma}_{2}^{-}\rangle\right),$

describing population inversion, spin-spin coherence within each ensemble, and
correlation between ensembles, respectively. In deriving Eq. (6) and (7), we
have dropped third order cumulants semi . We also factorize
$\langle\hat{\sigma}_{(A,B)i}^{z}\hat{\sigma}_{(A,B)j}^{z}\rangle\approx\langle\hat{\sigma}_{1}^{z}\rangle^{2}$,
which we find to be valid outside the regime of very weak pumping where a non-
factorizable subradiant dark state plays an important role Meiser101 . After
making these approximations, Eq. (5) to (7) form a closed set of equations.
The steady state is found by setting the time derivatives to zero and the
resulting algebraic equations can be solved exactly. These solutions are the
basis for the figures shown below.

Figure 2: (color online) Steady-state relative phase precession for two
ensembles as a function of detuning at $w=N\gamma_{c}/2$ for $N=100$ (blue
dashed line), $N=500$ (purple dot dashed line) and $N=10^{6}$ (red solid
line). The straight dotted line is $\delta=\Delta$.

In order to calculate the photon spectrum, we employ the quantum regression
theorem qr  to obtain the two-time correlation function of the light field,
$\langle\hat{a}^{\dagger}(\tau)\hat{a}(0)\rangle$, where time 0 denotes an
arbitrary time-origin in steady-state. In the limit $\delta\ll\kappa$,
according to Eq. (3), the phase diffusion of the atoms and light are the same,
i.e.
$\langle\hat{a}^{\dagger}(\tau)\hat{a}(0)\rangle\sim\langle\hat{J}^{+}(\tau)\hat{J}^{-}(0)\rangle$.
We begin by deriving equations of motion for
$\langle\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{B1}^{-}(0)\rangle$ and
$\langle\hat{\sigma}_{B1}^{+}(\tau)\hat{\sigma}_{B2}^{-}(0)\rangle$:

$\frac{d}{d\tau}\begin{pmatrix}\langle\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{B1}^{-}(0)\rangle\\\
\langle\hat{\sigma}_{B1}^{+}(\tau)\hat{\sigma}_{B2}^{-}(0)\rangle\end{pmatrix}=\frac{1}{2}\begin{pmatrix}X&Y\\\
Y&X^{*}\end{pmatrix}\begin{pmatrix}\langle\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{B1}^{-}(0)\rangle\\\
\langle\hat{\sigma}_{B1}^{+}(\tau)\hat{\sigma}_{B2}^{-}(0)\rangle\end{pmatrix},$
(8)

where
$X=\gamma_{c}(N-1)\langle\hat{\sigma}_{1}^{z}(0)\rangle-\gamma_{c}-w+i\delta\,,Y=\gamma_{c}N\langle\hat{\sigma}_{1}^{z}(0)\rangle\,.$
We have systematically factorized:

$\displaystyle\langle\hat{\sigma}_{1}^{z}(\tau)\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{B1}^{-}(0)\rangle$
$\displaystyle\approx$
$\displaystyle\langle\hat{\sigma}_{1}^{z}(0)\rangle\langle\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{B1}^{-}(0)\rangle\,,$
$\displaystyle\langle\hat{\sigma}_{1}^{z}(\tau)\hat{\sigma}_{B1}^{+}(\tau)\hat{\sigma}_{B2}^{-}(0)\rangle$
$\displaystyle\approx$
$\displaystyle\langle\hat{\sigma}_{1}^{z}(0)\rangle\langle\hat{\sigma}_{B1}^{+}(\tau)\hat{\sigma}_{B2}^{-}(0)\rangle\,.$
(9)

Similarly, one finds that
$\langle\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{A2}^{-}(0)\rangle$ and
$\langle\hat{\sigma}_{B1}^{+}(\tau)\hat{\sigma}_{A1}^{-}(0)\rangle$ satisfy
the same equation of motion as Eq. (8). The solution of this coupled set is
straightforward and shows that both
$\langle\hat{\sigma}_{A1}^{+}(\tau)\hat{\sigma}_{B1}^{-}(0)\rangle$ and thus
also $\langle\hat{a}^{\dagger}(\tau)\hat{a}(0)\rangle$ evolve in proportion to
the exponential:

$\exp\left[-\frac{1}{2}\left(w+\gamma_{c}-(N-1)\gamma_{c}\langle\hat{\sigma}_{1}^{z}\rangle-\sqrt{(N\gamma_{c}\langle\hat{\sigma}_{1}^{z}\rangle)^{2}-\delta^{2}}\right)\tau\right],$
(10)

which we parametrize by $\exp\left[-(\Gamma+i\Delta)\tau/2\right]$, where
$\Gamma$ represents the decay of the first-order correlation and $\Delta$ the
modulation frequency. Laplace transformation yields the photon spectrum which
consists of Lorentzians of halfwidth $\Gamma/2$ centered at frequencies
$\pm\Delta/2$.

Figure 3: (color online) (a) Nonequilibrium phase diagram of the quantum
synchronization represented by $\Gamma$ (in units of $\gamma_{c}$) on the
$w$-$\delta$ parameter plane, where the dissipative coupling $N\gamma_{c}$
($N=10^{4}$) is fixed. An abrupt peak is observed at the boundary between the
synchronized and unsynchronized phases. (b) As for (a) but on the
$w$-$N\gamma_{c}$ parameter plane.

The importance of the two-time correlation function is that it provides direct
access to the correlated phase dynamics of the two ensembles. The parameter
$\Delta$ physically represents the precession frequency of the phase of the
collective mesoscopic dipoles with respect to one another. In Fig. 2, we show
$\Delta$ as a function of $\delta$ at $w=N\gamma_{c}/2$ for several values of
$N$. For large detuning, $\Delta$ approaches $\delta$, indicating that the
dipoles precess independently at their uncoupled frequency. Below a critical
$\delta$, we find $\Delta$ to be zero, indicating synchronization and phase
locking.

The fact that this system undergoes a synchronization transition that is
fundamentally quantum mechanical and thus quite distinct from the classical
synchronization previously discussed for coupled oscillators is evident in the
observed properties of the linewidth $\Gamma$ of the Lorenzian peak(s),
representing the relative quantum phase diffusion of the collective dipoles.
This system has three independent control variables; the detuning $\delta$,
the dissipative coupling $N\gamma_{c}$ and the pumping $w$, so we show
$\Gamma$ on the $w$-$\delta$ parameter plane in Fig. 3(a) and on the
$w$-$N\gamma_{c}$ parameter plane in Fig. 3(b).

In the region of no quantum correlation, the quantum noise due to pumping
destroys the coherences between spins faster than the collective coupling
induced by the cavity field can reestablish them. Therefore the mesoscopic
dipole is destroyed and the observed spectra are broad. In both the
synchronized and unsynchronized regions, spins within each ensemble are well-
correlated so that the corresponding Lorenzian peaks have ultranarrow
linewidth. As is apparent in Fig. 3(a), the two ensembles cannot be
synchronized when $N\gamma_{c}<\delta$ since then the coherent coupling is not
sufficient to overcome the relative precession that arises from the detuning.

For strong coupling, $N\gamma_{c}>\delta$, the synchronization transition
occurs as the pump rate passes a critical value. The two phases on either side
of the critical region are abruptly separated. As one approaches the
synchronized phase from the unsynchronized one by variation of either $\delta$
or $w$, the linewidth increases rapidly, showing amplification of the effect
of quantum noise in vicinity of the critical point. After passage of the
critical region, the linewidth drops rapidly, leading to rigid phase locking
between the two collective dipoles.

We emphasize that the synchronization dynamics shown in Fig. 2 and 3 is a
dynamical phase transition Zoller10 ; Zoller11 ; Cirac12 ; Cirac13  that is
reminiscent of a second-order quantum phase transition.

Figure 4: (color online) Finite size scaling behavior of the quantum
criticality for $\delta=N\gamma_{c}/2$. For $N\rightarrow\infty$, the critical
pump rate is $w_{c}=\delta$. The red dots show the offset between the critical
pump rate $w_{N}$ for finite $N$ and $w_{c}$. The blue squares show $\Gamma$
(in units of $\gamma_{c}$) at $w_{N}$. Both exhibit linear scalings on the
log-log plot.

To capture features of the quantum criticality, we numerically study the
finite size scaling behavior. Fig. 4 shows both the critical pump rate $w_{N}$
for finite $N$ and the corresponding $\Gamma$ at $w_{N}$. The scaling laws of
$(w_{N}-w_{c})/w_{c}\simeq N^{-0.34}$ and $\Gamma/\gamma_{c}\simeq N^{0.66}$
can be identified.

In Hamiltonian systems, a quantum phase transition results from the
competition between two noncommuting Hamiltonian components with different
symmetries on changing their relative weight. The transition between the two
distinct quantum phases can be identified from the nonanalytical behavior of
an order parameter, and the scaling behavior of certain correlation functions
that diverge at the critical point. By analogy, the synchronization phase
transition is caused by the competition between unitary dynamics that is
parametrized by $\delta$ and enters asymmetrically for the two ensembles, and
driven-dissipative dynamics parametrized by $\gamma_{c}$ that is symmetric.
The order parameter $\Delta$ is zero in the synchronized phase and non-zero in
the unsynchronized phase. The critical behavior is encapsulated by the
divergence of the relative quantum phase diffusion. It should be emphasized
that the treatment given here is quite different to the typical analysis since
the transition is embodied by the characteristic features of the two-time
correlation functions, rather than the behavior of an energy gap or
correlation length.

In the thermodynamic limit, simple expressions for
$\langle\hat{\sigma}_{1}^{z}\rangle$ to leading order in $1/N$ can be
obtained:

$\langle\hat{\sigma}_{1}^{z}\rangle=\left\\{\begin{array}[]{rl}\frac{w}{2N\gamma_{c}},&\mbox{
if $\delta=0$}\\\ \frac{w^{2}+\delta^{2}}{2wN\gamma_{c}},&\mbox{ if
$0<\delta<w$}\\\ \frac{w}{N\gamma_{c}},&\mbox{ if $\delta\geq
w$}\end{array},\right.$ (11)

where $w$ should be such that $\langle\hat{\sigma}_{1}^{z}\rangle<1$. A
critical point at $w_{c}=\delta$ can be found by substituting Eq. (11) into
Eq. (10). In particular, $\Delta=(\delta^{2}-w^{2})^{1/2}$ in the
unsynchronized phase, which shows an analogous critical exponent to that of a
second-order quantum phase transition, i.e., $\beta=1/2$.

In conclusion, we have presented a system that exhibits quantum
synchronization as a modern analogue of the Huygens experiment but is
implemented using state-of-the-art neutral atom lattice clocks of the highest
precision. It will be intriguing in future work to study the many possible
extensions that are inspired by these results, such as the effect of an atom
number imbalance on the synchronization dynamics, and the sensitivity of the
phase-locking to external perturbation.

We acknowledge stimulating discussions with J. Cooper, J. G. Restrepo, D.
Meiser, K. Hazzard, and A. M. Rey. This work has been supported by the DARPA
QuASAR program, the NSF, and NIST.

## References

  * (1) S. H. Strogatz, Sync: The Emerging Science of Spontaneous Order (Hyperion, New York, 2003).
  * (2) M. Kapitaniak, K. Czolczynski, P. Perlikowski, A. Stefanski, and T. Kapitaniak, Phys. Rep. 517, 1 (2012).
  * (3) A. Pikovsky, M. Rosenblum, and J. Kurths, Synchronization: A Universal Concept in Nonlinear Sciences (Cambridge University Press, Cambridge, England, 2001).
  * (4) S. Bregni, Synchronization of Digital Telecommunications Networks (Wiley, Chichester, 2002).
  * (5) Y. kuramoto, Chemical Oscillations, Waves and Turbulence (Courier Dover Publications, 2003).
  * (6) J. A. Acebrón et al., Rev. Mod. Phys. 77, 137 (2005).
  * (7) M. C. Cross, A. Zumdieck, Ron Lifshitz, and J. L. Rogers, Phys. Rev. Lett. 93, 224101 (2004).
  * (8) C. A. Holmes, C. P. Meaney, and G. J. Milburn, Phys. Rev. E 85, 066203 (2012).
  * (9) G. Heinrich, M. Ludwig, J. Qian, B. Kubala, and F. Marquardt, Phys. Rev. Lett. 107, 043603 (2011).
  * (10) A. K. Jain, K. K. Likharev, J. E. Lukens, and J. E. Sauvageau, Phys. Rep. 109, 309 (1984).
  * (11) K. Wiesenfeld, P. Colet, and S. H. Strogatz, Phys. Rev. Lett. 76, 404 (1996).
  * (12) A. T. Winfree, J. Theor. Biol. 16, 15 (1967).
  * (13) O. Kogan, J. L. Rogers, M. C. Cross, and G. Refael, Phys. Rev. E 80, 036206 (2009).
  * (14) O. V. Zhirov and D. L. Shepelyansky, Phys. Rev. Lett. 100, 014101 (2008).
  * (15) O. V. Zhirov and D. L. Shepelyansky, Phys. Rev. B 80, 014519 (2009).
  * (16) P. P. Orth, D. Roosen, W. Hofstetter, and K. LeHur, Phys. Rev. B 82, 144423 (2010).
  * (17) T. E. Lee and M. C. Cross, arXiv:1209.0742v1 (2012).
  * (18) M. Zhang, G. S. Wiederhecker, S. Manipatruni, A. Barnard, P. McEuen, and M. Lipson, Phys. Rev. Lett. 109, 233906 (2012).
  * (19) A. Mari, A. Farace, N. Didier, V. Giovannetti, and R. Fazio, arXiv:1304.5925v1 (2013).
  * (20) A. M. Hriscu and Y. V. Nazarov, Phys. Rev. Lett. 110, 097002 (2013).
  * (21) D. Meiser, J. Ye, D. R. Carlson, and M. J. Holland, Phys. Rev. Lett. 102, 163601 (2009).
  * (22) D. Meiser and M. J. Holland, Phys. Rev. A 81, 033847 (2010); D. Meiser and M. J. Holland, Phys. Rev. A 81, 063827 (2010).
  * (23) J. G. Bohnet, Z. Chen, J. M. Weiner, D. Meiser, M. J. Holland, and J. K. Thompson, Nature 484, 78 (2012).
  * (24) J. G. Bohnet, Z. Chen, J. M. Weiner, K. C. Cox, and J. K. Thompson, Phys. Rev. Lett. 109, 253602 (2012).
  * (25) We have validated the closed set of Eq. (5)–Eq. (7) by comparison with exact solutions of the quantum master equation based on applying the SU(4) group theory (see Minghui Xu, D. A. Tieri, M. J. Holland, Phys. Rev. A 87, 062101 (2013)). Due to the presence of multiple ensembles it is difficult to implement exact calculations for more than about ten atoms.
  * (26) S. Diehl, A. Tomadin, A. Micheli, R. Fazio, and P. Zoller, Phys. Rev. Lett. 105, 015702 (2010).
  * (27) A. Tomadin, S. Diehl, and P. Zoller, Phys. Rev. A 83, 013611 (2011).
  * (28) E. M. Kessler, G. Giedke, A. Imamoglu, S. F. Yelin, M. D. Lukin, and J. I. Cirac, Phys. Rev. A 86, 012116 (2012).
  * (29) B. Horstmann, J. I. Cirac, and G. Giedke, Phys. Rev. A 87, 012108 (2013).
  * (30) M. Lax, Phys. Rev. 129, 2342 (1963); C. W. Gardiner, _Quantum Noise_ (Springer-Verlag, Berlin, 1991).