# Energy-constrained Self-training for Unsupervised Domain Adaptation

Xiaofeng Liu1,6†, Bo Hu1,3†, Xiongchang Liu4, Jun Lu1∗, Jane You5, Lingsheng
Kong2
1 Beth Israel Deaconess Medical Center and Harvard Medical School, Boston, MA,
USA
2 Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy
of Sciences, CAS, Changchun, China
3 Illinois Institute of Technology, Chicago, IL, USA
4 Dept. of Information and Electrical Engineering, China University of Mining
and Technology, China
5 Dept. of Computing, Hong Kong Polytechnic University, Hung Hom, Hong Kong
6 Fanhan Tech. Inc., Suzhou, Jiangsu, China.
${\dagger}$ contribute equally. *Corresponding Author.

###### Abstract

Unsupervised domain adaptation (UDA) aims to transfer the knowledge on a
labeled source domain distribution to perform well on an unlabeled target
domain. Recently, the deep self-training involves an iterative process of
predicting on the target domain and then taking the confident predictions as
hard pseudo-labels for retraining. However, the pseudo-labels are usually
unreliable, and easily leading to deviated solutions with propagated errors.
In this paper, we resort to the energy-based model and constrain the training
of the unlabeled target sample with the energy function minimization
objective. It can be applied as a simple additional regularization. In this
framework, it is possible to gain the benefits of the energy-based model,
while retaining strong discriminative performance following a plug-and-play
fashion. We deliver extensive experiments on the most popular and large scale
UDA benchmarks of image classification as well as semantic segmentation to
demonstrate its generality and effectiveness.

Figure 1: The illustration of our Energy-constrained Self-training framework
for UDA. Minimizing the pseudo label-irrelevant energy of
$E_{\mathbf{w}}(\mathbf{x}_{t})$ is introduced as additional objective for the
target sample.

##  I Introduction

Deep neural networks are usually data-starved and rely on the $i.i.d$
assumption of training and testing data [1, 2, 3, 4, 5, 6, 7, 8]. However, in
reality, the deployment target tasks are usually significantly diverse, and
collecting labeled data in the target domain is expensive or even prohibitive.
For instance, densely annotating a Cityscapes image on average takes about 90
minutes [9], which severely hinders the generalization of an autonomous
driving system in different cities.

Therefore, the unsupervised domain adaptation (UDA) seeks to transfer
knowledge from one labeled source domain to another target domain with the aid
of unlabeled target data [10]. Recently, one of the promising methods in UDA
is the self-training [11, 12], which iteratively generates a set of one-hot
pseudo-labels in the target domain, and then retrains network based on these
pseudo-labels with target data. Previous research of deep self-training
usually adopts the one-hot pseudo-label, and evidence that it is essentially
an entropy minimization process [11] that pushing network output to be as
sharp as hard pseudo-label.

However, the correctness of pseudo-labels cannot be guaranteed. Trusting all
selected pseudo-labels as “ground truth” by encoding them as hard labels can
lead to overconfident mistakes and propagated errors [12]. Actually, the
state-of-the-art accuracy of many UDA benchmarks are just around 50%.
Especially at the first few epochs, it is hard to produce reliable pseudo-
label. Besides, the labels of natural images can also be highly ambiguous.
Taking a sample image from VisDA17 [13] (see Fig. 1) as an example, both
person and car dominate significant portions of this image. Enforcing a model
to be very confident in only one of the classes during training can also hurt
the learning behavior [14], particularly within the context of no ground truth
label for the target samples.

The manually defined pseudo label smoothing or entropy minimization are
proposed to make the network more conservative [12]. Essentially, our previous
work [12] is modifying the inaccurate pseudo label histogram distribution to
be more smooth following a fixed smoothing operation for any data. For
instance, revising the three-classes one-hot pseudo label $[1,0,0]$ or
$[0,1,0]$ to $[0.9,0.05,0.05]$ or $[0.05,0.9,0.05]$. However, these
constraints for pseudo-label can not be adaptively adjusted for different
inputs or network parameters.

The aforementioned issues motivate us to introduce a regularization signal for
the target sample that depends on the input and the network parameters at the
present training iteration, and not related to the inaccurate pseudo label.

Recent study [15] points out that a standard discriminative classifier can be
essentially reinterpreted as an energy-based model (EBM) for the joint
distribution $p(\mathbf{x},\mathbf{y})$ [16, 17, 18, 19]. Both the energy-
based model and supervised discriminative model can be benefited from
simultaneously optimizing both $p(\mathbf{y}|\mathbf{x})$ with cross-entropy
(CE) loss and optimizing ${\rm log}p(\mathbf{x})$ with EBM [15]. Here, we
demonstrate that optimizing ${\rm log}p(\mathbf{x})$ can be even more
promising on self-training-based UDA. Since we do not have reliable labels on
target domain, and ${\rm log}p(\mathbf{x})$ can be an idea regularization
signal, which is correlated with the input $\mathbf{x}$ and network parameter,
and independent to pseudo label $\mathbf{y}$.

Therefore, we propose a simple and straightforward idea that configures the
energy minimization of data point $\mathbf{x}$ (i.e., ${\rm
log}p(\mathbf{x})$) as an additional regularization term. The target domain
examples are optimized with pseudo-label CE loss and EBM objective. With the
help of the energy-based model, our self-training is expected to be more
controllable.

In this paper, we propose a novel and intuitive framework to incorporate the
EBM into the self-training UDA as pseudo label-irrelevant adaptive
regularization signal. We empirically validate the effectiveness and
generality of the proposed method on multiple challenging benchmarks
(classification and semantic segmentation) and achieve state-of-the-art
performance.

##  II Related Works

Unsupervised domain adaptation (UDA) with deep networks [20, 21, 22, 23, 24]
targets to learn domain invariant embeddings by minimizing the cross-domain
difference of feature distributions with certain criteria [25, 26]. Examples
of these methods include maximum mean discrepancy (MMD), deep Correlation
Alignment (CORAL), sliced Wasserstein discrepancy, adversarial learning at
input-level, feature level, output space level, etc [10, 27]. Despite the
underlying difference, there exists an interesting connection between some of
these methods with conditional forms111E.g., class-wise adversarial learning,
or discriminators taking network predictions as input. and Self-training as
they can be broadly considered as EM algorithms, and such conditional
formulation has been widely proved to benefit the adaptation.

Self-training was initially investigated in semi-supervised learning [28]. A
subtle difference between self-training with fixed feature input and deep
self-training is that the latter involves the learning of deep embedding which
renders greater flexibility towards domain alignment than classifier-level
adaptation only. Recently, there have been multiple deep self-training/pseudo-
label based methods that are proposed for UDA [29, 30, 31].

Considering the noisy pseudo-label, our previous work [12] proposes to
construct a more conservative pseudo-label that smoothing the one-hot
distribution or regularize it with the entropy. The solution proposes in this
paper is orthogonal with [12], which resorts to the additional supervision
signal of EBM that independent of pseudo-label. Compared with the manually
defined label smoothing in [12], the energy-constraint can adaptively
regularize the training w.r.t. the input and the present network parameters.
We note that our EBM regularization can be simply added on state-of-the-art
self-training methods following a plug-and-play fashion.

Energy-Based Models (EBMs) capture dependencies between variables by
associating a scalar energy to each configuration of the variables [32].
Recently, [33, 15] approximated the expectation of the log-likelihood for a
single example $\mathbf{x}$ with respect to network parameter using a sampler
based on Stochastic Gradient Langevin Dynamics (SGLD) [34].

Targeting on the combination of classifier and EBMs, [35, 36] reinterpret the
logits to define a class-conditional EBM $p(\mathbf{x}|\mathbf{y})$, which
require additional parameters to be learned to derive a classifier and an
unconditional model. [37] is similar as well but is trained using a GAN-like
generator and is applied to different applications. Based on [33], [15] scales
the training of EBMs to high-dimensional data with Contrastive Divergence and
SGLD.

Here, we regard EBMs as the pseudo label-irrelevant optimization constrains
and explore its potentials on domain adaptation, especially UDA. This helps
realize the potential of energy-based models on downstream supervised
discriminative problems. Moreover, the thorough convergence analysis and
theoretically connection with CEM are investigated.

##  III Methodology

In the UDA setting, we can access to the labeled source samples
$(\mathbf{x}_{s},\mathbf{y}_{s})$ from source domain
$\\{\mathbf{X}_{S},\mathbf{Y}_{S}\\}$, and target samples $\mathbf{x}_{t}$
from unlabeled target domain data $\mathbf{X}_{T}$. Any target label
$\hat{\mathbf{y}}_{t}=(\hat{y}_{t}^{(1)},...,\hat{y}_{t}^{(K)})$ from
$\hat{\mathbf{Y}}_{T}$ is unknown. $K$ is the total number of classes. The
parametric network $f_{\mathbf{w}}:\mathbb{R}^{D}\rightarrow\mathbb{R}^{K}$
with the weights $\mathbf{w}$ is used to process the $D$-dim input sample. The
output $K$ real-valued numbers known as logits and usually followed by a
softmax normalization to produce ${p_{\mathbf{w}}}(k|\mathbf{x})={\rm
exp}({f_{\mathbf{w}}}({\mathbf{x}})[k])/{\sum_{k\in K}}{\rm
exp}({f_{\mathbf{w}}}({\mathbf{x}})[k])$ as the classifier’s softmax
probability for class $k$. $f_{\mathbf{w}}({\mathbf{x}})[k]$ indicates the
$k^{th}$ index of $f_{\mathbf{w}}({\mathbf{x}})$, i.e., the logit
corresponding the the $k^{th}$ class label.

The recent study [15] reveals that a standard classifier can be interpreted as
an energy based model of joint distribution
$p_{\mathbf{w}}(\mathbf{x},\mathbf{y})$, and optimizing the likelihood ${\rm
log}p_{\mathbf{w}}(\mathbf{x},\mathbf{y})={\rm
log}p_{\mathbf{w}}(\mathbf{x})+{\rm log}p_{\mathbf{w}}(\mathbf{y}|\mathbf{x})$
can be helpful for both the discrimination and generation task. Specifically,
optimizing $p(\mathbf{y}|\mathbf{x})$ is simply achieved by using the standard
cross-entropy loss as conventional classification. The additional optimization
objective ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ has been proven and evidenced
that can improve the confidence calibration and robustness for conventional
classification task [15].

Considering the target samples do not have ground truth label, the self-
training methods [12] utilize the inaccurate pseudo label to calculate the
cross-entropy loss. Therefore, optimizing ${\rm
log}p_{\mathbf{w}}(\mathbf{x})$ can potentially be more helpful for UDA
setting. Actually, ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ is adaptive w.r.t.
the input $\mathbf{x}$ and network parameter $\mathbf{w}$, and irrelevant to
the inaccurate pseudo label, which can be an ideal regularizer of self-
training based UDA.

However, how to modeling ${\rm log}p_{\mathbf{w}}(\mathbf{x})$ can be a
challenging task. In EBM, the probability density $p_{\mathbf{w}}(\mathbf{x})$
for $\mathbf{x}\in\mathbb{R}^{D}$ can be expressed as
$p_{\mathbf{w}}(\mathbf{x})={\rm
exp}(-E_{\mathbf{w}}(\mathbf{x}))/Z(\mathbf{w})$. The energy based function
$E_{\mathbf{w}}(\mathbf{x}):\mathbb{R}^{D}\rightarrow\mathbb{R}$ maps each
point of an input space to a single scalar, which is called “energy”. One can
parameterize an EBM using any function that takes $\mathbf{x}$ as the input
and returns a scalar. $Z(\mathbf{w})=\int_{\mathbf{x}}{\rm
exp}(-E_{\mathbf{w}}(\mathbf{x}))$ is the normalizing constant (with respect
to $\mathbf{x}$) also known as the partition function. Usually, the normalized
densities $p_{\mathbf{w}}(\mathbf{x})$ are intractable, since we cannot
reliably estimate $Z(\mathbf{w})$ for the most choice of
$E_{\mathbf{w}}(\mathbf{x})$. Usually we rely on the sophisticate Markov Chain
Monte Carlo sampler to train EBMs.

Considering $\frac{\partial{\rm
log}p_{\mathbf{w}}(\mathbf{x})}{\partial{\mathbf{w}}}$ can be approximated
with $-\frac{\partial E_{\mathbf{w}}(\mathbf{x})}{\partial{\mathbf{w}}}$ [15],
it is possible to modeling the energy function $E_{\mathbf{w}}(\mathbf{x})$
instead of ${\rm log}p_{\mathbf{w}}(\mathbf{x})$. Following [15], we can
define an EBM of the joint distribution
$p_{\mathbf{w}}(\mathbf{x},\mathbf{y})={\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])/Z({\mathbf{w}})$, by defining
$E_{\mathbf{w}}(\mathbf{x},\mathbf{y})=-f_{\mathbf{w}}({\mathbf{x}})[k]$. By
marginalizing out $\mathbf{y}$, we have
$p_{\mathbf{w}}(\mathbf{x})=\frac{\sum_{k}{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])}{Z({\mathbf{w}})}$ [15]. Considering
$p_{\mathbf{w}}(\mathbf{x})={\rm
exp}(-E_{\mathbf{w}}(\mathbf{x}))/Z(\mathbf{w})$, the energy function of
$\mathbf{x}$ can be

$\displaystyle E_{\mathbf{w}}(\mathbf{x})=-{\rm log}\sum_{k}{\rm
exp}{(f_{\mathbf{w}}({\mathbf{x}})[k])}$ (1)

In this setting,
$p_{\mathbf{w}}(\mathbf{x}|\mathbf{y})=\frac{p_{\mathbf{w}}(\mathbf{x},\mathbf{y})}{p_{\mathbf{w}}(\mathbf{x})}=\frac{{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])/Z({\mathbf{w}})}{{\sum_{k}{\rm
exp}(f_{\mathbf{w}}({\mathbf{x}})[k])}/{Z({\mathbf{w}})}}$. The normalization
constant $Z({\mathbf{w}})$ will be canceled out and yielding the standard
softmax function, which bridges the EMB and conventional classifiers.

Therefore, we can simultaneously optimize $p(\mathbf{y}|\mathbf{x})$ with
standard cross-entropy loss, and optimize ${\rm
log}p_{\mathbf{w}}(\mathbf{x})$ with Stochastic Gradient Langevin Dynamics
(SGLD), where gradients are taken with respect to $-{\rm log}\sum_{k}{\rm
exp}{(f_{\mathbf{w}}({\mathbf{x}})[k])}$ [15].

Method | Aero | Bike | Bus | Car | Horse | Knife | Motor | Person | Plant | Skateboard | Train | Truck | Mean
---|---|---|---|---|---|---|---|---|---|---|---|---|---
Source-Res101 [38] | 55.1 | 53.3 | 61.9 | 59.1 | 80.6 | 17.9 | 79.7 | 31.2 | 81.0 | 26.5 | 73.5 | 8.5 | 52.4
MMD [39] | 87.1 | 63.0 | 76.5 | 42.0 | 90.3 | 42.9 | 85.9 | 53.1 | 49.7 | 36.3 | 85.8 | 20.7 | 61.1
DANN [40] | 81.9 | 77.7 | 82.8 | 44.3 | 81.2 | 29.5 | 65.1 | 28.6 | 51.9 | 54.6 | 82.8 | 7.8 | 57.4
ENT [41] | 80.3 | 75.5 | 75.8 | 48.3 | 77.9 | 27.3 | 69.7 | 40.2 | 46.5 | 46.6 | 79.3 | 16.0 | 57.0
MCD [42] | 87.0 | 60.9 | 83.7 | 64.0 | 88.9 | 79.6 | 84.7 | 76.9 | 88.6 | 40.3 | 83.0 | 25.8 | 71.9
ADR [43] | 87.8 | 79.5 | 83.7 | 65.3 | 92.3 | 61.8 | 88.9 | 73.2 | 87.8 | 60.0 | 85.5 | 32.3 | 74.8
DEV [44] | 81.83 | 53.48 | 82.95 | 71.62 | 89.16 | 72.03 | 89.36 | 75.73 | 97.02 | 55.48 | 71.19 | 29.17 | 72.42
TDDA [38] | 88.2 | 78.5 | 79.7 | 71.1 | 90.0 | 81.6 | 84.9 | 72.3 | 92.0 | 52.6 | 82.9 | 18.4 | 74.03
CBST [31] | 87.1$\pm$1.2 | 79.5$\pm$2.3 | 58.3$\pm$2.6 | 50.4$\pm$3.9 | 82.8$\pm$2.1 | 73.7$\pm$7.2 | 80.9$\pm$2.6 | 71.8$\pm$3.1 | 81.6$\pm$3.2 | 88.4$\pm$3.3 | 75.2$\pm$1.2 | 68.4$\pm$3.4 | 74.8$\pm$0.5
CBST+$R_{EBM}$ | 87.9$\pm$1.6 | 79.6$\pm$1.5 | 68.5$\pm$1.3 | 68.6$\pm$1.9 | 83.2$\pm$1.2 | 78.4$\pm$1.9 | 83.5$\pm$1.5 | 72.2$\pm$1.5 | 82.2$\pm$1.6 | 84.3$\pm$1.5 | 80.9$\pm$1.4 | 67.5$\pm$1.3 | 77.0$\pm$0.6
CRST[12] | 89.2$\pm$1.6 | 79.6$\pm$4.6 | 64.2$\pm$4.0 | 57.8$\pm$3.4 | 87.8$\pm$1.9 | 79.6$\pm$8.5 | 85.6$\pm$2.6 | 75.9$\pm$4.2 | 86.5$\pm$2.2 | 85.1$\pm$2.4 | 77.7$\pm$2.2 | 68.5$\pm$0.9 | 78.1$\pm$0.7
CRST+$R_{EBM}$ |  90.3$\pm$1.5 | 82.6$\pm$1.2 | 72.4$\pm$1.5 | 71.7$\pm$1.8 | 87.6$\pm$1.8 | 81.8$\pm$1.9 | 85.4$\pm$1.5 | 80.8$\pm$1.5 | 87.1$\pm$1.6 | 89.9$\pm$1.5 | 83.6$\pm$1.6 | 71.5$\pm$1.3 | 80.2$\pm$0.5
SimNet* [45] | 94.3 | 82.3 | 73.5 | 47.2 | 87.9 | 49.2 | 75.1 | 79.7 | 85.3 | 68.5 | 81.1 | 50.3 | 72.9
GTA* [46] | - | - | - | - | - | - | - | - | - | - | - | - | 77.1
CRST+$R_{EBM}$* | 93.2$\pm$1.3 | 85.8$\pm$1.2 | 73.7$\pm$1.0 | 74.3$\pm$1.5 | 89.5$\pm$0.6 | 87.6$\pm$1.6 | 88.2$\pm$1.6 | 82.2$\pm$1.6 | 90.9$\pm$1.3 | 91.6$\pm$1.8 | 85.1$\pm$1.4 | 79.9$\pm$1.4 | 82.8$\pm$0.5

TABLE I: Experimental results for VisDA17-val setting. We use ResNet101 as
backbone except SimNet and GTA.*ResNet152 backbone.

###  III-A EBM as regularization for UDA

Self-training for UDA is an iterative loss minimization framework [12], which
regards pseudo-labels as learnable latent variables. The optimization
objective for unlabeled target example is the cross-entropy with one-hot [31]
or smoothed [12] pseudo-label. Therefore, a straightforward solution for
adapting EBM to UDA is to incorporate Eq. 1 as a regularization term. For the
target sample, it not only needs to achieve a good prediction of pseudo-label,
but also minimize the energy of $E_{\mathbf{w}}(\mathbf{x}_{t})$. Therefore,
our energy-constraint is essentially depended on $\mathbf{x}$ and
$\mathbf{w}$, which is more flexible than pre-defined label smoothing or
entropy regularization [12]. Considering that the pseudo-label is usually
noisy, the latter objective is expected to be even more important than the
setting of supervised learning.

Following the formulation in our CRST [12], the self-training with EBM
regularization (R-EBM) for target sample, i.e.,
$E_{\mathbf{w}}(\mathbf{x}_{t})$, can be formulated as

$\displaystyle\underset{\mathbf{w},{{{\hat{\mathbf{Y}}}}_{T}}}{\mathop{\min}}\,\mathcal{L}_{R-EBM}(\mathbf{w},{\hat{\mathbf{Y}}})=-\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{s}})}$
$\displaystyle-\sum\limits_{{{t}}\in{{T}}}\\{{\sum\limits_{k=1}^{K}{[\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}-\hat{y}_{t}^{(k)}{{\log\lambda}_{k}}]-\alpha
E_{\mathbf{w}}(\mathbf{x}_{t})\\}$ $\displaystyle
s.t.~{}{{{\hat{\mathbf{y}}}}_{t}}\in\Delta^{K-1}\cup\\{\mathbf{0}\\},\forall
t$ (2)

For each class $k$, $\lambda_{k}$ is determined by the confidence value
selecting the most confident $p$ portion of class $k$ predictions in the
entire target set [31]. If a sample’s predication is relatively confident with
$p_{\mathbf{w}}(k^{*}|\mathbf{x}_{t})>\lambda_{k^{*}}$, it is selected and
labeled as class
$k^{*}=\text{argmax}_{k}\\{\frac{p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}{\lambda_{k}}\\}$.
The less confident ones with
$p_{\mathbf{w}}(k^{*}|\mathbf{x}_{t})\leq\lambda_{k^{*}}$ are not selected.
The same class-balanced $\lambda_{k}$ strategy introduced in [31] is adopted
for all self-training methods in this work.

The feasible set is the union of $\\{\mathbf{0}\\}$ and a probability simplex
$\Delta^{K-1}$ [31]. $\alpha$ is a balancing hyper-parameter of the
regularization term, which does not directly relate to the label
$\hat{y}_{t}$. The self-training can be solved by an alternating optimization
scheme.

Step 1) Pseudo-label generation Fix $\mathbf{w}$ and solve:

$\displaystyle\underset{{{{\hat{\mathbf{Y}}}}_{T}}}{\mathop{\min}}-\sum\limits_{{{t}}\in{{T}}}\\{{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}[\log{p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}-\log{{{\lambda}_{k}}}]}-\alpha
E_{\mathbf{w}}(\mathbf{x}_{t})\\}$ $\displaystyle s.t.\text{
}{{{\hat{y}}}_{t}}\in\Delta^{K-1}\cup\\{\mathbf{0}\\},\forall t$ (3)

For solving step 1), there is a global optimizer for arbitrary
$\hat{\mathbf{y}}_{t}=(\hat{y}_{t}^{(1)},...,\hat{y}_{t}^{(K)})$ as [12]:

$\hat{y}_{t}^{(k)*}=\left\\{\begin{aligned} 1,&~{}\text{if}~{}k={~{}}^{\rm
argmax}_{~{}~{}~{}~{}k}~{}{\frac{p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}{\lambda_{k}}}\\\
&~{}~{}~{}~{}~{}\text{and}~{}~{}~{}p_{\mathbf{w}}(k|\mathbf{x}_{t})>\lambda_{k}\\\
0,&~{}\mathrm{otherwise}\end{aligned}\right.$ (4)

Despite a long period of little development, there has been recent work [36,
33] using the sampler based on SGLD to train the large-scale EBMs on high-
dimensional data, parameterized by deep neural networks.

Given that the goal of our work is to incorporate EBM training into the
standard classification setting, the classification part is the same as [12],
and we only change the regularizer. Therefore, we can follow [15] to train the
network with both cross-entropy loss and SGLD to ensure this distribution is
being optimized with an unbiased objective. Similar to [36] we also adopt the
contrastive divergence to estimate the expectation of derivative of the log-
likelihood for a single example $\mathbf{x}$ with respect to $\mathbf{w}$.
Since it gives an order of magnitude savings in computation compared to
seeding new chains at each iteration as in [33].

Step 2) Network retraining Fix $\hat{\mathbf{Y}}_{T}$ and minimize

$\displaystyle\tiny-\sum\limits_{{{s}}\in{{S}}}{\sum\limits_{k=1}^{K}{y_{s}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{s}})}-\sum\limits_{{{t}}\in{{T}}}{\sum\limits_{k=1}^{K}{\hat{y}_{t}^{(k)}}\log
p_{\mathbf{w}}(k|{\mathbf{x}_{t}})}$ (5)

w.r.t. $\mathbf{w}$. Carrying out step 1) and 2) for one time is defined as
one round in self-training.

##  IV Experiments

Method | Base Net | Road | SW | Build | Wall | Fence | Pole | TL | TS | Veg. | Terrain | Sky | PR | Rider | Car | Truck | Bus | Train | Motor | Bike | mIoU
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
Source | DRN26 | 42.7 | 26.3 | 51.7 | 5.5 | 6.8 | 13.8 | 23.6 | 6.9 | 75.5 | 11.5 | 36.8 | 49.3 | 0.9 | 46.7 | 3.4 | 5.0 | 0.0 | 5.0 | 1.4 | 21.7
CyCADA [47] |  | 79.1 | 33.1 | 77.9 | 23.4 | 17.3 | 32.1 | 33.3 | 31.8 | 81.5 | 26.7 | 69.0 | 62.8 | 14.7 | 74.5 | 20.9 | 25.6 | 6.9 | 18.8 | 20.4 | 39.5
Source | DRN105 | 36.4 | 14.2 | 67.4 | 16.4 | 12.0 | 20.1 | 8.7 | 0.7 | 69.8 | 13.3 | 56.9 | 37.0 | 0.4 | 53.6 | 10.6 | 3.2 | 0.2 | 0.9 | 0.0 | 22.2
MCD [42] |  | 90.3 | 31.0 | 78.5 | 19.7 | 17.3 | 28.6 | 30.9 | 16.1 | 83.7 | 30.0 | 69.1 | 58.5 | 19.6 | 81.5 | 23.8 | 30.0 | 5.7 | 25.7 | 14.3 | 39.7
Source | PSPNet | 69.9 | 22.3 | 75.6 | 15.8 | 20.1 | 18.8 | 28.2 | 17.1 | 75.6 | 8.00 | 73.5 | 55.0 | 2.9 | 66.9 | 34.4 | 30.8 | 0.0 | 18.4 | 0.0 | 33.3
DCAN [48] |  | 85.0 | 30.8 | 81.3 | 25.8 | 21.2 | 22.2 | 25.4 | 26.6 | 83.4 | 36.7 | 76.2 | 58.9 | 24.9 | 80.7 | 29.5 | 42.9 | 2.50 | 26.9 | 11.6 | 41.7
Source | DeepLabv2 | 75.8 | 16.8 | 77.2 | 12.5 | 21.0 | 25.5 | 30.1 | 20.1 | 81.3 | 24.6 | 70.3 | 53.8 | 26.4 | 49.9 | 17.2 | 25.9 | 6.5 | 25.3 | 36.0 | 36.6
AdaptSegNet [49] |  | 86.5 | 36.0 | 79.9 | 23.4 | 23.3 | 23.9 | 35.2 | 14.8 | 83.4 | 33.3 | 75.6 | 58.5 | 27.6 | 73.7 | 32.5 | 35.4 | 3.9 | 30.1 | 28.1 | 42.4
AdvEnt [50] | DeepLabv2 | 89.4 | 33.1 | 81.0 | 26.6 | 26.8 | 27.2 | 33.5 | 24.7 | 83.9 | 36.7 | 78.8 | 58.7 | 30.5 | 84.8 | 38.5 | 44.5 | 1.7 | 31.6 | 32.4 | 45.5
Source | DeepLabv2 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 29.2
FCAN [51] |  | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | 46.6
Source | DeepLabv2 | 75.8 | 16.8 | 77.2 | 12.5 | 21.0 | 25.5 | 30.1 | 20.1 | 81.3 | 24.6 | 70.3 | 53.8 | 26.4 | 49.9 | 17.2 | 25.9 | 6.5 | 25.3 | 36.0 | 36.6
DPR [52] |  | 92.3 | 51.9 | 82.1 | 29.2 | 25.1 | 24.5 | 33.8 | 33.0 | 82.4 | 32.8 | 82.2 | 58.6 | 27.2 | 84.3 | 33.4 | 46.3 | 2.2 | 29.5 | 32.3 | 46.5
Source | DeepLabv2 | 73.8 | 16.0 | 66.3 | 12.8 | 22.3 | 29.0 | 30.3 | 10.2 | 77.7 | 19.0 | 50.8 | 55.2 | 20.4 | 73.6 | 28.3 | 25.6 | 0.1 | 27.5 | 12.1 | 34.2
PyCDA [53] |  | 90.5 | 36.3 | 84.4 | 32.4 | 28.7 | 34.6 | 36.4 | 31.5 | 86.8 | 37.9 | 78.5 | 62.3 | 21.5 | 85.6 | 27.9 | 34.8 | 18.0 | 22.9 | 49.3 | 47.4
Source | DeepLabv2 | 71.3 | 19.2 | 69.1 | 18.4 | 10.0 | 35.7 | 27.3 | 6.8 | 79.6 | 24.8 | 72.1 | 57.6 | 19.5 | 55.5 | 15.5 | 15.1 | 11.7 | 21.1 | 12.0 | 33.8
CBST [31] |  | 89.9 | 55.0 | 79.9 | 29.5 | 20.6 | 37.8 | 32.9 | 13.9 | 84.0 | 31.2 | 75.5 | 60.2 | 27.1 | 81.8 | 29.7 | 40.5 | 7.62 | 28.7 | 41.4 | 45.6
CBST+$R_{EBM}$ |  | 91.1 | 53.9 | 80.6 | 31.6 | 21.0 | 40.4 | 35.0 | 19.8 | 86.8 | 35.9 | 76.4 | 63.3 | 31.4 | 83.0 | 22.5 | 38.6 | 24.2 | 32.2 | 39.4 | 47.8
Source | DeepLabv2 | 71.3 | 19.2 | 69.1 | 18.4 | 10.0 | 35.7 | 27.3 | 6.8 | 79.6 | 24.8 | 72.1 | 57.6 | 19.5 | 55.5 | 15.5 | 15.1 | 11.7 | 21.1 | 12.0 | 33.8
CRST [12] |  | 89.0 | 51.2 | 79.4 | 31.7 | 19.1 | 38.5 | 34.1 | 20.4 | 84.7 | 35.4 | 76.8 | 61.3 | 30.2 | 80.7 | 27.4 | 39.4 | 10.2 | 32.2 | 43.3 | 46.6
CRST+$R_{EBM}$ |  | 92.5 | 56.6 | 80.9 | 26.2 | 20.5 | 40.5 | 35.3 | 24.4 | 86.9 | 37.3 | 77.5 | 63.4 | 30.5 | 81.3 | 28.8 | 39.2 | 24.6 | 33.5 | 41.3 | 48.5

TABLE II: Experimental results for GTA5 to Cityscapes.
Figure 2: Sensitive analysis of hyper-parameter $\alpha$ in VisDA17 (left) and
CTA52Sityscapes (right) with CRST+$R_{EBM}$.

In this section, we provide comprehensive evaluations of the proposed EBM
objective both on image classification and semantic segmentation UDA tasks. We
implement our methods using the PyTorch toolbox [54].

###  IV-A Domain Adaptation for Image Classification

The VisDA17 [13] benchmark is a 12-class UDA classification problem. We follow
the standard protocol in [12, 46] where the source domain is the training set
including $152,409$ synthetic 2D images and the target domain is the
validation set including $55,400$ real images from COCO dataset.

To make a fair comparison with other methods, we use the same backbone network
(e.g., ResNet101 or ResNet152) for VisDA17. Both networks are pre-trained in
ImageNet as previous works and fine-tuned in source domain by SGD with a fixed
learning rate $1\times 10^{-3}$, weight decay $5\times 10^{-4}$, momentum
$0.9$ and batch size $32$. As shown in Fig. 2, the performance is not
sensitive to $\alpha$ when $\alpha\in[0.8,1.1]$, we simply choose $\alpha=1$
in all settings.

We present the results on VisDA 17 in Table I in terms of per-class accuracy
and mean accuracy. For each proposed approach, we independently run 5 times
and the average and standard deviation of the accuracy metrics are reported.

Class-balanced self-training (CBST) and Conservative regularized self-training
(CRST) [12] are the state-of-the-art self-training UDA methods. The
CRST+$R_{EBM}$ denote the CRST with EBM regularization.

Benefited by the additional EMB objective, CBST/CRST +$R_{EBM}$ can outperform
CBST/CRST significantly, and outperforms the recent UDA methods other than
self-training by a large margin. We note that the adversarial training can
further boost the performance of self-training [12]. Our energy constraint can
adaptively regularize the training w.r.t. the input $\mathbf{x}$ and the
present network parameters, which is more flexible than the manually pre-
defined label smoothing [12].

More appealingly, CRST+$R_{EBM}$ achieves the better performance than the
recent adversarial training [38], dropout [55] and moment matching methods,
which revokes the potential of self-training in UDA.

The more powerful backbones have also been applied and show better results
[45, 46]. The EBM objective with ResNet152 backbone outperforms the other
state-of-the-arts [45, 46]. It also demonstrate the flexibility of $R_{EBM}$
for different backbones.

Actually, our $R_{EBM}$ can be orthogonal with the recent progress of self-
training based UDA. CBST [31] is a pioneer of vanilla adversarial UDA, and its
performance on VisDA17 is reported on [12]. The label smoothing or the entropy
regularization used in CRST [12] or the vanilla self-training UDA [31] can be
simply add-on our $R_{EBM}$ to further improve the performance. CRST+$R_{EBM}$
obtains better or competitive performances in all settings, even compared with
a generative pixel-level domain adaptation method GTA, which is a very complex
algorithm in both architecture and objectives.

###  IV-B UDA for Semantic Segmentation

The semantic segmentation is essentially making the pixel-wise classification.
We consider the challenging segmentation adaptation settings in GTA5 [56] to
Cityscapes (19 classes are shared). GTA5 dataset includes 24,966 annotated
images with size 1,052$\times$1,914, which rendered by the GTA5 game engine.
Following the standard protocols [47, 49], we use the full set of GTA5 and
adapt the model to the Cityscapes train set with $2,975$ images. In testing,
we evaluate on the Cityscapes validation set with $500$ images.

To make a fair comparison with other methods, we use the ResNet101 as the
backbone network as [31, 12]. Noticing that in PSPNET [57], Wide ResNet38 is a
stronger basenet than ResNet101. The basenet is pre-trained in ImageNet and
fine-tuned in source domain by SGD with learning rate $2.5\times 10^{-4}$,
weight decay $5\times 10^{-4}$, momentum $0.9$, batch size $2$, patch size
$512\times 1024$ and data augmentation of multi-scale training ($0.5-1.5$) and
horizontal flipping. All results on this dataset in the main paper are unified
to report the mIoU of the models at the end of the $6^{th}$ epoch.

We compare CBST/CRST+$R_{EBM}$ with the other methods in Table II. Based on
the previous self-training UDA methods CBST or CRST, our additional EBM
objective outperforms CBST or CRST by about 2% w.r.t. the mean IoUs.

We achieve a new state-of-the-art, even compared with the generative pixel-
level domain adaptation method GTA [51], which is a relatively complex
algorithm in both architecture and objectives.

Introducing the EBM objective to self-training methods can significantly
improve the segmentation performance. Consistently with the classification,
$R_{EBM}$ can be better or comparable with adversarial learning, indicating
the self-training can still be a powerful methodology in UDA. Note that moment
matching-based methods are usually not well scaleable to segmentation.

##  V Conclusions

In this paper, we propose to introduce the energy based model’s objective into
the self-training unsupervised domain adaptation. Considering the lack of
target label, we resort to the pseudo-label which is usually noisy. The EBM
optimization objective provides additional signal that is independent of
pseudo-label. It can be even more promising in UDA setting than supervised
learning. It can be a regularization term. Our solution is orthogonal with the
recent progress of self-training and can be added on in a plug-and-play manner
without introducing large computation and the change of network structure.
Extensive experiments on both UDA classification and semantic segmentation
evidenced its effectiveness and generality. The more advanced EBM training and
self-training methods [30] can also be adopted to further improve the results.

##  VI Acknowledgements

This work was supported by the Jangsu Youth Programme [BK20200238], National
Natural Science Foundation of China, Younth Programme [grant number 61705221],
NIH [NS061841, NS095986], Fanhan Technology, and Hong Kong Government General
Research Fund GRF (Ref. No.152202/14E) are greatly appreciated.

## References

  * [1] T. Che, X. Liu, S. Li, Y. Ge, R. Zhang, C. Xiong, and Y. Bengio, “Deep verifier networks: Verification of deep discriminative models with deep generative models,” _arXiv preprint arXiv:1911.07421_ , 2019.
  * [2] X. Liu, Y. Lu, X. Liu, S. Bai, S. Li, and J. You, “Wasserstein loss with alternative reinforcement learning for severity-aware semantic segmentation,” _IEEE Transactions on Intelligent Transportation Systems_ , 2020.
  * [3] X. Liu, F. Fan, L. Kong, Z. Diao, W. Xie, J. Lu, and J. You, “Unimodal regularized neuron stick-breaking for ordinal classification,” _Neurocomputing_ , 2020.
  * [4] X. Liu, Y. Han, S. Bai, Y. Ge, T. Wang, X. Han, S. Li, J. You, and J. Lu, “Importance-aware semantic segmentation in self-driving with discrete wasserstein training.” in _AAAI_ , 2020, pp. 11 629–11 636.
  * [5] X. Liu, W. Ji, J. You, G. E. Fakhri, and J. Woo, “Severity-aware semantic segmentation with reinforced wasserstein training,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 12 566–12 575.
  * [6] G. He, X. Liu, F. Fan, and J. You, “Image2audio: Facilitating semi-supervised audio emotion recognition with facial expression image,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 912–913.
  * [7] Y. Han, X. Liu, Z. Sheng, Y. Ren, X. Han, J. You, R. Liu, and Z. Luo, “Wasserstein loss-based deep object detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 998–999.
  * [8] X. Liu, T. Che, Y. Lu, C. Yang, S. Li, and J. You, “Auto3d: Novel view synthesis through unsupervisely learned variational viewpoint and global 3d representation,” _arXiv preprint arXiv:2007.06620_ , 2020.
  * [9] M. Cordts, M. Omran, S. Ramos, T. Rehfeld, M. Enzweiler, R. Benenson, U. Franke, S. Roth, and B. Schiele, “The cityscapes dataset for semantic urban scene understanding,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 3213–3223.
  * [10] W. M. Kouw, “An introduction to domain adaptation and transfer learning,” _arXiv_ , 2018.
  * [11] D.-H. Lee, “Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” in _ICML Workshop on Challenges in Representation Learning_ , 2013.
  * [12] Y. Zou, Z. Yu, X. Liu, B. Kumar, and J. Wang, “Confidence regularized self-training,” _ICCV_ , 2019.
  * [13] X. Peng, B. Usman, N. Kaushik, D. Wang, J. Hoffman, K. Saenko, X. Roynard, J.-E. Deschaud, F. Goulette, T. L. Hayes _et al._ , “Visda: A synthetic-to-real benchmark for visual domain adaptation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , 2018, pp. 2021–2026.
  * [14] H. Bagherinezhad, M. Horton, M. Rastegari, and A. Farhadi, “Label refinery: Improving imagenet classification through label progression,” _arXiv preprint arXiv:1805.02641_ , 2018.
  * [15] W. Grathwohl, K. Wang, J. Jacobsen, D. Duvenaud, M. Norouzi, and K. Swersky, “Your classifier is secretly an energy based model and you should treat it like one,” _arXiv_ , 2019.
  * [16] X. Liu, B. Vijaya Kumar, C. Yang, Q. Tang, and J. You, “Dependency-aware attention control for unconstrained face recognition with image sets,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 548–565.
  * [17] X. Liu, Z. Guo, J. You, and B. Kumar, “Attention control with metric learning alignment for image set-based recognition,” _arXiv preprint arXiv:1908.01872_ , 2019.
  * [18] X. Liu, Z. Guo, S. Li, L. Kong, P. Jia, J. You, and B. Kumar, “Permutation-invariant feature restructuring for correlation-aware image set-based recognition,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 4986–4996.
  * [19] X. Liu, Z. Guo, J. You, and B. V. Kumar, “Dependency-aware attention control for image set-based face recognition,” _IEEE Transactions on Information Forensics and Security_ , vol. 15, pp. 1501–1512, 2019.
  * [20] G. He, X. Liu, F. Fan, and J. You, “Classification-aware semi-supervised domain adaptation,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2020, pp. 964–965.
  * [21] X. Liu, X. Han, Y. Qiao, Y. Ge, S. Li, and J. Lu, “Unimodal-uniform constrained wasserstein training for medical diagnosis,” in _Proceedings of the IEEE International Conference on Computer Vision Workshops_ , 2019, pp. 0–0.
  * [22] X. Liu, Y. Zou, T. Che, P. Ding, P. Jia, J. You, and B. Kumar, “Conservative wasserstein training for pose estimation,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019, pp. 8262–8272.
  * [23] X. Liu, Y. Zou, Y. Song, C. Yang, J. You, and B. K Vijaya Kumar, “Ordinal regression with neuron stick-breaking for medical diagnosis,” in _Proceedings of the European Conference on Computer Vision (ECCV)_ , 2018, pp. 0–0.
  * [24] X. Liu, Y. Zou, L. Kong, Z. Diao, J. Yan, J. Wang, S. Li, P. Jia, and J. You, “Data augmentation via latent space interpolation for image classification,” in _2018 24th International Conference on Pattern Recognition (ICPR)_. IEEE, 2018, pp. 728–733.
  * [25] X. Liu, L. Kong, Z. Diao, and P. Jia, “Line-scan system for continuous hand authentication,” _Optical Engineering_ , vol. 56, no. 3, p. 033106, 2017\.
  * [26] X. Liu, Z. Li, L. Kong, Z. Diao, J. Yan, Y. Zou, C. Yang, P. Jia, and J. You, “A joint optimization framework of low-dimensional projection and collaborative representation for discriminative classification,” in _2018 24th International Conference on Pattern Recognition (ICPR)_. IEEE, 2018, pp. 1493–1498.
  * [27] X. Liu, S. Li, L. Kong, W. Xie, P. Jia, J. You, and B. Kumar, “Feature-level frankenstein: Eliminating variations for discriminative recognition,” in _CVPR_ , 2019.
  * [28] I. Triguero, S. García, and F. Herrera, “Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study,” _Knowledge and Information Systems_ , vol. 42, no. 2, pp. 245–284, 2015.
  * [29] P. P. Busto, A. Iqbal, and J. Gall, “Open set domain adaptation for image and action recognition,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , 2018.
  * [30] L. Han, Y. Zou, R. Gao, L. Wang, and D. Metaxas, “Unsupervised domain adaptation via calibrating uncertainties,” in _CVPR Workshops_ , 2019.
  * [31] Y. Zou, Z. Yu, B. V. K. Vijaya Kumar, and J. Wang, “Unsupervised domain adaptation for semantic segmentation via class-balanced self-training,” in _ECCV_ , 2018.
  * [32] Y. LeCun, S. Chopra, R. Hadsell, M. Ranzato, and F. Huang, “A tutorial on energy-based learning,” _Predicting structured data_ , vol. 1, no. 0, 2006\.
  * [33] E. Nijkamp, S.-C. Zhu, and Y. N. Wu, “On learning non-convergent short-run mcmc toward energy-based model,” _arXiv_ , 2019.
  * [34] M. Welling and Y. W. Teh, “Bayesian learning via stochastic gradient langevin dynamics,” in _ICML_ , 2011.
  * [35] J. Xie, Y. Lu, S.-C. Zhu, and Y. Wu, “A theory of generative convnet,” in _ICML_ , 2016, pp. 2635–2644.
  * [36] Y. Du and I. Mordatch, “Implicit generation and generalization in energy-based models,” _arXiv_ , 2019.
  * [37] Y. Song and Z. Ou, “Learning neural random fields with inclusive auxiliary generators,” _arXiv_ , 2018.
  * [38] B. Gholami, P. Sahu, M. Kim, and V. Pavlovic, “Task-discriminative domain alignment for unsupervised domain adaptation,” in _ICCV_ , 2019.
  * [39] M. Long, Y. Cao, J. Wang, and M. I. Jordan, “Learning transferable features with deep adaptation networks,” in _ICML_ , 2015.
  * [40] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” _The Journal of Machine Learning Research_ , vol. 17, no. 1, pp. 2096–2030, 2016.
  * [41] Y. Grandvalet and Y. Bengio, “Semi-supervised learning by entropy minimization,” in _NIPS_ , 2005.
  * [42] K. Saito, K. Watanabe, Y. Ushiku, and T. Harada, “Maximum classifier discrepancy for unsupervised domain adaptation,” 2017.
  * [43] K. Saito, Y. Ushiku, T. Harada, and K. Saenko, “Adversarial dropout regularization,” in _ICLR_ , 2018.
  * [44] K. You, X. Wang, M. Long, and M. Jordan, “Towards accurate model selection in deep unsupervised domain adaptation,” in _ICML_. ACM, 2019.
  * [45] P. O. Pinheiro, “Unsupervised domain adaptation with similarity learning,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 8004–8013.
  * [46] S. Sankaranarayanan, Y. Balaji, C. D. Castillo, and R. Chellappa, “Generate to adapt: Aligning domains using generative adversarial networks,” in _CVPR_ , 2018.
  * [47] J. Hoffman, E. Tzeng, T. Park, J.-Y. Zhu, P. Isola, K. Saenko, A. A. Efros, and T. Darrell, “Cycada: Cycle-consistent adversarial domain adaptation,” in _ICML_ , 2018.
  * [48] Z. Wu, X. Han, Y.-L. Lin, M. Gokhan Uzunbas, T. Goldstein, S. Nam Lim, and L. S. Davis, “Dcan: Dual channel-wise alignment networks for unsupervised scene adaptation,” in _The European Conference on Computer Vision (ECCV)_ , September 2018.
  * [49] Y.-H. Tsai, W.-C. Hung, S. Schulter, K. Sohn, M.-H. Yang, and M. Chandraker, “Learning to adapt structured output space for semantic segmentation,” in _CVPR_ , 2018.
  * [50] T.-H. Vu, H. Jain, M. Bucher, M. Cord, and P. Pérez, “Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation,” in _CVPR_ , 2019.
  * [51] Y. Zhang, Z. Qiu, T. Yao, D. Liu, and T. Mei, “Fully convolutional adaptation networks for semantic segmentation,” in _CVPR_ , 2018.
  * [52] Y.-H. Tsai, K. Sohn, S. Schulter, and M. Chandraker, “Domain adaptation for structured output via discriminative representations,” _ICCV_ , 2019.
  * [53] Q. Lian, F. Lv, L. Duan, and B. Gong, “Constructing self-motivated pyramid curriculums for cross-domain semantic segmentation: A non-adversarial approach,” _ICCV_ , 2019.
  * [54] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer, “Automatic differentiation in pytorch,” 2017.
  * [55] K. Saito, Y. Ushiku, T. Harada, and K. Saenko, “Adversarial dropout regularization,” _arXiv preprint arXiv:1711.01575_ , 2017.
  * [56] S. R. Richter, V. Vineet, S. Roth, and V. Koltun, “Playing for data: Ground truth from computer games,” in _ECCV_ , 2016.
  * [57] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 2881–2890.


# Secure Spatial Signal Design for ISAC in a Cell-Free MIMO Network

Steven Rivetti, Emil Björnson, Mikael Skoglund,
School of Electrical Engineering and Computer Science (EECS), KTH Royal
Institute of Technology, Sweden

###### Abstract

In this paper, we study a cell-free multiple-input multiple-output network
equipped with integrated sensing and communication (ISAC) access points (APs).
The distributed APs are used to jointly serve the communication needs of user
equipments (UEs) while sensing a target, assumed to be an eavesdropper (Eve).
To increase the system’s robustness towards said Eve, we develop an ISAC
waveform model that includes artificial noise (AN) aimed at degrading the Eve
channel quality. The central processing unit receives the observations from
each AP and calculates the optimal precoding and AN covariance matrices by
solving a semi-definite relaxation of a constrained Cramer-Rao bound (CRB)
minimization problem. Simulation results highlight an underlying trade-off
between sensing and communication performances: in particular, the UEs signal-
to-noise and interference ratio and the maximum Eve’s signal to noise ratio
are directly proportional to the CRB. Furthermore, the optimal AN covariance
matrix is rank-1 and has a peak in the eve’s direction, leading to a
surprising inverse-proportionality between the UEs-Eve distance and optimal-
CRB magnitude.

###### Index Terms:

Cell-free MIMO, ISAC, CRB, signal design.

##  I Introduction

6G networks and other future communication systems are expected to include
sensing functionalities as a fundamental component [1]. Assuming an efficient
sharing of hardware and wireless resources, the communication infrastructure
can have low-cost sensing capabilities and the sensing frequency bands can be
used for wireless communications. To do that, however, the integrated sensing
and communications (ISAC) system’s different components, including the
transmission waveform, received data post-processing and the multiple-input
multiple-output (MIMO) beamforming must be carefully co-designed. The single
ISAC access point (AP) case has been the main focus of earlier studies [2]. In
reality, however, numerous ISAC APs will operate in the same area, frequency
range, and period of time, thus creating interference between each other. This
encourages these dispersed APs to work together to enhance their performance
and mitigate the aforementioned interference [3]. Because of the shared usage
of the spectrum and the broadcasting nature of wireless transmission, ISAC
systems’ security is facing several challenges. On the one hand, Rician
channels, which are frequently occurring at mmWave frequencies and contain a
line of sight (LoS) component, are inextricably linked to the sensing channel.
This differs from traditional physical layer security (PLS) research [4] in
communication systems with the premise that the legitimate user channels and
intercept channels are independently and identically distributed. The
confidential information designated to the user equipments is included in the
dual-functional waveform, namely a waveform that jointly serves the
communication UEs and senses the sensing target [5], and is therefore
vulnerable to being intercepted by the sensing target. Reasons for
implementing such safety measures can stem from various factors. For instance,
in scenarios involving vehicle tracking for safety purposes, it is essential
to enable vehicle monitoring while preventing unauthorized access to the
ongoing communication. Let us consider a communication network deployed in an
open-air industrial area, where incoming vehicles deliver goods. The system
aims to track the vehicles autonomously, avoiding accidents, and
simultaneously safeguarding the confidentiality of the data traffic to prevent
industrial espionage.

From the sensing side, a crucial and intriguing trade-off appears: the power
is expected to be directed towards the sensing target, however, the useful
signal information must be protected from being intercepted by said target,
who is identified as a malicious agent. On the other hand, regardless of the
sensing target’s actions and intentions, it is important to increase the
system’s estimation and detection performance. In conventional communication
systems, security issues are addressed at the higher levels of the protocol
stack with cryptographic techniques [6], even though it is worth mentioning
that studies on cryptography frequently assume that the physical layer
provides a link that is error-free, but in reality, wireless communications
are subject to assaults, increasing the risk of information loss.

###  I-A Related work

In [7], the authors consider a cell-free MIMO network equipped with ISAC APs
and devise a transmit waveform co-design strategy aimed at maximizing the
signal to noise ratio (SNR) of the sensing target, here assumed to be a benign
agent, under a minimum signal to noise and interference ratio (SINR)
constraint for the communication users. The performance of this transmit
strategy, in terms of achieved communication SINR and sensing SNR, are
compared against well-established sensing-prioritized and communication-
prioritized design strategies, demonstrating the performance gain achieved by
the proposed co-design strategy. On the other hand, in [8], the idea of adding
artificial noise (AN) to the transmitted waveform to improve the network PLS
is introduced . This technique is an alternative to coding-based schemes aimed
at achieving the same goal [9]. This work considers a single AP with multiple
targets, assumed to be eavedroppers (Eves), and derives the optimal
beamforming strategy by solving a weighted optimization problem. The objective
functions bound together by the weighted optimization are the Cramer-Rao bound
(CRB) onto the targets’ angles and the system’s secrecy rate, defined as the
minimum difference between each user’s achievable rate and a sensing rate,
defined as $\log_{2}(1+\acs{SNR})$.

###  I-B Contributions

Contrary to the existing literature, this paper investigates how AN can be
used by distributed ISAC APs, cooperating to increase the PLS of a cell-free
MIMO network. Said network is made of $M$ APs that serve multiple
communication users and simultaneously sensing a target, assumed to be an Eve,
using the same signal. In addition to its monostatic observation, every AP
receives the reflected echos from all the other APs and sends them to a
central processing unit (CPU) trough a back-haul link here assumed error-free.

The main contributions of this paper are the following:

  * •

We propose a novel transmit waveform model to be used by distributed ISAC APs
in a cell-free MIMO network.

  * •

We calculate the optimal transmit waveform by solving a constrained CRB
minimization problem. The relaxed version of the original problem is proved to
be tight for the scenario at hand

  * •

We characterize the optimal AN structure, showing that it’s rank-1 and
directed towards the eve.

  * •

Numerical simulation prove the trade-off between sensing and communication
performances and show the effects that the UEs-Eve proximity has on the
optimal CRB.

##  II System model

Consider a cell-free MIMO network consisting of $M$ APs, $K$ UEs and a sensing
target,assumed to be an eve. To protect the privacy and security of the data
traffic from the eve, we introduce an AN vector $\boldsymbol{\xi}$ [8] that
will be used to reduce the sensing target’s SNR. Each AP is equipped with a
uniform linear array (ULA) comprising $N$ transmit and $N$ receive antennas.

The inter-element spacing within the ULA is set to $\lambda/2$, where
$\lambda$ is the carrier wavelength. Conversely, each UE is equipped with a
single antenna. Digital beamforming capabilities are assumed for each AP,
which is connected to a CPU responsible for designing the transmit signals. In
each timeslot $t$, AP $m$ transmits a waveform denoted by the complex vector
$\boldsymbol{\phi}_{m}^{t}\in\mathbb{C}^{N}$, which can modeled be as

$\displaystyle\boldsymbol{\phi}_{m}^{t}=\sum_{s=1}^{S}\mathbf{f}_{m,s}x_{m,s}^{t}+\boldsymbol{\xi}_{m}^{t},$
(1)

where $\mathbf{f}_{m,s}\in\mathbb{C}^{N}$ is the precoding vector used by AP
$m$ for stream $s$. $S$ represents the number of transmission streams: each UE
is served by one transmission stream and an additional stream is allocated for
sensing purposes. Consequently, $S=K+1$. On the other hand, $x_{m,s}^{t}$
represents the complex-valued unitary-power information symbol and the vector
$\boldsymbol{\xi}_{t}^{m}\in\mathbb{C}^{N}$ represents the AN vector
transmitted during timeslot $t$ by AP $m$. This noise is statistically
independent of the transmitted symbols. In order to maximize its entropy, said
noise is modeled as a complex Gaussian distributed vector with zero mean and
covariance matrix $\mathbf{R}_{\mathbf{\Xi}_{m}}$. We can get rid of the
summation and express (1) using matrices:

$\displaystyle\boldsymbol{\phi}_{m}^{t}=\mathbf{F}_{m}\mathbf{x}_{m}^{t}+\boldsymbol{\xi}_{m}^{t},$
(2)

where
$\mathbf{F}_{m}=[\mathbf{f}_{m,1}\dots\mathbf{f}_{m,S}]\in\mathbb{C}^{N\times
S}$ and $\mathbf{x}_{m}^{t}=[x_{m,1}^{t}\ldots
x_{m,S}^{t}]^{\top}\in\mathbb{C}^{S}$, All the following derivations assume
that symbols belonging to different streams and different APs are
statistically uncorrelated, i.e.,
$\mathbb{E}[(x_{m,s}^{t})^{\mathsf{H}}(x_{m^{\prime},s^{\prime}}^{t})]=1$ if
and only if $m=m^{\prime}$ and $s=s^{\prime}$. This condition can be achieved
through pseudo-random coding [10].

###  II-A Communication observation

The channel vector between UE $u$ and AP $m$ is denoted by the vector
$\mathbf{h}_{m,k}\in\mathbb{C}^{N}$. This allow us to define the received
signal observed by UE $k$ during the timeslot $t$ as

$\displaystyle y_{k,t}^{\text{c}}$
$\displaystyle=\sum_{m=1}^{M}\mathbf{h}_{m,k}^{\mathsf{H}}\boldsymbol{\phi}_{m}^{t}+w_{u}=\sum_{m=1}^{M}\mathbf{h}_{m,k}^{\mathsf{H}}\mathbf{f}_{m,k}x_{m,k}^{t}+$
(3) $\displaystyle\sum_{\begin{subarray}{c}s\neq k\\\
s=1\end{subarray}}^{S}\sum_{m=1}^{M}\mathbf{h}_{m,k}^{\mathsf{H}}\mathbf{f}_{m,s}x_{m,s}^{t}+\sum_{m=1}^{M}\mathbf{h}_{m,k}^{\mathsf{H}}\boldsymbol{\xi}_{m}^{t}+w_{k},$

where $w_{k}\sim\mathcal{CN}(0,\sigma_{\text{c}}^{2})$ is the receiver noise,
assumed to be uncorrelated from the transmitted waveform.

Figure 1: illustration of the Cell-free MIMO network where ISAC APs serve
communication users while sensing an Eve

###  II-B Multistatic sensing

As shown in Fig. 1, the APs implement multistatic sensing: every AP receives
the echo of its own transmitted signal plus the reflections coming from all of
the other APs. The Eve is modeled as a point reflector having a LoS path with
every AP. Under these assumptions, we can define the radar observations
collected by AP $m$ during timeslot $t$ as [7]

$\displaystyle\mathbf{y}_{m,t}^{\text{s}}$
$\displaystyle=\sum_{s=1}^{S}\underbrace{\sum_{m^{\prime}=1}^{M}\alpha_{m^{\prime}}^{m}\mathbf{a}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\mathbf{\phi}_{m^{\prime},s}^{t}}_{\mathbf{g}_{m,s}^{t}}+\mathbf{n}_{m}$
$\displaystyle=\sum_{s=1}^{S}\sum_{m^{\prime}=1}^{M}\alpha^{m}_{m^{\prime}}\mathbf{a}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\mathbf{f}_{m^{\prime},s}x_{m^{\prime},s}^{t}$
$\displaystyle+\sum_{s=1}^{S}\sum_{m^{\prime}=1}^{M}\alpha_{m^{\prime}}^{m}\mathbf{a}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\boldsymbol{\xi}_{m^{\prime}}^{t}+\mathbf{n}_{m},$
(4)

where $|\alpha_{m^{\prime}}^{m}|^{2}$ is the channel gain between the
transmitting AP $m^{\prime}$ and receiving AP $m$. To take into account the
effects of pathloss and radar cross section (RCS), the channel gain is modeled
according to the Swerling-I model, that is
$\alpha_{m^{\prime}}^{m}\sim\mathcal{CN}(0,(\delta_{m^{\prime}}^{m})^{2})$.
The angles $\theta_{m}$ and $\theta_{m^{\prime}}$ represent the angle of
arrival (AoA) to AP $m$ and the angle of departure (AoD) from AP $m^{\prime}$,
whereas $\mathbf{a}(\cdot)$ is the steering vector of a ULA. The thermal noise
vector is denoted by
$\mathbf{n}_{m}\sim\mathcal{CN}(0,\sigma_{\text{s}}^{2}\mathbf{I}_{N})$, and
is assumed to be uncorrelated from the transmitted waveform.

##  III Transmitted waveform joint design

The sensing objective we have chosen to minimize is the CRB on
$\theta_{1},\ldots\theta_{m}$. Assuming that the optimal waveform is computed
during timeslot $t$, we assume that $\theta_{1},\ldots\theta_{m}$ have been
estimated (e.g. using the MUSIC algorithm with omnidirectional pilots [11])
during timeslot $t-1$ and that, given a low mobility of the Eve, said
estimations still hold true during the present timeslot. In this way, using
the initial estimation, The cell-free MIMO network can compute a transmit
waveform that boosts its future estimation performances while simultaneously
degrading the Eve channel quality by means of the AN. In this section, we
first derive the Fisher information matrix (FIM) and CRB and then use them to
define the optimization problem trough which we design the optimal transmitted
waveform, semi-definite relaxation (SDR) is used to obtain a relaxed problem
solvable trough complex optimization solvers.

###  III-A CRB derivation

Let $\boldsymbol{\eta}\in\mathbb{R}^{(2M^{2}+M)}$ denote the vector of unknown
channel parameters

$\displaystyle\boldsymbol{\eta}=[\Re\\{\alpha_{1}^{1}\\},\Im\\{\alpha_{1}^{1}\\},\ldots,\Re\\{\alpha_{M}^{M}\\},\Im\\{\alpha_{M}^{M}\\},\theta_{1},\ldots,\theta_{M}]^{\top},$
(5)

where $\Re\\{\alpha_{1}^{1}\\}$ and $\Im\\{\alpha_{1}^{1}\\}$ represent the
real and imaginary part of the complex channel coefficient $\alpha_{1}^{1}$
Let us then define the vectors
$\mathbf{y}_{t}^{\text{s}}=[\mathbf{y}_{1,t}^{\text{s}}\ldots\mathbf{y}_{M,t}^{\text{s}}]^{\top}\in\mathbb{C}^{M\times
N}$ and
$\mathbf{g}^{t}_{s}=[\mathbf{g}^{t}_{1,s}\ldots\mathbf{g}^{t}_{M,s}]^{\top}\in\mathbb{C}^{M\times
N}$. The FIM $\boldsymbol{J_{\eta}}\in\mathbb{R}^{(2M^{2}+M)\times(2M^{2}+M)}$
can be then calculated as

$\displaystyle\boldsymbol{J_{\eta}}$
$\displaystyle=\mathbb{E}_{\mathbf{y}_{t}^{\text{s}}|\boldsymbol{\eta}}\Bigg{[}-\frac{\partial^{2}\log
f(\mathbf{y}_{t}^{\text{s}}|\boldsymbol{\eta})}{\partial\boldsymbol{\eta}\partial\boldsymbol{\eta}^{\top}}\Bigg{]}$
$\displaystyle=\frac{2}{\sigma_{\text{s}}^{2}}\sum_{s=1}^{S}\Re\Bigg{\\{}\bigg{(}\frac{\partial\mathbf{g}^{t}_{s}}{\partial\boldsymbol{\eta}}\Bigg{)}^{\mathsf{H}}\bigg{(}\frac{\partial\mathbf{g}^{t}_{s}}{\partial\boldsymbol{\eta}}\Bigg{)}\Bigg{\\}},$
(6)

where $f(\mathbf{y}_{t}^{\text{r}}|\boldsymbol{\eta})$ is the conditional
likelihood function of the stacked observation vector
$\mathbf{y}_{t}^{\text{r}}$ given $\boldsymbol{\eta}$. The CRB for the
estimation of $\theta_{m}$ is then defined as

$\displaystyle\text{CRB}_{\theta_{m}}=\sqrt{[\boldsymbol{J_{\eta}}^{-1}]_{2M^{2}+m,2M^{2}+m}}.$
(7)

The square root has been included to facilitate future comparison with the
estimation root mean square error.

###  III-B Optimal transmit waveform

The chosen optimization metric is practically translated into the minimization
of the FIM’s inverse trace. The optimization variables are the AP precoding
matrices $\mathbf{F}_{m}$ and the AN covariance matrices
$\mathbf{R}_{\mathbf{\Xi}_{m}}$. In order to define the constraints of the
optimization problem, we first need to mathematically define the UE SINR and
the Eve SNR. The communication channel $\mathbf{h}_{m,k}$ is assumed to be
deterministic, a condition achievable through the acquisition of channel-state
information at the transmitter, and the multi-user interference is treated as
noise [7]. Following (3), the SINR for user $k$ can be defined as

$\displaystyle\text{SINR}_{k}=$ (8)
$\displaystyle\frac{\sum_{m=1}^{M}|\mathbf{h}_{m,k}^{\text{H}}\mathbf{f}_{m,k}|^{2}}{\sum_{m=1}^{M}\sum_{\begin{subarray}{c}s\neq
k\\\
s=1\end{subarray}}^{S}|\mathbf{h}_{m,k}^{\text{H}}\mathbf{f}_{m,s}|^{2}+\sum_{m=1}^{M}|\mathbf{h}_{m,k}^{\text{H}}\mathbf{R}_{\mathbf{\Xi}_{m}}\mathbf{h}_{m,k}|+\sigma_{\text{c}}^{2}}.$

On the other hand, the eve’s SNR can be modelled as

$\displaystyle\text{SNR}_{\text{E}}=\frac{\sum_{m=1}^{M}\sum_{s=1}^{S}(\delta_{m}^{m})^{2}|\mathbf{a}(\theta_{m})^{\text{H}}\mathbf{f}_{s,m}|^{2}}{\sum_{m=1}^{M}(\delta_{m}^{m})^{2}|\mathbf{a}(\theta_{m})^{\text{H}}\mathbf{R}_{\mathbf{\Xi}_{m}}\mathbf{a}(\theta_{m})|+\sigma_{\text{s}}^{2}}.$
(9)

We are now able to define the proposed transmit waveform design strategy

$\displaystyle\min_{\\{\mathbf{R}_{\mathbf{\Xi}_{m}}\\}_{1}^{M},\\{\mathbf{f}_{m,s}\\}_{1,1}^{M,S}}$
$\displaystyle~{}\text{Tr}(\mathbf{J}_{\boldsymbol{\eta}}^{-1}),$ (10a) s.t.
$\displaystyle\text{SINR}_{k}\geq\gamma_{k},~{}k=1,\ldots,K$ (10b)
$\displaystyle\text{SNR}_{\text{E}}\leq\psi,$ (10c)
$\displaystyle||\boldsymbol{\phi}_{t}||^{2}\leq P_{m},~{}m=1\ldots M,$ (10d)

where $P_{m},\gamma_{k}$ and $\psi$ represent the available power at AP $m$,
the required SINR for user $k$, and the maximum SNR that the Eve should be
allowed to achieve.

###  III-C Semidefinite reformulation

Problem (10) is non-convex in the variable $\mathbf{f}_{m,s}$ due to the
quadratic terms occurring in both the SINR and SNR. To solve this, we
reformulate (10) as an semi-definite program (SDP)[12]. In particular, we
define the variable $\mathbf{W}_{s}=\mathbf{f}_{s}\mathbf{f}_{s}^{\text{H}}$,
where
$\mathbf{f}_{s}=[\mathbf{f}_{1,s}^{\top}\dots\mathbf{f}_{M,s}^{\top}]^{\top}\in\mathbb{C}^{NM}$
and express the objective function and the constraints as a function of this
variable. We can then reformulate (10b) as

$\displaystyle\text{Tr}(\mathbf{h}_{k}\mathbf{h}_{k}^{\text{H}}\mathbf{W}_{k})\geq$
(11) $\displaystyle\gamma_{k}\Big{(}\sum_{\begin{subarray}{c}s\neq k\\\
s=1\end{subarray}}^{S}\text{Tr}(\mathbf{h}_{k}\mathbf{h}_{k}^{\text{H}}\mathbf{W}_{s})+\sum_{m=1}^{M}\text{Tr}(\mathbf{h}_{m,k}\mathbf{h}_{m,k}^{\text{H}}\mathbf{R}_{\mathbf{\Xi}_{m}})\
+\sigma^{2}_{\text{c}}\Big{)},$

where
$\mathbf{h}_{k}=[\mathbf{h}_{1,k}^{\top}\dots\mathbf{h}_{M,k}^{\top}]^{\top}\in\mathbb{C}^{NM}$.
Similarly, the Eve’s SNR constraint in (10c) can be redefined as

$\displaystyle\sum_{m=1}^{M}\sum_{s=1}^{S}(\delta_{m}^{m})^{2}\text{Tr}(\mathbf{a}(\theta_{m})\mathbf{a}(\theta_{m})^{\text{H}}\mathbf{W}_{m,s})\leq$
$\displaystyle\psi\Big{(}\sum_{m=1}^{M}(\delta_{m}^{m})^{2}\text{Tr}(\mathbf{a}(\theta_{m})\mathbf{a}(\theta_{m})^{\text{H}}\mathbf{R}_{\mathbf{\Xi}_{m}})+\sigma_{\text{s}}^{2}\Big{)},$
(12)

where $\mathbf{W}_{m,s}=\mathbf{f}_{m,s}\mathbf{f}_{m,s}^{\mathsf{H}}$ and
this is the $m^{\text{th}}$ $N\times N$ block onto the main diagonal of
$\mathbf{W}_{s}$. Leveraging the lack of correlation between the terms in (2),
(10d) can be re-written as:

$\displaystyle||\boldsymbol{\phi}_{m}^{t}||^{2}$
$\displaystyle=\text{Tr}(\sum_{s=1}^{S}\mathbf{W}_{m,s}+\mathbf{R}_{\mathbf{\Xi}_{m}})\leq
P_{m}.$ (13)

Eventually, the SDP-equivalent problem can be defined as :

$\displaystyle\min_{\\{\mathbf{R}_{\mathbf{\Xi}_{m}}\\}_{1}^{M},\\{\mathbf{W}_{s}\\}_{1}^{S}}$
$\displaystyle~{}\text{Tr}(\mathbf{J}_{\boldsymbol{\eta}}^{-1})$ (14a) s.t.
$\displaystyle\eqref{SINR_sdp}~{}k=1\ldots K,$ (14b)
$\displaystyle\eqref{SNR_sdp}$ (14c)
$\displaystyle\eqref{power_constr_sdp}~{}m=1\ldots M,$ (14d)
$\displaystyle\text{rank}(\mathbf{W}_{s})=1,~{}~{}s=1\ldots S$ (14e)
$\displaystyle\mathbf{W}_{s}\in\mathbb{S}^{+},~{}~{}s=1\ldots S,$ (14f)

where $\mathbb{S}^{+}$ represents the set of Hermitian positive semidefinite
matrices, the reformulation of $\mathbf{J}_{\boldsymbol{\eta}}$ elements as a
function of the relaxed variables is shown in the appendix. This problem can
be relaxed by removing constraint (14e) and then solved via convex
optimization solvers [13]:

$\displaystyle\min_{\\{\mathbf{R}_{\mathbf{\Xi}_{m}}\\}_{1}^{M},\\{\mathbf{W}_{s}\\}_{1}^{S}}$
$\displaystyle~{}\text{Tr}(\mathbf{J}_{\boldsymbol{\eta}}^{-1})$ (15a) s.t.
$\displaystyle\eqref{SINR_sdp}~{}k=1\ldots K,$ (15b)
$\displaystyle\eqref{SNR_sdp}$ (15c)
$\displaystyle\eqref{power_constr_sdp}~{}m=1\ldots M,$ (15d)
$\displaystyle\mathbf{W}_{s}\in\mathbb{S}^{+},~{}~{}s=1\ldots S.$ (15e)

In the next section we show that the optimal solution of problem 15 yields
rank-$1$ matrices, making said solution optimal for problem 14 and the
relaxation tight.

##  IV Simulation results

Figure 2: simulated APs and UEs layout, the UEs are randomly placed withing
the green box Figure 3: AP $1$ (top) and AP $2$ (bottom) CRB on $\theta_{m}$
for different $\psi$ Figure 4: achieved eve’s SNR vs $\gamma$ for different
values of $\psi$

In this section, we now present the performance of the proposed waveform
design strategy in a cell-free MIMO network with $M=2$ APs, each of them
equipped with a ULA of $N=30$ antennas, using the same waveform to serve $K=4$
UEs and sense one Eve. The UEs are randomly distributed in a $40\times 40$ m
area in front of the APs, located at the coordinates $[10,0]$ and $[80,0]$. We
then choose $P_{m}=1$ W the noise variances are
$\sigma^{2}_{\text{c}}=\sigma^{2}_{\text{s}}=1$ . The Swerling-I model
variance $(\delta_{m^{\prime}}^{m})^{2}$ has been set to $0.1$ for all the
$m,m^{\prime}$ pairs. Unless otherwise specified, the maximum ’s SNR $\psi$
has been set to $0$ dB. All the UEs are envisioned to be guaranteed the same
SINR, thus $\gamma_{1}=\ldots=\gamma_{K}=\gamma$

###  IV-A CRB dependency by $\psi$

In this subsection we investigate how the choice of $\psi$ affects the
achieved CRB on $\theta_{1}$ and $\theta_{2}$. Fig. 3 shows the achieved CRB
as a function of $\gamma$ for three different $\psi$: We observe that larger
$\gamma$ values correspond to higher CRB values. This is a manifestation of
the sensing-communication trade-off: as the SINR requirements become more
demanding, the system needs to allocate more resources on the communication
side, thus degrading the sensing performances, namely achieving a higher CRB.
A second trade-off arises between the CRB and $\psi$: a more stringent
constraint onto the Eve’s SNR corresponds to worse sensing performances. This
can be ascribed to the direct proportionality between the system’s sensing
capabilities, namely the CRB, and the amount of power beamformed towards the
target, i.e. the SNR. Fig. 3 shows that constraining the sensing SNR to
$\psi=-5$ dB cause sensibly higher CRB than the case where $\psi=0$ dB,
whereas setting $\psi=-3$ dB generates a CRB very similar to the former. This
implies that a $3$ dB decrease of the allowed sensing SNR, i.e. from $0$ dB to
$-3$dB, cause a very little degradation in the sensing performance and thus
the latter is preferrable to the former. Fig. 4 indicates that constraint
(III-C) is tight as the achieved Eve’s SNR is equal to $\psi$

###  IV-B Relaxation’s tightness

The optimal $\mathbf{W}_{s}$ for the communication streams (i.e. $s=1\ldots
K$) are all rank-$1$ matrices.The communication beamforming vectors can be
retrieved as

$\displaystyle\mathbf{f}_{s,m}=[\sqrt{\epsilon_{s}}\boldsymbol{u}]_{(m-1)N+1~{}:~{}mN},$
(16) $\displaystyle m=1\ldots M,~{}s=1\ldots K,$

where $\epsilon_{k}$ and $\boldsymbol{u}$ are the non-zero eigenvalue and
respective eigenvector associated to $\mathbf{W}_{s}$. As for the sensing
stream ( i.e. $s=K+1$ ) The optimal $\mathbf{W}_{s}$ is not rank-1 as all its
eigenvalues are in the order of $10^{-5}$ : We can then say that
$\mathbf{W}_{s}$ has rank-almost zero, making the presence of a sensing stream
superfluous. Lastly, the The optimal AN covariance matrices
$\\{\mathbf{R}_{\mathbf{\Xi}_{m}}^{\text{opt}}\\}_{1}^{M}$ are rank-1: this
result is to be expected as the AN needs to be concentrated solely towards the
Eve. Given the previous considerations we can claim that a transmit waveform
originated by the solution of problem (15) constitutes an optimal of problem
(14), thus making the relaxation tight.

###  IV-C Artificial noise characterization

It can be seen that the AN covariance matrix $\mathbf{R}_{\mathbf{\Xi}_{m}}$
is used both in the UE SINR and in the Eve SNR definitions. In both equations,
$\mathbf{R}_{\mathbf{\Xi}_{m}}$ appears in the denominator, either multiplied
by the communication channel vector $\mathbf{h}_{m,k}$ or the steering vector
$\mathbf{a}(\theta_{m})$. The latter multiplication inevitably gives
$\mathbf{R}_{\mathbf{\Xi}_{m}}$ a directional selectivity, as shown in Fig. 5

Figure 5: optimal AN beampattern obtained for $\gamma=4$ Figure 6: AP $1$’s
optimal AN beampattern for different values of $\gamma$ Figure 7: AP $1$ (top)
and AP $2$ (bottom) CRB on $\theta_{m}$ vs $\gamma$ for different UEs-Eve
proximity conditions

The AN beam pattern peaks around $\theta_{m}$ with a rather narrow beamwidth
and is otherwise well below $0$ dB,in line with the fact that
$\mathbf{R}_{\mathbf{\Xi}_{m}}$ is rank-1. Furthermore, We see that the power
beamformed toward the target by AP $2$ is lower than its counterpart, this
means that the CPU is able to adapt the AN power between APs. Another
manifestation of the previously mentioned performance tradeoff can be seen in
Fig. 6, where it’s shown that an increase in $\gamma$ corresponds to a
decrease in the AN peak value. A possible explanation for this is that as the
system tries to guarantee a higher SINR to its users, the term
$|\mathbf{h}_{m,k}^{\text{H}}\mathbf{R}_{\mathbf{\Xi}_{m}}\mathbf{h}_{m,k}|$
must be lower, leading to a progressive decrease of the beampattern peak.

###  IV-D CRB vs SINR and proximity

Fig. 7, two curves are shown, labeled ”distant” and ”close”, respectively.
(The ”distant” configuration is shown in Fig. 2.) The ”close” configuration
was created by placing two of the UEs at a distance of $0.5$ m from the
target, one on the left and one on the right. This configuration has been
considered to investigate how the optimization problem would change if one or
more UEs angular coordinates would fall within the AN main lobe depicted in
Fig. 5. Fig. 7 confirms the logical intuition that such a proximity between
the Eve and one or more UE would inevitably lead to a degradation of the
optimization function value. It is also interesting to note that the CRB’s
proximity-induced degradation is proportional to $\gamma$. This behavior can
be intuitively attributed to the fact that the system can allocate more
resources to the Eve when $\gamma$ is small, effectively mitigating the
disadvantage of proximity. However, when $\gamma$ becomes larger, the SINR
constraint becomes tighter and the system can no longer allocate additional
resources to compensate for the proximity, resulting in a higher CRB.

##  V Conclusions

In this paper, we have considered a secure transmit waveform design used by
ISAC APs in a cell-free MIMO environment to serve the communication users
while preventing an Eve from spoofing the communication information. To do
this, the transmitted dual waveform includes an AN vector whose goal is to
degrade the Eve’s SNR. The optimal precoding and AN covariance matrices are
computed by minimizing the CRB on the monostatic observation angles under SINR
constraints for the communication users and maximum SNR constraint for the
Eve. Our simulations have shown the inherent tradeoff between sensing and
communication performances and how, given the angular directionality of the
AN,UEs-Eve proximity degrade the system’s performances, making this design
strategy very efficient in limiting the impact of an Eve that’s physically
separated from the legitimate communication users.

##  Appendix A FIM as a function of $\mathbf{W}_{s}$

Here we show how the FIM elements can be expressed as a function of the SDP
variable $\mathbf{W}_{s}$:

$\displaystyle\Bigg{(}\frac{\partial\mathbf{g}^{t}_{m,s}}{\partial\Re\\{\alpha_{m}^{m^{\prime}}\\}}\Bigg{)}^{\mathsf{H}}\frac{\partial\mathbf{g}^{t}_{m,s}}{\partial\Re\\{\alpha_{m}^{m^{\prime}}\\}}=$
(17)
$\displaystyle(\boldsymbol{\phi}_{m^{\prime},s}^{t})^{\mathsf{H}}\mathbf{a}(\theta_{m^{\prime}})\mathbf{a}(\theta_{m})^{\mathsf{H}}\mathbf{a}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\boldsymbol{\phi}_{m^{\prime},s}^{t}=$
(18) $\displaystyle
N\text{Tr}((\boldsymbol{\phi}_{m^{\prime},s}^{t})^{\mathsf{H}}\boldsymbol{\phi}_{m^{\prime},s}^{t}\mathbf{a}(\theta_{m^{\prime}})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}})\stackrel{{\scriptstyle(a)}}{{=}}$
(19) $\displaystyle
N<(\mathbf{W}_{m^{\prime},s}+\mathbf{R}_{\mathbf{\Xi}_{m^{\prime}}});\mathbf{a}(\theta_{m^{\prime}})>,$
(20)

where in $(a)$ We have used the uncorrelation between the components of
$\boldsymbol{\phi}_{m,s}^{t}$, as shown in section II. Let us now move on and
present the second type of product

$\displaystyle\Bigg{(}\frac{\partial\mathbf{g}^{t}_{m,s}}{\partial\Re\\{\alpha_{m}^{m^{\prime}}\\}}\Bigg{)}^{\mathsf{H}}\frac{\partial\mathbf{g}^{t}_{m,s}}{\partial\theta_{m}}=$
(21)
$\displaystyle(\boldsymbol{\phi}_{m^{\prime},s}^{t})^{\mathsf{H}}\mathbf{a}(\theta_{m^{\prime}})\mathbf{a}(\theta_{m})^{\mathsf{H}}\Big{[}\alpha_{m}^{m}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})^{\mathsf{H}}\boldsymbol{\phi}_{m,s}^{t}+$
$\displaystyle\sum_{\begin{subarray}{c}m^{\prime}\neq m\\\
m=1\end{subarray}}^{M}\alpha_{m}^{m^{\prime}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\boldsymbol{\phi}_{m^{\prime},s}^{t}\Big{]}=$
(22)
$\displaystyle(\boldsymbol{\phi}_{m^{\prime},s}^{t})^{\mathsf{H}}\mathbf{a}(\theta_{m^{\prime}})\mathbf{a}(\theta_{m})^{\mathsf{H}}\alpha_{m}^{m^{\prime}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\boldsymbol{\phi}_{m^{\prime},s}^{t}=$
(23)
$\displaystyle\alpha_{m}^{m^{\prime}}\text{Tr}((\mathbf{W}_{m^{\prime},s}+\mathbf{R}_{\mathbf{\Xi}_{m^{\prime}}})\mathbf{a}(\theta_{m^{\prime}})\mathbf{a}(\theta_{m})^{\mathsf{H}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\tilde{\mathbf{a}}(\theta_{m^{\prime}})^{\mathsf{H}})$
(24)

where we use the fact that
$\mathbb{E}[(x_{m,s}^{t})^{\mathsf{H}}(x_{m^{\prime},s^{\prime}}^{t})]=0$ if
$m\neq m^{\prime}$ and $\tilde{\mathbf{a}}(\cdot)$ is equal to
$\accentset{\mbox{\large.}}{\mathbf{a}}(\cdot)$ if $m^{\prime}=m$ and
$\mathbf{a}(\cdot)$ otherwise.

$\displaystyle\Bigg{(}\frac{\partial\mathbf{g}^{t}_{m,s}}{\partial\theta_{m}}\Bigg{)}^{\mathsf{H}}\frac{\partial\mathbf{g}^{t}_{m,s}}{\partial\theta_{m}}=$
(25)
$\displaystyle\Big{[}\alpha_{m}^{m}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})^{\mathsf{H}}\boldsymbol{\phi}_{m,s}^{t}+\sum_{\begin{subarray}{c}m^{\prime}\neq
m\\\
m=1\end{subarray}}^{M}\alpha_{m}^{m^{\prime}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\boldsymbol{\phi}_{m^{\prime},s}^{t}\Big{]}^{\mathsf{H}}$
$\displaystyle\Big{[}\alpha_{m}^{m}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})^{\mathsf{H}}\boldsymbol{\phi}_{m,s}^{t}+\sum_{\begin{subarray}{c}m^{\prime}\neq
m\\\
m=1\end{subarray}}^{M}\alpha_{m}^{m^{\prime}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\mathbf{a}(\theta_{m^{\prime}})^{\mathsf{H}}\boldsymbol{\phi}_{m^{\prime},s}^{t}\Big{]}=$
(26)
$\displaystyle|\alpha_{m}^{m}|^{2}\text{Tr}((\mathbf{W}_{m,s}+\mathbf{R}_{\mathbf{\Xi}_{m}})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})^{\mathsf{H}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})^{\mathsf{H}})+$
$\displaystyle|\alpha_{m}^{m^{\prime}}|^{2}\text{Tr}((\mathbf{W}_{m^{\prime},s}+\mathbf{R}_{\mathbf{\Xi}_{m^{\prime}}})\mathbf{a}(\theta_{m^{\prime}})\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})^{\mathsf{H}}\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})\mathbf{a}(\theta_{m}^{\prime})^{\mathsf{H}})$
(27)

where $\accentset{\mbox{\large.}}{\mathbf{a}}(\cdot)$ is the steering vector’s
first derivative, defined as
$[\accentset{\mbox{\large.}}{\mathbf{a}}(\theta_{m})]_{n}=j\pi
n\cos(\theta_{m})e^{j\pi n\sin(\theta_{m})}$. All the other kind of products
appearing in the FIM have been omitted due to space constraints.

## References

  * [1] F. Liu, Y. Cui, C. Masouros, J. Xu, T. X. Han, Y. C. Eldar, and S. Buzzi, “Integrated sensing and communications: Towards dual-functional wireless networks for 6g and beyond,” _IEEE journal on selected areas in communications_ , 2022.
  * [2] J. M. Mateos-Ramos, C. Häger, M. F. Keskin, L. L. Magoarou, and H. Wymeersch, “Model-driven end-to-end learning for integrated sensing and communication,” _arXiv preprint arXiv:2212.10211_ , 2022.
  * [3] Z. Behdad, Ö. T. Demir, K. W. Sung, E. Björnson, and C. Cavdar, “Power allocation for joint communication and sensing in cell-free massive mimo,” in _GLOBECOM 2022-2022 IEEE Global Communications Conference_. IEEE, 2022, pp. 4081–4086.
  * [4] Y. Liu, Z. Qin, M. Elkashlan, Y. Gao, and L. Hanzo, “Enhancing the physical layer security of non-orthogonal multiple access in large-scale networks,” _IEEE Transactions on Wireless Communications_ , vol. 16, no. 3, pp. 1656–1672, 2017.
  * [5] S. Shi, Z. Wang, Z. He, and Z. Cheng, “Constrained waveform design for dual-functional mimo radar-communication system,” _Signal Processing_ , vol. 171, p. 107530, 2020.
  * [6] R. Melki, H. N. Noura, M. M. Mansour, and A. Chehab, “A survey on ofdm physical layer security,” _Physical Communication_ , vol. 32, pp. 1–30, 2019.
  * [7] U. Demirhan and A. Alkhateeb, “Cell-free isac mimo systems: Joint sensing and communication beamforming,” _arXiv preprint arXiv:2301.11328_ , 2023.
  * [8] N. Su, F. Liu, and C. Masouros, “Sensing-assisted eavesdropper estimation: An isac breakthrough in physical layer security,” _arXiv preprint arXiv:2210.08286_ , 2022.
  * [9] P. Zhang, Y. Jiang, C. Lin, Y. Fan, and X. Shen, “P-coding: secure network coding against eavesdropping attacks,” in _2010 Proceedings IEEE INFOCOM_. IEEE, 2010, pp. 1–9.
  * [10] X. Liu, T. Huang, N. Shlezinger, Y. Liu, J. Zhou, and Y. C. Eldar, “Joint transmit beamforming for multiuser mimo communications and mimo radar,” _IEEE Transactions on Signal Processing_ , vol. 68, pp. 3929–3944, 2020.
  * [11] A. Zahernia, M. J. Dehghani, and R. Javidan, “Music algorithm for doa estimation using mimo arrays,” in _2011 6th International Conference on Telecommunication Systems, Services, and Applications (TSSA)_. IEEE, 2011, pp. 149–153.
  * [12] A. B. Gershman, N. D. Sidiropoulos, S. Shahbazpanahi, M. Bengtsson, and B. Ottersten, “Convex optimization-based beamforming,” _IEEE Signal Processing Magazine_ , vol. 27, no. 3, pp. 62–75, 2010.
  * [13] Z.-Q. Luo, W.-K. Ma, A. M.-C. So, Y. Ye, and S. Zhang, “Semidefinite relaxation of quadratic optimization problems,” _IEEE Signal Processing Magazine_ , vol. 27, no. 3, pp. 20–34, 2010.

  *[ISAC]: integrated sensing and communications
  *[MIMO]: multiple-input multiple-output
  *[AP]: access point
  *[APs]: access point
  *[LoS]: line of sight
  *[PLS]: physical layer security
  *[UEs]: user equipment
  *[SNR]: signal to noise ratio
  *[SINR]: signal to noise and interference ratio
  *[AN]: artificial noise
  *[CRB]: Cramer-Rao bound
  *[CPU]: central processing unit
  *[ULA]: uniform linear array
  *[UE]: user equipment
  *[RCS]: radar cross section
  *[AoA]: angle of arrival
  *[AoD]: angle of departure
  *[FIM]: Fisher information matrix
  *[SDR]: semi-definite relaxation
  *[SDP]: semi-definite program