#  $\alpha$-leakage by Rényi Divergence and Sibson Mutual Information
††thanks: The two authors only contribute to Proposition 1: the second author
derived another proof of Proposition 1, similar to the proof of [5, Th. 1].
But, the interpretation and relationship between $f$\- and $\tilde{f}$-mean
measure have not been pointed out until this paper.

Ni Ding University of Auckland
New Zealand
<EMAIL_ADDRESS>Mohammad Amin Zarrabian Australian National University
Australia
<EMAIL_ADDRESS>Parastoo Sadeghi University of New South Wales
Australia
<EMAIL_ADDRESS>
###### Abstract

For $\tilde{f}(t)=\exp(\frac{\alpha-1}{\alpha}t)$, this paper proposes a
$\tilde{f}$-mean information gain measure. Rényi divergence is shown to be the
maximum $\tilde{f}$-mean information gain incurred at each elementary event
$y$ of channel output $Y$ and Sibson mutual information is the
$\tilde{f}$-mean of this $Y$-elementary information gain. Both are proposed as
$\alpha$-leakage measures, indicating the most information an adversary can
obtain on sensitive data. It is shown that the existing $\alpha$-leakage by
Arimoto mutual information can be expressed as $\tilde{f}$-mean measures by a
scaled probability. Further, Sibson mutual information is interpreted as the
maximum $\tilde{f}$-mean information gain over all estimation decisions
applied to channel output.

###### Index Terms:

Information gain, $\alpha$-leakage.

##  I Introduction

Information leakage was first proposed in a statistical inference framework
[1]. For an adversary observing published data, the information gain, or
uncertainty reduction, on sensitive attribute from the prior belief (the side
information obtained by the adversary) indicates the quantity of privacy
leakage. While mutual information was interpreted as the mean privacy measure
in [1, 2], Issa _et al._ proposed maximal leakage in [3] focusing on the
worst-case event that incurs the maximal amount of privacy flow to the
adversary. They were later generalized by an $\alpha$-leakage [4], tunable
between average and high-risk events. It was revealed in [5], [4, Theorem 1]
that this $\alpha$-leakage is the same as a Rényi measure called Arimoto
mutual information, where the $\alpha$-order uncertainty is quantified by
Rényi entropy [6] and Arimoto conditional entropy [7] respectively for prior
and posterior beliefs and the difference measures uncertainty reduction.

While the existing leakages are (essentially) using $\alpha$-order entropy
measures, it is worth noting that Rényi has also defined the $\alpha$-order
relative information in [6], quantifying the expected uncertainty in a
probability distribution in addition to another one. It was specifically
called $\alpha$-order information gain in [8], whereas the well-known name is
Rényi divergence. The idea is to collect the information gain, the logarithm
of Radon-Nikodym derivative (also called relative information [9, eq.(6)]),
incurred at each elementary event and get the $f$-mean of them for
$f(t)=\exp((\alpha-1)t)$ w.r.t. the frequency of appearance for each
elementary event. This naturally suggests Rényi divergence and Sibson mutual
information [8], the information radius defined in terms of Rényi divergence,
for $\alpha$-order information leakage measure. However, existing studies [10,
3, 4] only reveal that they upper bound privacy leakage of all sensitive
attributes (of channel input) for fixed channel and input distribution.

In this paper, we propose Rényi divergence and Sibson mutual information as
the exact $\alpha$-leakage of a sensitive attribute. We first define a
$\tilde{f}$-mean information gain measure, where
$\tilde{f}(t)=\exp(\frac{\alpha-1}{\alpha}t)$. Viewing the posterior belief as
a soft decision chosen by the adversary to estimate sensitive attribute, Rényi
divergence is shown to be the maximum $\tilde{f}$-mean information gain
incurred at each elementary event $y$ of channel output $Y$. It is then
proposed as $Y$-elementary $\alpha$-leakage, and the $\tilde{f}$-mean of it is
measured by Sibson mutual information. The existing leakages in [3, 4, 10] can
be expressed by the proposed $\alpha$-leakage measures via a scaled
probability distribution, by which the leakage upper bound results [4,
Ths.1&2], [3, Th.1], [10, Th.1] are straightforward by post-processing
property.

###  I-A Notation

Capital and lowercase letters denote random variable (r.v.) and its elementary
event or instance, respectively, e.g., $x$ is an instance of $X$. Calligraphic
letters denote sets, e.g., $\mathcal{X}$ refers to the alphabet of $X$. We
assume finite countable alphabet. Denote $P_{X}(x)$ the probability of outcome
$X=x$. For $\mathcal{B}\subseteq\mathcal{X}$,
$\mathbf{P}_{X}(\mathcal{B})=(P_{X}(x):x\in\mathcal{B})$ is a probability
vector indexed by $\mathcal{B}$. For singleton $B=\\{x\\}$,
$\mathbf{P}_{X}(\\{x\\})$ is simplified to $P_{X}(x)$. The probability mass
function $\mathbf{P}_{X}(\mathcal{X})$ is expressed by notation
$\mathbf{P}_{X}$. The support of probability mass is denoted by
$\text{supp}(P_{X})=\\{x\colon P_{X}(x)>0\\}$. Each $\mathbf{P}_{X}$ is a
vector in the $|\mathcal{X}|-1$-dimensional probability simplex, denoted by
$\triangle_{\mathcal{X}}$. An optimizations over $\mathbf{P}_{X}$ has the
constraint set being probability simplex
$\\{\mathbf{P}_{X}\in\mathbb{R}_{+}^{|\mathcal{X}|}\colon\sum_{x\in\mathcal{X}}P_{X}(x)=1\\}$.
The expected value of $f(X)$ w.r.t. probability  $\mathbf{P}_{X}$ is denoted
by $\mathbb{E}_{P_{X}}[f(X)]=\sum_{x\in\mathcal{X}}P_{X}(x)f(x)$. For the
conditional probability $\mathbf{P}_{Y|X}=(P_{Y|X}(y|x)\colon
x\in\mathcal{X},y\in\mathcal{Y})$, $\mathbf{P}_{Y|X=x}=(P_{Y|X}(y|x)\colon
y\in\mathcal{Y})$ denotes the probability of $Y$ given the outcome $X=x$.

##  II Preliminaries

Let $f(t)=\exp((\alpha-1)t)$. The $f$-mean is
$\bar{Z}=f^{-1}(\mathbb{E}[f(Z)])$, also called Kolmogorov-Nagumo average [11,
12]. Alfréd Rényi has defined the $\alpha$-order relative information in [6]
as an $f$-mean as follows. For two probability distributions
$\mathbf{P}_{X},\mathbf{Q}_{X}$ such that $\mathbf{P}_{X}\ll\mathbf{Q}_{X}$,
the relative information for any event subset
$\mathcal{B}\subseteq\mathcal{X}$ is111The underlying assumption for relative
information in [6, pp.553, Sec.3] is that $\mathbf{Q}$ refers to the original
unconditional distribution of an r.v., while $\mathbf{P}$ is the distribution
of the same r.v. conditioned on some event. In this case, $\mathbf{P}$ is
absolutely continuous w.r.t. $\mathbf{Q}$ and the Radon–Nikodym derivative
$\frac{\differential P}{\differential Q}$ is well defined. Alfréd Rényi has
defined a pair of entropy and relative entropy measures in [6]: $\alpha$-order
uncertainty as $f(-t)=\exp((1-\alpha)t)$-mean and $\alpha$-order $I$-relative
information as $f(t)=\exp((\alpha-1)t)$-mean, later denoted by Rényi entropy
$H_{\alpha}(\cdot)$ and Rényi divergence $D_{\alpha}(\cdot\|\cdot)$,
respectively.

$\displaystyle
D_{\alpha}(\mathbf{P}_{X}(\mathcal{B})\|\mathbf{Q}_{X}(\mathcal{B}))$ (1)
$\displaystyle\scalebox{1.1}{$=f^{-1}\Big{(}\sum\limits_{x\in\mathcal{B}}\frac{P_{X}(x)}{\sum\limits_{x\in\mathcal{B}}P_{X}(x)}f\big{(}D_{\alpha}(P_{X}(x)\|Q_{X}(x))\big{)}\Big{)}$},$

where $P_{X}(x)/\sum_{x\in\mathcal{B}}P_{X}(x),\forall x\in\mathcal{B}$ is a
normalized probability for each $\mathcal{B}$. The elementary information gain
at each $x\in\mathcal{X}$ is still obtained by (1) as

$D_{\alpha}(P_{X}(x)\|Q_{X}(x))=\log\frac{P_{X}(x)}{Q_{X}(x)}.$ (2)

Note, elementary information gain is independent of $\alpha$. It is the
logarithm of Radon–Nikodym derivative [9, eq.(6)] and called information lift
in [13, 14, 15, 16, 17]. The well known Rényi divergence expression is the
definition (1) for $\mathcal{B}=\mathcal{X}$:

$\displaystyle D_{\alpha}(\mathbf{P}_{X}\|\mathbf{Q}_{X})$
$\displaystyle=\frac{1}{\alpha-1}\log\sum_{x\in\mathcal{X}}P_{X}(x)\Big{(}\frac{P_{X}(x)}{Q_{X}(x)}\Big{)}^{\alpha-1}$
(3)
$\displaystyle=f^{-1}(\mathbb{E}_{P_{X}}[f\big{(}D_{\alpha}(P_{X}(x)\|Q_{X}(x))\big{)}]).$

This relative information quantifies the expected uncertainty in
$\mathbf{P}_{X}$ in addition to $\mathbf{Q}_{X}$, where the expectation is
taken w.r.t. $\mathbf{P}_{X}$ denoting the probability of each outcome $X=x$.
Therefore, $D_{\alpha}(\mathbf{P}_{X}\|\mathbf{Q}_{X})$ is specifically called
$\alpha$-order information gain in [8].

##  III $\tilde{f}$-mean Information Gain

The role of $\mathbf{P}_{X}$ in Rényi divergence (3) is two-fold: the
probability to be measured, where the multiplicative information gain or the
exponential of elementary information gain is raised to order $\alpha-1$:
$\exp((\alpha-1)D_{\alpha}(P_{X}(x)\|Q_{X}(x)))=(P_{X}(x)/Q_{X}(x))^{\alpha-1}$;
the probability that indicates the appearance frequency of each elementary
information gain.

Let $\tilde{f}(t)=\exp(\frac{\alpha-1}{\alpha}t)$. We propose a new
information gain measure as a $\tilde{f}$-mean, where the probability to be
measured is different from frequency probability. Assume that we want to
quantify the information increase in $\boldsymbol{\Phi}_{X}$ from a reference
probability $\mathbf{Q}_{X}$, where another probability $\mathbf{P}_{X}$
governs how often the relative information appears at each elementary event
$x$. For each $\mathcal{B}\subseteq\mathcal{X}$, the $\tilde{f}$-mean
information gain is

$\displaystyle\tilde{D}_{\alpha}\big{(}\boldsymbol{\Phi}_{X}(\mathcal{B})\|\mathbf{Q}_{X}(\mathcal{B})|\mathbf{P}_{X}(\mathcal{B})\big{)}$
(4)
$\displaystyle\scalebox{1.2}{$=\tilde{f}^{-1}\Big{(}\sum\limits_{x\in\mathcal{B}}\frac{P_{X}(x)}{\sum\limits_{x\in\mathcal{B}}P_{X}(x)}\tilde{f}\big{(}\tilde{D}_{\alpha}(\Phi_{X}(x)\|Q_{X}(x))\big{)}\Big{)}$},$

where the elementary information gain
$\tilde{D}_{\alpha}(\Phi_{X}(x)\|Q_{X}(x))=\log\frac{\Phi_{X}(x)}{Q_{X}(x)}$
equals (2).222The elementary measure is always independent of $\alpha$. This
is because for deterministic $Z$, the $f$-mean
$\bar{Z}=f^{-1}(\mathbb{E}[f(Z)])=Z$ is independent of $f$. We keep the
subscript $\alpha$ in elementary measures $D_{\alpha}(P_{X}(x)\|Q_{X}(x))$ and
$\tilde{D}_{\alpha}(\Phi_{X}(x)\|Q_{X}(x))$ only to show that they can be
obtained by the prototype definitions (1) and (4), respectively. For
$\mathcal{B}=\mathcal{X}$, we have

$\displaystyle\tilde{D}_{\alpha}\big{(}\boldsymbol{\Phi}_{X}\|\mathbf{Q}_{X}|\mathbf{P}_{X}\big{)}=$
$\displaystyle\begin{cases}\frac{\alpha}{\alpha-1}\log\sum_{x\in\mathcal{X}}P_{X}(x)\Big{(}\frac{\Phi_{X}(x)}{Q_{X}(x)}\Big{)}^{\frac{\alpha-1}{\alpha}}&\alpha\in(0,1)\cup(1,\infty),\\\
\log\min_{x\in\text{supp}(P_{X})}\frac{\Phi_{X}(x)}{Q_{X}(x)}&\alpha=0,\\\
\sum_{x\in\text{supp}(P_{X})}P_{X}(x)\log\frac{\Phi_{X}(x)}{Q_{X}(x)}&\alpha=1,\\\
\log\sum_{x\in\mathcal{X}}P_{X}(x)\frac{\Phi_{X}(x)}{Q_{X}(x)}&\alpha=\infty.\end{cases}$

We show in Proposition 1 below that for given reference probability
$\mathbf{Q}_{X}$ and frequency probability $\mathbf{P}_{X}$, the maximum
$\tilde{f}$-mean information gain is attained at Rényi divergence. This
proposition will be used to derive the main result Theorem 1 in Section IV.

###### Proposition 1

For all $\alpha\in[0,\infty)$,

$D_{\alpha}(\mathbf{P}_{X}\|\mathbf{Q}_{X})=\max_{\boldsymbol{\Phi}_{X}}\tilde{D}_{\alpha}\big{(}\boldsymbol{\Phi}_{X}\|\mathbf{Q}_{X}|\mathbf{P}_{X}\big{)}$
(5)

with the maximizer
$\Phi_{X}^{*}(x)=\frac{P_{X}^{\alpha}(x)/Q_{X}^{\alpha-1}(x)}{\sum_{x}P_{X}^{\alpha}(x)/Q_{X}^{\alpha-1}(x)}$
for all $x$.

###### Proof:

For $\alpha\in(1,\infty)$, $\frac{\alpha-1}{\alpha}\in(0,1)$; for
$\alpha\in(0,1)$, $\frac{\alpha-1}{\alpha}\in(-\infty,0)$. Then,

$\displaystyle\max_{\boldsymbol{\Phi}_{X}}\tilde{D}_{\alpha}\big{(}\boldsymbol{\Phi}_{X}\|\mathbf{Q}_{X}|\mathbf{P}_{X}\big{)}$
$\displaystyle=\max_{\boldsymbol{\Phi}_{X}}\log\Big(\sum_{x\in\mathcal{X}}P_{X}(x)\big{(}\frac{\Phi_{X}(x)}{Q_{X}(x)}\big{)}^{\frac{\alpha-1}{\alpha}}\Big{missing})^{\frac{\alpha}{\alpha-1}}$
$\displaystyle=\begin{cases}\log\big(\max\limits_{\boldsymbol{\Phi}_{X}}\sum\limits_{x\in\mathcal{X}}P_{X}(x)\big{(}\frac{\Phi_{X}(x)}{Q_{X}(x)}\big{)}^{\frac{\alpha-1}{\alpha}}\big{missing})^{\frac{\alpha}{\alpha-1}}&\alpha\in(1,\infty),\\\
\log\big(\min\limits_{\boldsymbol{\Phi}_{X}}\sum\limits_{x\in\mathcal{X}}P_{X}(x)\big{(}\frac{\Phi_{X}(x)}{Q_{X}(x)}\big{)}^{-\frac{1-\alpha}{\alpha}}\big{missing})^{\frac{\alpha}{\alpha-1}}&\alpha\in(0,1).\end{cases}$

In both cases, the optimization is convex programming with the solution being
$\boldsymbol{\Phi}_{X}^{*}$. At extended orders,

$\displaystyle\max_{\boldsymbol{\Phi}_{X}}\tilde{D}_{0}\big{(}\boldsymbol{\Phi}_{X}\|\mathbf{Q}_{X}|\mathbf{P}_{X}\big{)}$
$\displaystyle=\max_{\boldsymbol{\Phi}_{X}}\log\min_{x\in\text{supp}(P_{X})}\frac{\Phi_{X}(x)}{Q_{X}(x)}$
$\displaystyle=D_{0}(\mathbf{P}_{X}\|\mathbf{Q}_{X}),$
$\displaystyle\max_{\boldsymbol{\Phi}_{X}}\tilde{D}_{1}\big{(}\boldsymbol{\Phi}_{X}\|\mathbf{Q}_{X}|\mathbf{P}_{X}\big{)}$
$\displaystyle=\max_{\boldsymbol{\Phi}_{X}}\sum_{x\in\text{supp}(P_{X})}P_{X}(x)\log\frac{\Phi_{X}(x)}{Q_{X}(x)}$
$\displaystyle=D_{1}(\mathbf{P}_{X}\|\mathbf{Q}_{X}),$
$\displaystyle\max_{\boldsymbol{\Phi}_{X}}\tilde{D}_{\infty}\big{(}\boldsymbol{\Phi}_{X}\|\mathbf{Q}_{X}|\mathbf{P}_{X}\big{)}$
$\displaystyle=\max_{\boldsymbol{\Phi}_{X}}\log\sum_{x\in\mathcal{X}}P_{X}(x)\frac{\Phi_{X}(x)}{Q_{X}(x)}$
$\displaystyle=\max_{x\in\mathcal{X}}\log\frac{P_{X}(x)}{Q_{X}(x)}=D_{\infty}(\mathbf{P}_{X}\|\mathbf{Q}_{X}),$

with the maximizers $\boldsymbol{\Phi}_{X}^{*}=\mathbf{Q}_{X}$ for $\alpha=0$,
$\boldsymbol{\Phi}_{X}^{*}=\mathbf{P}_{X}$ for $\alpha=1$ and
$\Phi_{X}^{*}(x)=1/|\operatorname*{arg\,max}_{x}\frac{P_{X}(x)}{Q_{X}(x)}|$ if
$x\in\operatorname*{arg\,max}_{x}\frac{P_{X}(x)}{Q_{X}(x)}$ and 0 otherwise
for $\alpha\rightarrow\infty$. ∎

##  IV $\alpha$-Leakage: Maximum Information Gain

Information leakage is defined as an estimation problem as follows [1, 3, 4,
10]. Given a privacy-preserving channel $\mathbf{P}_{Y|X}$, an input $X$ will
induce a channel output $Y$ that is accessible to all users, including
malicious ones. An adversary can obtain an estimation of $X$, denoted by
$\hat{X}$, by applying a soft decision $P_{\hat{X}|Y}$ to $Y$. This induces a
Markov chain $X-Y-\hat{X}$. For $\mathbf{P}_{X}$ being the adversary’s prior
belief, information gain is measurable for each decision or posterior belief
$\mathbf{P}_{\hat{X}|Y}$. The adversary will seek the optimal decision
$\mathbf{P}_{\hat{X}|Y}^{*}$ that maximizes the information gain, where the
maximum indicates the worst-case amount of information on $X$ that is leaked
to the adversary and is defined as information leakage.

For each $\mathbf{P}_{X}$ and $\mathbf{P}_{Y|X}$, the Sibson mutual
information [8]

$I_{\alpha}^{\text{S}}(\mathbf{P}_{X},\mathbf{P}_{Y|X})=\frac{\alpha}{\alpha-1}\log\sum_{y\in\mathcal{Y}}\Big{(}\sum_{x\in\mathcal{X}}P_{X}(x)P_{Y|X}^{\alpha}(y|x)\Big{)}^{\frac{1}{\alpha}}$

is the information radius of $f$-mean Rényi divergence.333Information radius,
as defined in [8, Sec. 2], is a probability distribution that minimizes
$f$-mean Rényi divergence from a given set of probabilities.  The following
theorem shows that the Rényi divergence is the maximum $\tilde{f}$-mean
information gain incurred at each channel output $y\in\mathcal{Y}$. We call it
$Y$-elementary information leakage. Sibson mutual information is then
interpreted as the $\tilde{f}$-mean of this $Y$-elementary information
leakage.

###### Theorem 1

For all $\alpha\in[0,\infty)$,

$\displaystyle D_{\alpha}(\mathbf{P}_{X|Y=y}\|\mathbf{P}_{X})$
$\displaystyle\quad\quad=\max_{\mathbf{P}_{\hat{X}|Y=y}}\tilde{D}_{\alpha}\big{(}\mathbf{P}_{\hat{X}|Y=y}\|\mathbf{P}_{X}|\mathbf{P}_{X|Y=y}\big{)},\
\forall y\in\mathcal{Y},$ (6) $\displaystyle
I_{\alpha}^{\text{S}}(\mathbf{P}_{X},\mathbf{P}_{Y|X})$
$\displaystyle\quad\quad=\max_{\mathbf{P}_{\hat{X}|Y}}\tilde{D}_{\alpha}\big{(}\mathbf{P}_{\hat{X}|Y=y}\|\mathbf{P}_{X}|\mathbf{P}_{Y|X}\otimes\mathbf{P}_{X}\big{)},$
(7)

with the maximizer

$P_{\hat{X}|Y}^{*}(x|y)=\frac{P_{X|Y}^{\alpha}(x|y)/P_{X}^{\alpha-1}(x)}{\sum_{x}P_{X|Y}^{\alpha}(x|y)/P_{X}^{\alpha-1}(x)}$

for all $(x,y)\in\mathcal{X}\times\mathcal{Y}$. In (7),
$\mathbf{P}_{Y|X}\otimes\mathbf{P}_{X}(x,y)=(P_{Y|X}(y|x)P_{X}(x)\colon(x,y)\in\mathcal{X}\times\mathcal{Y})$.

###### Proof:

Equation (6) is a direct result of Proposition 1. For Sibson mutual
information,

$\displaystyle I_{\alpha}^{\text{S}}(X;Y)$ (8) (9) (10)

where $\mathbf{P}_{Y}$ is the channel output probability such that
$P_{Y}(y)=\sum_{x\in\mathcal{X}}P_{Y|X}(y|x)P_{X}(x),\forall y\in\mathcal{Y}$.
The maximand in (9) is a $\tilde{f}$-mean of $Y$-elementary
$D_{\alpha}(\mathbf{P}_{X|Y=y}\|\mathbf{P}_{X})$; the maximand in (10) is a
$\tilde{f}$-mean of $XY$-elementary information gain
$D_{\alpha}(\mathbf{P}_{X|Y}(y|x)\|\mathbf{P}_{X}(x))$ given the frequency
probability $\mathbf{P}_{Y|X}\otimes\mathbf{P}_{X}$, which is denoted by
$\tilde{D}_{\alpha}\big{(}\mathbf{P}_{\hat{X}|Y=y}\|\mathbf{P}_{X}|\mathbf{P}_{Y|X}\otimes\mathbf{P}_{X}\big{)}$.
The maximizer of (10) is $\mathbf{P}_{\hat{X}|Y}^{*}$ by Proposition 1. ∎

###  IV-A Existing $\alpha$-Leakage

The information-theoretic privacy leakages in [3, 4, 10] are actually defined
based on Markov chain $U-X-Y-\hat{U}$, where $U$ is a sensitive attribute of
input data $X$ and the adversary want to gain information on $U$. In this
case, simply substituting $U$ to $X$ in (6) and (7), we have the information
leakage measures from $U$ to $Y$. They are upper bounded by the leakages from
$X$ to $Y$.

###### Corollary 1

Assume Markov chain $U-X-Y-\hat{U}$. For all $\alpha\in[0,\infty)$,444The
supremum $\sup_{\mathbf{P}_{U}}$ in Section IV-A is over all $U$ such that
Markov chain $U-X-Y-\hat{U}$ is formed for fixed $\mathbf{P}_{X}$ and
$\mathbf{P}_{Y|X}$.

$\displaystyle\sup_{\mathbf{P}_{U}}D_{\alpha}(\mathbf{P}_{U|Y=y}\|\mathbf{P}_{U})=D_{\alpha}(\mathbf{P}_{X|Y=y}\|\mathbf{P}_{X}),\
\forall y\in\mathcal{Y},$ (11)
$\displaystyle\sup_{\mathbf{P}_{U}}I_{\alpha}^{\text{S}}(\mathbf{P}_{U},\mathbf{P}_{Y|U})=I_{\alpha}^{\text{S}}(\mathbf{P}_{X},\mathbf{P}_{Y|X}).$
(12)

The proof is omitted as Corollary 1 just recites the post-processing
inequality of Rényi divergence and Sibson mutual information [18, 19]. Similar
results can be found in [4, Ths.1&2], [3, Th.1], [10, Th.1] for a different
notion of $\alpha$-leakage: optimal estimation decision is obtained separately
for prior and posterior belief and the difference in the resulting information
gain, or uncertainty reduction, defines the leakage. According to [5], the
$\alpha$-leakage defined in [4]
is555$I_{\alpha}^{\text{S}}(\mathbf{P}_{U_{\alpha}},\mathbf{P}_{Y|U})$ is the
Arimoto mutual information [7], i.e., the authors in [4] actually reveal an
interpretation of Arimoto mutual information in privacy leakage.

$\displaystyle\mathcal{L}_{\alpha}(U$ $\displaystyle\rightarrow Y)$
$\displaystyle=I_{\alpha}^{\text{S}}(\mathbf{P}_{U_{\alpha}},\mathbf{P}_{Y|U})$
$\displaystyle=\frac{\alpha}{\alpha-1}\log\mathbb{E}_{P_{Y}}\big{[}\exp\big(\frac{\alpha-1}{\alpha}D_{\alpha}(\mathbf{P}_{U_{\alpha}|Y=y}\|\mathbf{P}_{U_{\alpha}})\big{missing})\big{]},$

where $\mathbf{P}_{U_{\alpha}}$ is a scaled probability of $\mathbf{P}_{U}$
such that
$P_{U_{\alpha}}(u)=\frac{P_{U}^{\alpha}(u)}{\sum_{u\in}P_{U}^{\alpha}(u)}$ for
all $u\in\mathcal{U}$ and
$P_{U_{\alpha}|Y}(u|y)=\frac{P_{Y|U}(y|u)P_{U_{\alpha}}(u)}{P_{Y}(y)}$ for all
$(u,y)\in\mathcal{U}\times\mathcal{Y}$. Then,

$\displaystyle\sup_{\mathbf{P}_{U_{\alpha}}}D_{\alpha}(\mathbf{P}_{U_{\alpha}|Y=y}\|\mathbf{P}_{U_{\alpha}})$
$\displaystyle\qquad\qquad\qquad=\sup_{\mathbf{P}_{U}}D_{\alpha}(\mathbf{P}_{U|Y=y}\|\mathbf{P}_{U}),\quad\forall
y\in\mathcal{Y}$ (13)
$\displaystyle\sup_{\mathbf{P}_{U_{\alpha}}}I_{\alpha}^{\text{S}}(\mathbf{P}_{U_{\alpha}},\mathbf{P}_{Y|U})=\sup_{\mathbf{P}_{U}}I_{\alpha}^{\text{S}}(\mathbf{P}_{U},\mathbf{P}_{Y|U}).$
(14)

The first equality is because
$D_{\alpha}(\mathbf{P}_{U_{\alpha}|Y}\|\mathbf{P}_{U_{\alpha}})=\frac{1}{\alpha-1}\log\sum_{u\in\mathcal{U}}P_{U_{\alpha}}(u)P_{Y|U}^{\alpha}(y|u)/P_{Y}^{\alpha}(y)$.
Equation (14) is the equivalence of Arimoto and Sibson mutual information when
they are maximized over channel input [19, Th.5][20]. It is used to prove
$\sup_{\mathbf{P}_{U}}\mathcal{L}_{\alpha}(U\rightarrow
Y)=I_{\alpha}^{\text{S}}(\mathbf{P}_{X},\mathbf{P}_{Y|X})$ via (12) in [4,
Appendix A]. Clearly from (11) and (13), the $Y$-elementary leakage is also
upper bounded as
$\sup_{\mathbf{P}_{U_{\alpha}}}D_{\alpha}(\mathbf{P}_{U_{\alpha}|Y=y}\|\mathbf{P}_{U_{\alpha}})=D_{\alpha}(\mathbf{P}_{X|Y=y}\|\mathbf{P}_{X})$.
This equality for $\alpha=\infty$ is proved in [10], where
$D_{\infty}(\mathbf{P}_{X|Y=y}\|\mathbf{P}_{X})$ is called pointwise maximal
leakage.

##  V Acknowledgement

The first author would like to thank A/Prof Xiangyun (Sean) Zhou for handling
an authorship dispute raised two days before ITW2024 deadline. This paper was
revised from the original sole author paper by removing a whole section.

##  VI Conclusion

We proposed a $\tilde{f}$-mean information gain that quantifies the
information in a probability distribution $\boldsymbol{\Phi}_{X}$ in addition
to a reference probability $\mathbf{Q}_{X}$ conditioned on a frequency
probability $\mathbf{P}_{X}$. We proved that the maximum of the
$\tilde{f}$-mean information gain is attained at Rényi divergence between
$\mathbf{Q}_{X}$ and $\mathbf{P}_{X}$. This result was used to propose Rényi
divergence and Sibson mutual information as $\alpha$-leakage measures.

With the cross entropy proposed in [5], we have a pair of $\tilde{f}$-mean
information measures corresponding to the existing $f$-mean measures, Rényi
entropy and divergence. We have shown in Theorem 1 and [5, Th. 1] that the
optimization of $\tilde{f}$-mean measures give $f$-mean measures. The
interplay between $f$\- and $\tilde{f}$-mean should be further explored.

## References

  * [1] F. du Pin Calmon and N. Fawaz, “Privacy against statistical inference,” in _Proceedings of 50th Annual Allerton Conference on Communication, Control, and Computing_ , Monticello, IL, 2012, pp. 1401–1408.
  * [2] L. Sankar, S. R. Rajagopalan, and H. V. Poor, “Utility-privacy tradeoffs in databases: An information-theoretic approach,” _IEEE Transactions on Information Forensics and Security_ , vol. 8, no. 6, pp. 838–852, Jun. 2013.
  * [3] I. Issa, A. B. Wagner, and S. Kamath, “An operational approach to information leakage,” _IEEE Transactions on Information Theory_ , vol. 66, no. 3, pp. 1625–1657, Mar. 2020.
  * [4] J. Liao, O. Kosut, L. Sankar, and F. du Pin Calmon, “Tunable measures for information leakage and applications to privacy-utility tradeoffs,” _IEEE Transactions on Information Theory_ , vol. 65, no. 12, pp. 8043–8066, Dec. 2019.
  * [5] N. Ding, M. A. Zarrabian, and P. Sadeghi, “A cross entropy interpretation of rényi entropy for $\alpha$-leakage,” in _Proceedings of IEEE International Symposium on Information Theory_ , Athens, 2024. [Online]. Available: https://arxiv.org/abs/2401.15202
  * [6] A. Rényi, “On measures of entropy and information,” in _Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics_ , vol. 4. University of California Press, 1961, pp. 547–562.
  * [7] S. Arimoto, “Information measures and capacity of order $\alpha$ for discrete memoryless channels,” _Topics in information theory_ , 1977.
  * [8] R. Sibson, “Information radius,” _Zeitschrift für Wahrscheinlichkeitstheorie und verwandte Gebiete_ , vol. 14, no. 2, pp. 149–160, 1969.
  * [9] S. Verdú, “Error exponents and $\alpha$-mutual information,” _Entropy_ , vol. 23, no. 2, p. 199, Feb. 2021.
  * [10] S. Saeidian, G. Cervia, T. J. Oechtering, and M. Skoglund, “Pointwise maximal leakage,” _IEEE Transactions on Information Theory_ , vol. 69, no. 12, pp. 8054–8080, Dec. 2023.
  * [11] A. Kołgomorov, “Sur la notion de la moyenne, alti accad,” _Naz. Lincei_ , vol. 12, no. 6, pp. 388–391, 1930.
  * [12] M. Nagumo, “Über eine klasse der mittelwerte,” _Japanese journal of mathematics: transactions and abstracts_ , vol. 7, no. 0, pp. 71–79, 1930.
  * [13] N. Ding, M. A. Zarrabian, and P. Sadeghi, “$\alpha$-information-theoretic privacy watchdog and optimal privatization scheme,” in _Proceedings of IEEE International Symposium on Information Theory_ , Melbourne, 2021, pp. 2584–2589.
  * [14] M. A. Zarrabian, N. Ding, P. Sadeghi, and T. Rakotoarivelo, “Enhancing utility in the watchdog privacy mechanism,” in _Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing_ , Singapore, 2022, pp. 2979–2983.
  * [15] H. Hsu, S. Asoodeh, and F. P. Calmon, “Information-theoretic privacy watchdogs,” in _Proceedings of IEEE International Symposium on Information Theory_ , Paris, 2019, pp. 552–556.
  * [16] M. A. Zarrabian, N. Ding, and P. Sadeghi, “Asymmetric local information privacy and the watchdog mechanism,” in _Proceedings of IEEE Information Theory Workshop_ , Mumbai, 2022, pp. 7–12.
  * [17] ——, “On the lift, related privacy measures, and applications to privacy–utility trade-offs,” _Entropy_ , vol. 25, no. 4, p. 679, Apr. 2023\.
  * [18] Y. Polyanskiy and S. Verdu, “Arimoto channel coding converse and Rényi divergence,” in _Proceedings of 48th Annual Allerton Conference on Communication, Control, and Computing_ , Monticello, IL, 2010, pp. 1327–1333.
  * [19] S. Verdú, “$\alpha$-mutual information,” in _Proceedings of Information Theory and Applications Workshop_ , San Diego, CA, 2015, pp. 1–6.
  * [20] I. Csiszar, “Generalized cutoff rates and Rényi’s information measures,” _IEEE Transactions on Information Theory_ , vol. 41, no. 1, pp. 26–34, Jan. 1995.


# Quarkyonic Mean Field Theory

Dyana C. Duarte  Departamento de Física, Universidade Federal de Santa Maria,
Santa Maria, RS 97105-900, Brazil  Departamento de Física, Instituto
Tecnológico de Aeronáutica, São José dos Campos, SP, 12228-900, Brazil  Saul
Hernandez-Ortiz  Department of Physics and Astronomy, Iowa State University,
Ames, IA 50010  Instituto de Física y Matemáticas, Universidad Michoacana de
San Nicolás de Hidalgo, Edificio C-3, Ciudad Universitaria, Francisco J.
Mújica S/N Col. Felícitas del Río, 58040 Morelia, Michoacán, México.  Kie Sang
Jeong  Larry D. McLerran<EMAIL_ADDRESS>Institute for Nuclear Theory,
University of Washington, Box 351550, Seattle, WA, 98195, USA

###### Abstract

We discuss mean field theory of Quarkyonic matter at zero temperature. We
treat the nucleons with contact interactions in mean field approximation,
discussing both vector and scalar mean field interactions. We treat the quarks
without mean field vector interactions, but allow mass terms to be generated
consistent from a scalar mean field consistent with the additive quark model
for quark masses. Quarkyonic matter is composed of a shell of nucleons that
under-occupy the total available phase space associated with the underlying
quark degrees of freedom. The fully occupied Fermi sphere beneath this shell
of nucleons at high densities is thought of as quarks, but when this fully
occupied distribution of states first appears, although the phase space is
filled, the matter is at low density. For the transition between this low
density and high density saturated matter, we advocate a dual description of
the fully filled Fermi sea in terms of hadrons, and make a phenomenological
hypothesis for the equation of state of this matter. We then proceed to an
example where the mean field interactions are all vector and only associated
with the nucleons, ignoring the effects of mass change associated with the
scalar interactions. Except for the effects of Pauli blocking, the nucleons
and quarks do not interact. To get a reasonable transition to Quarkyonic
matter the interaction of the quarks among themselves are assumed to be non-
perturbative, and a simple phenomenological relation between quark Fermi
energy and density is introduced.

††preprint: INT-PUB-23-003

##  I Introduction

In a recent paper, we argued that Quarkyonic MatterMcLerran:2007qj  might be
described by a field theory with nucleon, ghost and quark degrees of
freedomDuarte:2021tsx . The ghosts are introduced to avoid double counting of
states where the quarks inside of nucleons might occupy the same physical
states as those associated with quarks. Such a picture has been clearly
elaborated in the work of Kojo Kojo:2021ugu ; Kojo:2021hqh , and was the
starting point of various previous studies McLerran:2018hbz ; Jeong:2019lhv ;
Sen:2020peq ; Cao:2020byn ; Kovensky:2020xif .

In the field theory with ghosts, there are three chemical potentials that need
to be determined, the nucleon chemical potential that are $\mu_{N}$, the ghost
chemical potential, $\mu_{G}$ and the quark chemical potential $\mu_{Q}$. The
ghost chemical potential is determined in terms of the quarks to avoid double
counting of states. The quark chemical potential is determined by extremizing
the pressure of the system, or equivalently at zero temperature, minimizing
the energy per nucleon at fixed total baryon number. Finally, the nucleon
chemical potential is determined by the total baryon number of the system.

It is the purpose of this paper to understand how all of this might work in
detail in a simple model. We will consider a mean field theory of nucleon
interactions interacting with a vector potential. We show how a scalar mean
field might modify these considerations, but we explicitly consider only the
case of vector mean field interactions. The quarks are introduced by low
energy QCD effective model. In our simple model, they are allowed to interact
among themselves but not with the nucleons. Their interactions are strong, and
modify the free quark relation between density and chemical potential. We will
introduce a simple phenomenological parameterization of this relationship. The
ghost fields enforce the constraint that nucleons and quarks do not occupy the
same phase space. At high density, the quarks are an almost free gas of
quarks, but at low density when they first appear, the filled Fermi sea of
quarks may be thought of as a gas of nucleons and their excited states that
completely fill the quark energy levels.

We then turn to the issue of computing the properties of Quarkyonic matter in
the limit of a large number of colors $N_{c}$ tHooft:1973alw ; Witten:1979kh .
We show that if interactions of quarks are ignored, and if the constituent
quark masses are $M_{Q}=M_{N}/N_{c}$ and the nucleons are treated in mean
field, then Quarkyonic Matter does not form. There are only two phases: one of
nucleonic matter at low density and the other of quarks at high density.

We then argue that the region where all the quark states are filled may be
thought of as matter state composed of the nucleon and nucleon resonances
where the interdistance between the baryons is short so that the quasi-baryon
properties can be approximated by the constituent quark dynamics (quark-hadron
duality). In this configuration, all the available quark state looks fully
occupied since the independent constituent quark wave functions from the
nucleon ground state and higher resonances interfere with each other and
double occupancy of same quantum number is prohibited by Pauli principle. We
then show that in this circumstance, if this exotic state is treated as a non-
interacting gas of constituent quarks, then there is a first order phase
transition to Quarkyonic Matter.

When the compressed state understood as the filled quark sea first appears,
the typical momentum scale is small and interactions cannot be ignored. We
construct a simple model of the properties of this low density matter in a
density expansion around zero density. We argue that it is possible that the
effect of such interactions may convert the transition between nucleonic
matter into Quarkyonic matter as a continuous transition. This continuous
transition is needed to phenomenologically explain the rapid rise of the sound
velocity extracted from equations of state of nuclear matter appropriate for
neutron stars Drischler:2021bup ; Tews:2018kmu ; Steiner:2017vmg ;
Bedaque:2014sqa .

We should emphasize at the outset, that our goal is not to provide a
phenomenologically viable model of Quarkyonic matter. This paper is simply an
exploration of how Quarkyonic matter might appear in a field theory with both
nucleon and quark degrees of freedom present, but not allowing simultaneous
multiple occupation of phase space of the quarks within nucleon with those of
the filled Fermi sea of quarks. It is a first very small step towards
constructing viable theories at finite density and temperatures, that properly
include the effect of interactions, and the low occupation number nucleon
states at the Fermi surface.

##  II Review of Mean Field Theory for Applications to Quarkyonic Matter

Before proceeding to a mean field description of Quarkyonic Matter, we review
the basic ingredients of mean field theory that we will use in the following.
We will take all of the nucleon interactions to be given by contact
interactions. First consider a single species of baryons at finite density.
The theory with a scalar interaction is

$S=\int~{}d^{4}x\left\\{\overline{\psi}\left({1\over
i}\not{\partial}-\gamma^{0}\mu^{*}+M-g_{s}\sigma\right)\psi+{M_{s}^{2}\over
2}\sigma^{2}\right\\}\,.$ (1)

In the entire paper, $\mu^{*}$ denotes the bare chemical potentials, without
any kind of interactions. The inclusion of the scalar field generates a
nucleon-nucleon contact interaction. Integrating out the scalar field gives

$S=\int~{}d^{4}x\left\\{\overline{\psi}\left({1\over
i}\not{\partial}-\gamma^{0}\mu^{*}+M\right)\psi-{g_{s}^{2}\over{2M_{s}^{2}}}(\overline{\psi}\psi)^{2}\right\\}\,.$
(2)

In mean field approximation, we replace

$\overline{\psi}\psi\rightarrow\overline{\psi}\psi+n_{s}\,,$ (3)

where

$n_{s}=\langle\overline{\psi}\psi\rangle\,.$ (4)

The action becomes

$\displaystyle S$ $\displaystyle=$
$\displaystyle\int~{}d^{4}x\Biggl{\\{}\overline{\psi}\left({1\over
i}\not{\partial}-\gamma^{0}\mu^{*}+M_{\text{eff}}\right)\psi$ (5)
$\displaystyle-{g_{s}^{2}\over{2M_{s}^{2}}}\left[(\overline{\psi}\psi)^{2}+\langle\overline{\psi}\psi\rangle^{2}\right]\Biggl{\\}}\,.$

The mean field approximation consists of including the explicit term that
involves the contribution of $\langle\overline{\psi}\psi\rangle^{2}$ and the
contribution of the ideal gas term with the
$\langle\overline{\psi}\psi\rangle$ computed in mean field approximation. For
the energy density, we obtain

$\epsilon=\epsilon_{\text{kin}}-{g_{s}^{2}\over{2M_{s}^{2}}}n_{s}^{2}\,.$ (6)

The kinetic energy is

$\epsilon_{\text{kin}}=\int{{d^{3}p}\over{(2\pi)^{3}}}~{}n(p)~{}\sqrt{p^{2}+M^{2}_{\text{eff}}}\,,$
(7)

and

$n_{s}=\int~{}{{d^{3}p}\over{(2\pi)^{3}}}~{}n(p)~{}{M_{\text{eff}}\over\sqrt{p^{2}+M_{\text{eff}}^{2}}}\,.$
(8)

For the vector field treatment, the chemical potential is the maximum energy
difference beyond the ground state energy. The vector potential shifts the
overall zero of energy by the interactions energy with the zeroth component of
the vector field $g_{v}V^{0}$. The vector field shifts the ground state energy
of the fermion by a constant in mean field approximation. An overall shift by
a constant does not affect the one loop contribution to the action. The only
effect is to include the interaction energy in the mean field energy
functional. So the result is

$\epsilon=\epsilon_{\text{kin}}+{g_{v}^{2}\over{2M_{v}^{2}}}n_{v}^{2}-{g_{s}^{2}\over{2M_{s}^{2}}}n_{s}^{2}\,,$
(9)

where

$n_{v}=\int~{}{{d^{3}p}\over{(2\pi)^{3}}}(n_{\text{particle}}-n_{\text{antiparticle}})\,.$
(10)

At zero temperature, which is the case that we consider through the rest of
this work

$n_{v}=\int~{}{{d^{3}p}\over{(2\pi)^{3}}}\,.$ (11)

For purposes of $N_{c}$ counting, one should note that $g_{s}^{2}$ and
$g_{v}^{2}$ are both of order $N_{c}$.

When we generalize this discussion to Quarkyonic matter, we add in ghost
fields. The ghost fields are bosonic spinor fields. The Lagrangian for the
nucleon, including the effect of ghost is

$\displaystyle S$ $\displaystyle=$
$\displaystyle\int~{}d^{4}x\Biggl{\\{}\overline{\psi}\left({1\over
i}\not{\partial}-g_{v}\not{V}-\gamma^{0}\mu_{N}^{*}+M-g_{s}\sigma\right)\psi$
(12) $\displaystyle+\overline{G}\left({1\over
i}\not{\partial}-g_{v}\not{V}-\gamma^{0}\mu_{G}^{*}+M-g_{s}\sigma\right)G$
$\displaystyle+{M_{s}^{2}\over 2}\sigma^{2}+{M_{V}^{2}\over
2}A^{2}\Biggl{\\}}\,.$

In mean field theory, the energy function for nucleons is modified by the
ghost by $n_{v}\rightarrow n^{N}_{v}-n^{G}_{v}$ and $n_{s}\rightarrow
n_{s}^{N}-n_{s}^{G}$.

For Quarkyonic matter, we need the contributions of the nucleons at chemical
potential $\mu_{B}$, ghosts at chemical potentials $\mu_{G}$. The quarks at
chemical potential $\mu_{Q}$ will be introduced via QCD effective model. In
what follow, we will ignore the QCD interactions of quarks with ghost and
nucleons. They know each other only through the Fermi exclusion principle, and
this will be implemented by requiring the ghost potential is

$\mu_{G}=N_{c}\mu_{Q}+g_{v}V^{0}\,,$ (13)

where $V^{0}=(g_{v}/M_{v}^{2})n_{v}$ is the background vector potential. This
means that ghost states block the states of nucleons where their kinetic
momentum is $k_{G}=N_{c}k_{Q}$, at least in the limit where the constituent
quark mass is $M_{Q}=M_{N}/N_{c}$. If the scalar mean field is included the
nucleon develops an effective mass $M_{\text{eff}}=M_{N}-g_{s}\sigma$ that is
a function of the baryon density. In this case, the scalar condensate $\sigma$
would be obtained by extremizing the pressure with respect to the nucleon
effective mass to obtain

$\sigma=\frac{g_{s}}{m_{s}^{2}}n_{s}\;.$ (14)

In what follows, we will consider several cases. In all we consider, we ignore
the effect of a scalar mean field. A scalar field could be implemented in more
detailed models. We first consider the case where quarks are free and have
$M_{Q}=M_{N}/N_{c}$. We then add $\Delta M_{Q}$ to the quark mass, and finally
we will develop a phenomenological model of the quark interactions. The
reasons for these increasing levels of complication arise because we want to
eventually obtain a model where there is a continuous transition between
nucleonic matter and Quarkyonic matter. The motivation for these
generalizations and their consequences are the subject of the following
sections.

##  III An Ideal Gas of Quarks

At very high density, the picture of Quarkyonic Matter in momentum space is
that of a thin shell of nucleonic matter which surrounds a filled Fermi sea of
quarks. At very high density, the typical momentum of the Fermi sphere of
quarks is large compared to the QCD scale, and one can assume that the gas is
a quasi-free gas of quarks, with interactions controlled by a coupling which
is small at the scale of interest.

When Quarkyonic matter first appears however, we must be careful in such
considerations. As we approach high density in a gas of nucleons, excited
nucleon states begin to occur. They of course fill states with minimal Fermi
momenta, in order to minimize the energy as they first appear. Up to the
intermediate densities, the system may have some independent Fermi sea of
baryons. As yet higher energy density is approached, the ground state nucleons
and the resonances get closer and the constituent quark wave functions start
to interfere with each other. These excited nucleon states should not be
viewed like thermal excitations, but states with different wave functions of
the quark degrees of freedom so that they can fill up all the available vacant
quark energy levels in such a compressed matter configuration. This will
continue until a density is achieved where all of the quark states at lower
momentum in the Fermi sphere are filled. In such a picture, there is indeed a
Fermi surface of nucleon states. The Fermi sphere, eventually corresponding to
all the quark states appearing, is a Fermi sphere surrounded by nucleons. If
instead of thinking about nucleon and excited nucleon degrees of freedom, we
think of the states occupied by excited nucleon states as simply in terms of
quark degrees of freedom, we recover the Quarkyonic description. The excited
nucleon states are dual to the quark description, and thinking in terms of
excited nucleon states is probably most convenient when the nontrivial states
first appear, and the typical momentum scale is not large compared to the QCD
scale. The picture we advocate here is essentially that from Refs.
Kojo:2021ugu ; Kojo:2021hqh ; Park:2019bsz ; Park:2021hqb .

We can imagine various approximate descriptions of this Fermi sphere of filled
quark states. First, in the analysis below, we consider the filled Fermi
sphere to be that of constituent quarks, with non-interacting quarks and the
quark constituent mass to be $M_{Q}=M_{N}/N_{c}$. The nucleons self interact
with a vector mean field.

We will set the coupling of the scalar field for quarks to be $1/N_{c}$ times
that for the scalar field for the nucleons and ghosts, so that the additive
quark parton model is preserved at any density, $M_{N}=N_{c}M_{Q}$. We ignore
the vector interaction for quarks, as we expect that the scale dependent self
interaction of nucleons will be replaced by interactions terms of quark with
gluons. A proper treatment of the quarks for the vector interaction must
inevitably involve a proper treatment of quarks and gluons. The energy density
of the system of quarks, gluons and ghosts is:

$\displaystyle\epsilon$ $\displaystyle=$
$\displaystyle\epsilon_{\text{kin}}^{N}(\mu_{B})-\left(1-{1\over
N_{c}^{3}}\right)\epsilon_{\text{kin}}^{N}(N_{c}\mu_{Q})$
$\displaystyle+{g_{v}^{2}\over{2M_{v}^{2}}}(n_{v}(\mu_{B})-n_{v}(N_{c}\mu_{Q}))^{2}$
$\displaystyle-{g_{s}^{2}\over{2M_{s}^{2}}}\left(n_{s}(\mu_{B})-\left(1-{1\over
N_{c}^{3}}\right)n_{s}(N_{c}\mu_{Q})\right)^{2}.$

The nucleon energy density is $\epsilon^{N}$ and we used the scaling relation
for the masses of quarks to relate the quark energy density at chemical
potential $\mu_{Q}$ to that of the ghosts at chemical potential $N_{c}\mu_{Q}$
Duarte:2021tsx . There is a constraint on the total baryon number:

$n_{B}=n_{v}(\mu_{B})-\left(1-{1\over N_{c}^{3}}\right)n_{v}(N_{c}\mu_{Q})\,.$
(16)

Since the matter we consider in the filled Fermi sphere can be thought of as
the compressed state of the ground state nucleon and resonances, it is
reasonable to assume that a description in terms of constituent quarks has
constituent quarks masses $M_{Q}=M_{N}/N_{c}+\Delta M$. In this case one
should shift the ghost chemical potential by the threshold mass

$\mu_{G}=N_{c}(\mu_{Q}-\Delta M)+g_{v}V^{0}$ (17)

so that we begin having ghost states when the quark states appear
Duarte:2021tsx .

Figure 1: Energy density as a function of quark density
$n_{Q}^{B}=n_{Q}/N_{c}$ including only vector interactions, for the free gas
of quarks comparing $M_{Q}=M_{N}/N_{c}$ (dashed) and $M_{Q}=M_{N}/N_{c}+\Delta
M$ (solid), for different values of total baryon density $n_{B}$. Here we have
used $\Delta M=70$ MeV.

In Fig. 1 we plot the energy density as a function of quark density for both
$M_{Q}=M_{N}/N_{c}$ and $M_{Q}=M_{N}/N_{c}+\Delta M$ and different values of
$n_{B}/\rho_{0}$, where $\rho_{0}$ is the normal nuclear density. The
subtracted term, $\varepsilon(n_{B},n_{Q}^{B}=0)$, only shifts the energy
density to the origin when the total baryon density is zero. When there is no
deviation of quark mass from $M_{N}/N_{c}$ one may see that the global minimum
occurs at $n_{Q}^{B}=0$ or, for high density, at $n_{Q}^{B}=n_{B}$, which
means that there are only two phases: entirely quarks and entirely nucleons,
separated by a first order phase transition. For the case where $\Delta M\neq
0$ the low density matter is nucleonic and at high density it is Quarkyonic,
and they are separated by a first order phase transition. In this figure we
have considered only vector interactions, therefore the energy density is
given by the first two lines of Eq. LABEL:EnDens, and the parameters are
$M_{N}=939$ MeV, $M_{v}=783$ MeV and $g_{v}=8$.

##  IV Quarkyonic Matter in a Phenomenological Model of Quark Interactions

While a description in terms of free constituent quarks may be reasonable at
very high density, it is certainly not at the low densities at which the
filled shell of quark degrees of freedom first appear. Interactions are
important. The final model we consider will be to allow these quark degree of
freedom to have a phenomenological relationship between density and chemical
potential. In particular, at low density, we will assume that

$\mu_{Q}=m_{Q}+{{n_{B}^{Q}}\over\Lambda^{2}}\,,$ (18)

where ${n_{B}^{Q}}$ is the baryon number associated with quarks, and $\Lambda$
is parameter of order of $\Lambda_{\text{QCD}}$. In such a model, we will
assume that there is no mean field associated with the quarks. Although this
look like a harmless low density expansion, it makes the assumption that there
is not free kinetic energy term associated with quarks which would be non-
analytic for small $n_{B}$ and would contribute a term proportional to
$(n_{B}^{Q})^{1/3}$ at high density. Numerically this is implemented as

$\displaystyle\mu_{Q}^{B}$ $\displaystyle=$ $\displaystyle
N_{c}\left[\left(m_{Q}+\frac{a}{m_{Q}^{2}}n_{B}^{Q}\right)\Theta(r-n_{B}^{Q})\right.$
(19)
$\displaystyle\left.+\left(\frac{3\pi^{2}}{2}\right)^{1/3}(n_{B}^{Q})^{1/3}\Theta(n_{B}^{Q}-r)\right]\,,$

where $r$ is the solution of the equation
$\left(m_{Q}+\frac{a}{m_{Q}^{2}}r\right)-\left(\frac{3\pi^{2}}{2}\right)^{1/3}r^{1/3}=0$.
The linear dependence of quark chemical potential with the density was the
simplest form we could find that allowed for a continuous transition between
nuclear matter and Quarkyonic matter, and such behavior is favored by the data
extracted from neutron stars concerning the high density matter equation of
state.

Since there is no free kinectic energy associated with the quarks, it is not
possible to define directly a connection between the quark density $n_{Q}^{B}$
and the quark fermion momentum $k_{Q}$. We follow then the approach suggested
in Ref. Duarte:2021tsx , by requiring the ghost potential $\mu_{G}$ is related
to the quark chemical potential $\mu_{Q}$ as

$\mu_{G}=\mu_{G}^{*}+g_{v}V^{0}\,,\;\;\;\;\;\;\;\;\;\;\;\mu_{G}^{*}=N_{c}\mu_{Q}$
(20)

where $V^{0}=(g_{v}/M_{v}^{2})n_{v}$ is the background vector potential, and
the quark chemical potential is given by Eq. (18). If one considers the
possible larger mass scale of the constituent quarks due to the compressed
mixture of the nucleon and resonances, the ghost chemical potential should be
shifted by the threshold mass:

$\mu_{G}=\mu_{G}^{*}+g_{v}V^{0}\,,\;\;\;\;\;\;\;\;\;\;\;\mu_{G}^{*}=N_{c}(\mu_{Q}-\Delta
M),$ (21)

so that we begin having ghost states when the quark states appear
Duarte:2021tsx . In this approach the ghost Fermi momentum in defined
numerically as $k_{G}=\sqrt{(\mu_{G}^{*})^{2}-m_{G}^{2}}$, and the nucleon
Fermi momentum, $k_{N}$, is defined in terms of the total baryon density
relation $n_{B}=n_{N}-n_{G}+n_{Q}^{B}:$

$k_{N}=\left[\frac{3\pi^{2}}{2}\left(n_{B}+\frac{2}{3\pi^{2}}k_{G}^{3}-n_{Q}^{B}\right)\right]^{1/3}.$
(22)

Figure 2: Left: Energy density as a function of quark density $n_{Q}^{B}$, for
an interacting quark model and different values of total baryon density
$n_{B}$, including only vector interactions in the nucleon sector. Right:
Shell thickness, defined as the difference between the nucleon and ghost Fermi
momenta $k_{N}-k_{G}$.

A continuous transition from nuclear to Quarkyonic matter may be observed by
choosing $\Lambda^{2}=m_{Q}^{2}/a$, where $a$ is a parameter to be adjusted.
By using $a=1.2$, $\Delta M=70$ MeV, and $g_{v}=6$ one may observe from the
left panel of Fig. 2 that between 2-3$\rho_{0}$ a global minimum appears in
the energy density as a function of the quark density $n_{Q}^{B}$ close to the
origin, which is a characteristic of smooth transitions. As the density
increases and the quarks start to take the low phase space, the correspondent
appearance of the ghosts guarantees that quarks inside nucleons do not occupy
the same states where there are quarks, leading to a dynamical formation of a
nucleon shell. This is illustrated on the right panel of Fig. 2: for low
densities, any increment of the total baryon density corresponds to an
increase of the nucleon phase space, whose Fermi momentum is $k_{N}$. Around
$n_{B}\sim 2.2\rho_{0}$ quarks are saturated, and the shell thickness can be
defined as the difference between the nucleon and ghost Fermi momenta,
$k_{N}-k_{G}$. As the density increases, the shell gets thin and it is
expected to disappear at asymptotically large densities when the quarks
eventually deconfine.

In Ref. Koch:2022act  it was shown that in the isospin symmetric quasi-
particle model of baryon-quark mixture, a configuration in which the Fermi sea
is filled with confined baryons surrounded by a shell of deconfined quarks is
energetically favorable compared to quarkyonic matter. This scenario, called
Baryquark matter, represents another possible realization of the Pauli
exclusion principle, where the momentum shell structure is also generated
dynamically, being particularly preferred in transport simulations.

The mean-field approach considered in this work may generate Baryquark matter
if one defines the ghost and the mean-field potential of the nucleons in
different ways. The ghost field for the Baryquark matter should be defined to
exclude the quarks occupying the inner part of the quark Fermi sea and the
weak interaction should be assumed for the constituent quark sea. For the
nucleon interaction side, one may have to consider the attractive scalar mean-
field potential beyond linear density order to match the attractive potential
considered in Baryquark matter Koch:2022act . Also, as the excluded volume
approach in the baryon sector Rischke:1991ke ; Vovchenko:2015vxa  generates
both Quarkyonic and Baryquark configuration Jeong:2019lhv ; Koch:2022act , the
strong repulsive vector channel will be required for the nucleon potential.
The details of model construction will be pursued in the future work on the
transport modelling.

##  V Summary and Conclusions

In this work, we extend the idea of a Quarkyonic matter description in terms
of a field theory composed of nucleons, ghosts, and quarks. The ghost field is
introduced to prevent the overcounting of states where the quarks inside of
nucleons might occupy the same physical states as those associated with the
states in the filled quark sea. To describe Quarkyonic matter, we reviewed the
simplest ingredients in the mean field theory framework required by the
consideration of ghost field and subsequent modification of the interaction
between the nucleons and quarks. In the simple phenomenological point of view,
the filled quark states is approximated as the compressed state of the quasi-
nucleons and its higher mass resonances, where the constituent quark wave
functions start to interfere with each other. In this point of view, it is
natural to take the nonzero threshold mass $\Delta M$ in the modelling of
Quarkyonic matter. The physical role of the threshold mass becomes clear in
the analysis of the free quark sea case. If $\Delta M=0$, there is a first
order phase transition from nuclear to quark phase. If one considers $\Delta
M\neq 0$ from the compressed matter states of the nucleons and resonances a
kind of first order transition to Quarkyonic matter is obtained.

On the other hand, as the interactions of the first quark degrees of freedom
appearing at low intermediate densities are significant, we implemented a
relation between the quark chemical potential and the correspondent quark
density in the form of a low density expansion, which eventually converges to
the free gas approximation at high densities. Under this approximation, the
Quarkyonic phase appears around $n_{B}\sim 2.2\rho_{0}$. As the baryon number
density increases, the shell thickness gets thinner and the thin-shell limit
is expected to appear as claimed in the large $N_{c}$ limit. This assumption
allows a continuous transition between nuclear and Quarkyonic matter that is
suitable to take into account the rapid rise of the sound velocity required in
the density evolution of the equations of state adequate to describe the
observations of neutron stars. Although it is a model dependent result, the
transition may occur at relatively low densities (around $n_{B}\simeq
2.0\rho_{0}$) which implies that the Quarkyonic phase can be explored not only
in the context of the astrophysical observations but also in the future
planned heavy-ion collision experiments Senger:2022bjo ; Lovato:2022vgq .

###### Acknowledgements.

K.S. Jeong, and L. McLerran was supported by the U.S. DOE under Grant No. DE-
FG02- 00ER4113. K.S. Jeong acknowledge the support from the Simons Foundation
on the Multifarious minds Grant No. 557037 to the Institute for Nuclear
Theory. D.C. Duarte also acknowledge Fundação de Amparo à Pesquisa do Estado
de São Paulo (FAPESP) grant No. 17/26111-4. S. Hernandez-Ortiz was supported
by the Department of Energy Nuclear Physics Quantum Horizons program through
the Early Career Award DE-SC0021892. Larry McLerran acknowledges useful
insight provided by Jin-feng Liao, Rob Pisarski and Sanjay Reddy.

## References

  * (1) L. McLerran and R. D. Pisarski, Nucl. Phys. A 796, 83-100 (2007).
  * (2) D. C. Duarte, S. Hernandez-Ortiz, K. S. Jeong and L. D. McLerran, Phys. Rev. D 104, no.9, L091901 (2021).
  * (3) T. Kojo, Phys. Rev. D 104, no.7, 074005 (2021).
  * (4) T. Kojo and D. Suenaga, Phys. Rev. D 105, no.7, 076001 (2022).
  * (5) L. McLerran and S. Reddy, Phys. Rev. Lett. 122, no.12, 122701 (2019).
  * (6) K. S. Jeong, L. McLerran and S. Sen, Phys. Rev. C 101, no.3, 035201 (2020).
  * (7) S. Sen and N. C. Warrington, Nucl. Phys. A 1006, 122059 (2021).
  * (8) G. Cao and J. Liao, JHEP 10, 168 (2020).
  * (9) N. Kovensky and A. Schmitt, JHEP 09, 112 (2020).
  * (10) G. ’t Hooft, Nucl. Phys. B 72, 461 (1974).
  * (11) E. Witten, Nucl. Phys. B 160, 57-115 (1979).
  * (12) C. Drischler, S. Han and S. Reddy, Phys. Rev. C 105, no.3, 035808 (2022).
  * (13) I. Tews, J. Carlson, S. Gandolfi and S. Reddy, Astrophys. J. 860, no.2, 149 (2018).
  * (14) A. W. Steiner, C. O. Heinke, S. Bogdanov, C. Li, W. C. G. Ho, A. Bahramian and S. Han, Mon. Not. Roy. Astron. Soc. 476, no.1, 421-435 (2018).
  * (15) P. Bedaque and A. W. Steiner, Phys. Rev. Lett. 114, no.3, 031103 (2015).
  * (16) A. Park, S. H. Lee, T. Inoue and T. Hatsuda, Eur. Phys. J. A 56, no.3, 93 (2020).
  * (17) A. Park, K. S. Jeong and S. H. Lee, Phys. Rev. D 104, no.9, 094024 (2021).
  * (18) V. Koch and V. Vovchenko, [arXiv:2211.14674 [nucl-th]].
  * (19) D. H. Rischke, M. I. Gorenstein, H. Stoecker and W. Greiner, Z. Phys. C 51, 485-490 (1991).
  * (20) V. Vovchenko, D. V. Anchishkin and M. I. Gorenstein, Phys. Rev. C 91, no. 6, 064314 (2015).
  * (21) P. Senger, Phys. Scripta 97, no.6, 064003 (2022).
  * (22) A. Lovato, T. Dore, R. D. Pisarski, B. Schenke, K. Chatziioannou, J. S. Read, P. Landry, P. Danielewicz, D. Lee and S. Pratt, et al. [arXiv:2211.02224 [nucl-th]].


# Aerial Base Station Positioning and Power Control for Securing
Communications: A Deep Q-Network Approach

Aly Sabri Abdalla1, Ali Behfarnia2, and Vuk Marojevic1
Email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>1Department of Electrical and Computer Engineering,
Mississippi State University, MS 39762, USA
2Department of Engineering, University of Tennessee at Martin, TN, USA

###### Abstract

The unmanned aerial vehicle (UAV) is one of the technological breakthroughs
that supports a variety of services, including communications. UAV will play a
critical role in enhancing the physical layer security of wireless networks.
This paper defines the problem of eavesdropping on the link between the ground
user and the UAV, which serves as an aerial base station (ABS). The
reinforcement learning algorithms Q-learning and deep Q-network (DQN) are
proposed for optimizing the position of the ABS and the transmission power to
enhance the data rate of the ground user. This increases the secrecy capacity
without the system knowing the location of the eavesdropper. Simulation
results show fast convergence and the highest secrecy capacity of the proposed
DQN compared to Q-learning and baseline approaches.

Keywords: Deep reinforcement learning, Q-learning, eavesdropping, UAV,
security.

##  I Introduction

Unmanned aerial vehicles (UAVs) support various applications in advanced
cellular networks. A UAV can be an aerial base station (ABS), aerial relay
(AR), or aerial user equipment (AUE) in the cellular network. As a network
support node, it can enhance the performance of the end users. Steps to
identify the challenges and solutions of emerging cellular networks serving
UAVs are being taken by the 3rd Generation Partnership Project (3GPP) [1].
Major challenges are radio frequency (RF) interference and security. Security
is increasingly important in modern wireless networks and needs to be ensured
for establishing communications links between UAVs and terrestrial nodes [2,
3, 4].

Different types of attacks have been studied with the aim of strengthening the
security of wireless communications links with trusted cellular network-
connected UAVs. Eavesdropping is a passive attack that can compromise the
confidentiality and privacy of control and user data. Early research [5, 6]
considering both hovering and moving UAVs study the performance of an AR for
safeguarding the ground communications links between terrestrial nodes against
eavesdropping attacks. Reference [7] investigates the same problem but for
multiple ground users and multiple eavesdroppers by optimizing the position of
the AR and the transmission power. In [8], the UAV is deployed as an ABS to
serve ground users under attack with the goal of optimizing the 3D position of
the UAV for maximizing the secrecy rate. The authors of [9] maximize the
secrecy rate of legitimate users while the ABS position is optimized without
eavesdropper location information.

In other lines of work, learning approaches have been used for solving the
above problem. Reference [10] develops model-free reinforcement learning (RL)
algorithms to maximize the system secrecy rate by transmitting artificial
noise with optimized beamforming from the AR. A multi-agent deep RL has been
proposed in [11] to maximize the secrecy capacity by jointly optimizing the
trajectory, transmit power, and jamming power for both relay and friendly
jamming UAVs protecting against eavesdroppers with known locations.
Nevertheless, limited study items utilized RL in secrecy rate analysis, such
as [10, 11]. These approaches assume knowledge of the eavesdroppers and their
locations. Also, learning methods are only recently being explored for
physical layer security. To the best of our knowledge, none of recent studies
have explored the potential of RL solutions without assuming the availability
of imperfect or perfect location information of the eavesdropper.

In this paper, we propose a RL algorithm for an ABS to assist with the uplink
transmission of ground users that are subject to eavesdropping. Our solution
is based on a deep Q-network (DQN) that aims to maximize the secrecy capacity
by optimizing the legitimate capacity of the ground user. This is achieved by
finding the position of the ABS and the transmission power that maximizes the
data and secrecy rates without the eavesdropper’s location information.

The rest of the paper is organized as follows. Section II provides the system
model and problem formulation. Section III introduces the DQN as our learning-
based physical layer security solution to eavesdropping. Section IV presents
the numerical analysis and Section V derives the conclusions.

##  II System Model and Problem Formulation

The scenario studied in this paper is illustrated in Fig. 1. The ground user
communicates with the ABS in the presence of a terrestrial eavesdropper. The
user and eavesdropper are independent of each other. The ABS is positioned to
provide a secure and reliable uplink (UL) for the user under attack. It
achieves this by leveraging its 3D mobility and strong line of sight (LoS)
channel that allows low power transmission. Without loss of generality, we
model and analyze the UL channel here. The same principles can be applied for
the downlink (DL). In the rest of this paper, we use the UAV and ABS
interchangeably.

Figure 1: The simulation scenario.

We define the location of the ABS, the eavesdropper, and the user in the 3D
Cartesian coordinate system as ($x_{u},y_{u},z_{u}$), ($x_{j},y_{j},z_{j}$),
and ($x,y,z$), respectively. For practicality, time slots are used to capture
the different radio frames and statistical channel conditions, as well as the
momentarily static position of the nodes. Therefore, the ABS coordinates in
time slot $t$ are expressed as ($x_{u}[t],y_{u}[t],z_{u}[t]$).

###  II-A Communications Channel

The received signal at the ABS when the ground user equipment (UE) transmits
the signal $s$ with power $P$ is as follows

${r}=\sqrt{P\hbar}s+n,$ (1)

where $\hbar$ is the air-to-ground (A2G) channel gain between the UE and the
ABS, and $n$ denotes the additive white Gaussian noise (AWGN) of zero mean and
$\sigma^{2}$ variance. Based on the measurements presented in [12], the LoS
model is a good approximation for the A2G channel in rural areas. The channel
gain has a path loss exponent of two and can be written as

$\displaystyle\hbar[t]$ $\displaystyle=\zeta_{0}d^{-2}_{u}[t]$ (2)
$\displaystyle=\frac{\zeta_{0}}{(x[t]-x_{u}[t])^{2}+(y[t]-y_{u}[t])^{2}+(z[t]-z_{u}[t])^{2}},$

where $\zeta_{0}$ is the channel gain at the reference point $d_{0}$ = 1 m.
Parameter $\zeta_{0}$ is the same for the A2G and the ground-to-ground (G2G)
channel because of the same the antenna gains and carrier frequency in both
cases. Parameter $d_{u}[t]$ denotes the distance between the ground user and
the ABS.

The capacity of the channel between the UE and the ABS in time slot $t$ is
calculated as follows:

${C_{u}[t]}=\log_{2}\left(1+\frac{{P}[t]\hbar[t]}{\sigma^{2}}\right)=\log_{2}\left(1+\frac{\zeta_{0}{P}[t]}{d^{2}_{u}[t]\sigma^{2}}\right),$
(3)

where $\zeta_{0}/\sigma^{2}$ is the signal-to-noise ratio (SNR) at the
reference point.

###  II-B Eavesdropping Channel

Often, UEs have fewer antennas and transmit in broadcast mode without
beamforming. The eavesdropper is intercepting the transmitted signal from the
ground UE and the received signal can be formulated as

${v}=\sqrt{P{\emptyset}}s+w,$ (4)

where $w$ denotes the zero-mean AWGN with a variance of $\sigma^{2}$ at the
eavesdropper. As a result of the G2G communications link between the UE and
the eavesdroppers, the exponent of the path loss for this link is assumed to
be four. Within the time slot $t$, the G2G channel gain represented by
$\emptyset$ between the UE and eavesdropper can be modeled as

$\displaystyle\emptyset$ $\displaystyle=\zeta_{0}d^{-4}_{j}[t]$ (5)
$\displaystyle=$
$\displaystyle\frac{\zeta_{0}}{({(x[t]-x_{j}[t])^{2}+(y[t]-y_{j}[t])^{2}+(z[t]-z_{j}[t])^{2})}^{2}},$

where $d_{j}[t]$ corresponds to the distance between the UE and the
eavesdropper in the $t$th time slot. The corresponding capacity of the wiretap
channel is

${C_{j}[t]}=\log_{2}\left(1+\frac{{P}[t]{\emptyset}[t]}{\sigma^{2}}\right)=\log_{2}\left(1+\frac{\zeta_{0}{P}[t]}{d^{4}_{j}[t]\sigma^{2}}\right).$
(6)

###  II-C Secrecy Capacity Metric

The security of the system is evaluated using the secrecy capacity metric
which is commonly employed in the literature for analyzing eavesdropping
security problems. The secrecy capacity is the rate at which the malicious
node cannot decode any data when the legitimate channel capacity is higher
than the wiretap channel capacity [13]. The secrecy capacity over $T$ time
slots is then obtained as

${C_{sec}}=\frac{1}{T}\sum\limits_{t=1}^{T}\bigg{(}C_{u}[t]-C_{j}[t]\bigg{)}^{+},$
(7)

where $[\omega]^{+}\triangleq max(\omega,0)$.

###  II-D Problem Formulation

The optimization problem is defined according to the following assumptions:
First, for the sake of simplicity and without loss of generality, we assume
that an ABS flies at a constant height that facilitates a LoS link between the
ABS and the ground UE. In general, the lower the UAV height can be, the higher
the resulting legitimate channel and secrecy capacity in the considered
context. Second, we assume that the location of the passive eavesdropper is
unknown. This scenario is of interest in practice where it is difficult to
detect the presence and location of eavesdroppers because of their passive
nature. Thus, we consider the capacity of the legitimate user in the above
problem formulation. The eavesdropper location is used only for the
calculation of the resulting secrecy rate to evaluate the performance of the
proposed solution.

The objective of this paper is thus to maximize the legitimate capacity
$C_{u}$ by selecting the position for the ABS and controlling transmit power
for the UE.

The optimization of the UE capacity will result in an improved secrecy
capacity of the system. However, the UL capacity of the ground user relies on
having a short distance to the ABS with a strong LoS link. Therefore, the ABS
position constraint is formulated as follows:

$(x_{u}[t],y_{u}[t])\leq(L_{x},L_{y}),\forall{t},$ (8)

where $(L_{x},L_{y})$ represent the maximum 2D coordinates of the UAV ground
location projection. We introduce the peak transmission power $P_{max}$ and
the UL transmission power constraint for the legitimate UE per time slot as

$0\leq P[t]\leq P_{max},\forall{t}.$ (9)

The optimization problem of the UE capacity is then given as

$\displaystyle(P1):\max\limits_{x_{u},y_{u},P}\frac{1}{T}\sum\limits_{t=1}^{T}\Bigg{[}$
$\displaystyle\log_{2}\left(1+\frac{\zeta_{0}{P}[t]}{d^{2}_{u}[t]\sigma^{2}}\right)\Bigg{]},$
(10) $\displaystyle s.t.\hskip 2.0pt(8),(9).$

where $x_{u}$ and $y_{u}$ are the UAV positioning parameters and $P$ is the
uplink transmission power controlled by the ABS.

##  III Proposed Solution

The optimization problem (10) is challenging because it needs a joint UAV
positioning and UE transmission power adjustment in the presence of an
eavesdropper. Since the objective function is non-convex with respect to
parameters $x_{u}$, $y_{u}$, and $P$, and the constraints, the problem becomes
NP-hard [7] [11] [14]. Alternatively, the ABS position and the UE transmission
power will be selected through a transition process based on the current
system state. Since the next state of the system is independent from the
previous state and action, the process can be modeled as a Markov decision
process (MDP). This allows applying a RL algorithm to a UAV agent without
requiring the knowledge of the system model. In this regard, instead of
solving the problem using conventional optimization algorithms, we apply a RL
method that can solve the problem in an efficient and accurate way, and
thereby improve the secrecy rate in the network.

In what follows, we first describe the MDP model by defining the settings that
include states, actions, and the reward for the UAV agent. Then, we introduce
the Q-learning method based on the defined MDP settings to solve the problem.
In order to avoid intractably high dimensionality for the high state-action
space, we propose the DQN method in which a deep neural network (DNN) is
employed to estimate the action value function for the UAV agent.

###  III-A MDP Settings

The MDP for the UAV agent is composed of the state space $\mathcal{S}$, the
action space $\mathcal{A}$, the reward space $\mathcal{R}$, and the transition
probability space $\mathcal{T}$, i.e., $\mathcal{(S,A,R,T)}$. At time slot
$t$, the agent observes the state $s_{t}\in\mathcal{S}$, and based on its
policy, it takes an action $a_{t}\in\mathcal{A}$. Depending on the
distribution of the transition probability $\mathcal{T}(s_{t+1}|s_{t},a_{t})$,
the agent will be transferred to the new state $s_{t+1}$. Since the transition
probability is highly dependent on a specific environment and is difficult to
obtain, we choose the Q-learning method as a model-free algorithm to directly
find the best policy for each action in each state. This means that we do not
need to know $\mathcal{T}$, but we need to carefully define states, actions,
and the reward of the agent as follows.

State: The set of states is defined as

$\mathcal{S}=\\{s_{1},s_{2},...,s_{t},..,s_{T}\\},$ (11)

where $T$ is the total number of time slots. Each state $s_{t}$ at a time slot
$t$ has three elements that are defined as

$s_{t}=\\{\Delta x,\Delta y,\Delta z\\},$ (12)

where $\Delta x,\Delta y$, and $\Delta z$ represent the distance difference
between the UAV and UE along the $x,y$, and $z$ axes, respectively. It is
worth noting that the value of each state affects the channel gain and, hence,
the SNR.

Action: The states are transited according to the defined actions. A set of
actions is defined as

$\mathcal{A}=\\{a_{1},a_{2},...,a_{t},..,a_{T}\\},$ (13)

where each action at time $t$ consists of two parts related to the UAV
movement and one part related to the transmit power adjustment. That is,

$a_{t}=\\{\delta_{x},\delta_{y},\delta_{p}\\},$ (14)

where $\delta_{x}$ and $\delta_{y}$ represent the movement in the $x$ and $y$
directions and $\delta_{p}$ denotes the change in power. The altitude of the
UAV is assumed to be constant.

As a sample configuration, the movement in $x$ and $y$, i.e., $\delta_{x}$ and
$\delta_{y}$, can be assumed to change by $+1$ unit or $-1$ unit, and the
power level, i.e., $\delta_{p}$, can be assumed to change by $p_{1}$, $0$, or
$-p_{1}$, where $p_{1}$ is an arbitrary number. Hence, here we consider $4$
possible directional movements and $3$ power level changes, resulting in $12$
possible actions for the ABS, which controls the UE transmission power, in any
state.

Reward: After taking an action $a_{t}$ in a state $s_{t}$ at time slot $t$,
the UAV agent will receive a reward $R_{t}(s_{t},a_{t})$. The UAV should get
more rewards for the actions that may lead it higher secrecy rates. In this
respect, we define the reward function of the system based on the
instantaneous SNR between the UAV and the UE as

${R_{t}(s_{t},a_{t})}=\frac{\zeta_{0}{P}[t]}{d^{2}_{u}[t]\sigma^{2}}.$ (15)

###  III-B Q-Learning Method

The UAV agent can apply the Q-learning method to find the best policy for the
state-action relationship. Q-learning is a classical table-based RL algorithm
in which the state-action pair has the value of $Q(s,a)$. The rows and columns
of the Q-table consist of the environmental states and possible actions of the
UAV agent, respectively. For example, for the aforementioned sample of states
and actions, the table has $3$ rows and $12$ columns. The Q-values in the
table are initially filled by random numbers. Then, the Bellman equation is
used to obtain the optimal state-action pairs in the table [15]:

$\displaystyle
Q^{*}(s,a)=E_{s^{\prime}}\bigg{[}R(s,a)+\gamma\times\max_{a\in\mathcal{A}}\
Q(s^{\prime},a^{\prime})\bigg{]},$ (16)

where the $s^{\prime}$ and $a^{\prime}$ symbolize the next state and action.
The parameter $\gamma\in(0,1)$ denotes the discount factor that affects the
importance of the future reward. The Bellman equation is used through an
iterative process to update the Q-values, where a learning rate parameter
$\alpha$ is applied to determine how quickly an agent leaves the previous
Q-value for the new Q-value in the table. That is,
$Q^{\text{new}}(s,a)=(1-\alpha)\ Q_{i-1}(s,a)+\alpha\ Q_{i}(s,a)$, where
subscript $i$ indicates the $i$-th iteration in the process. Finally, in each
step, the Q-value is updated while applying the $\epsilon-$greedy algorithm.
This algorithm is used to balance between the exploration and the exploitation
of the environment [15].

Although Q-learning provides a general framework for RL, it requires to store
the Q-values for each state-action pair in the table. The number of state-
action pairs in the table grows quickly with the number of states and actions.
As the Q-table becomes large, the process becomes more time-consuming and
impractical. Therefore, we consider the DQN method where a DNN is used to
estimate the $Q(s,a)$ values, as opposed to Q-learning, where the Q-table is
used to estimate the state-action values. This has advantages in terms of
handling a large number of states and actions and the associated processing
time.

Figure 2: Block diagram of the proposed DQN architecture.

###  III-C Deep Q-Network Method

The DQN, initially proposed by Google Deep Mind [16], integrates the RL and
deep learning methods. This technique uses the power of nonlinear functions,
specifically DNNs, in order to approximate the Q-values and handle highly
dimensional state-action problems.

Figure 2 shows the block diagram of the proposed DQN method. There are two
DNNs of the same structure: a training network and a target network. The
training network outputs the Q-values associated with the actions of the UAV
in each state. The target network supervises the training network by providing
the target Q-values obtained from the Bellman equation. The target values are
compared with the outputs of the training network to minimize the loss
function described below. Also, the target network prevents the learning model
from suffering from the noise in the environment. The inputs to the networks
are the states of the UAV agent in the environment, see (11)-(12). The outputs
of the network are the Q-values corresponding to the actions of the UAV agent,
i.e., $Q(s,a;\theta)$, where $\theta$ denotes the weights of the DNNs.

As the UAV takes an action, the system generates a record of experience. At
time step $t$, the experience contains the current state $s_{t}$, the action
$a_{t}$, the reward $r_{t}$, and the next state $s_{t+1}$, formed as a tuple
$e_{t}=(s_{t},a_{t},r_{t},s_{t+1})$. Each such experience is stored in a
replay memory with the capacity of $N$, such that
$\mathcal{M}=\\{e_{1},...,e_{t},...,e_{N}\\}$. The memory is a queue-like
buffer that stores the latest N experience vectors. We use a mini-batch sample
from the replay memory to feed the input of the training network as shown in
Fig. 2. The main reason for using the mini-batch samples from the reply memory
is to break possible correlations between sequential states of the
environment, and thereby facilitate generalization.

In order to minimize the error prediction of the DNNs, a loss function is used
that is defined as

$\displaystyle
L(\theta)=\mathbb{E}\Bigg{[}\bigg{(}\Big{[}r_{t}+\gamma\times\max_{a\in\mathcal{A}}\
Q(s_{t+1},a_{t+1};\theta^{\dagger})\Big{]}-$
$\displaystyle\Big{[}Q(s_{t},a_{t};\theta)\Big{]}\bigg{)}^{2}\Bigg{]},$ (17)

where the Q-value of the first term is obtained from the target network and
the Q-value of the second term from the training network. Parameters
$\theta^{\dagger}$ and $\theta$ denote the weights of the target network and
training network, respectively. The $\theta^{\dagger}$ coefficients are
updated every few time slots in order to ensure the stability of the target
values and, hence, facilitate stable learning.

The UAV applies a gradient descent algorithm,

$\displaystyle\nabla_{\theta}\,L(\theta)$ $\displaystyle=-\mathbb{E}\Bigg{[}2\
\nabla_{\theta}Q(s_{t},a_{t};\theta)\bigg{(}\,r_{t}+\,\gamma\ \times$
$\displaystyle\qquad\max_{a\in\mathcal{A}}\
Q(s_{t+1},a_{t+1};\theta^{\dagger})-Q(s_{t},a_{t};\theta)\bigg{)}\Bigg{]},$
(18)

to update $\theta$ an $\theta^{\dagger}$ as the weights of the DNNs with the
aim of minimizing the prediction error.

Finally, we apply the $\epsilon-$greedy algorithm to select an action while
balancing the exploration and the exploitation of the UAV in the environment
(Fig. 2). In this algorithm, the UAV explores the environment with the
probability of $\epsilon$ by choosing a random action. More precisely, the UAV
exploits the environment with the probability of $1-\epsilon$ by choosing the
actions that maximize the Q-value function, i.e.,
$a^{*}=\text{argmax}_{a\in\mathcal{A}}\ Q(s,a;\theta)$. A high value of
$\epsilon$ is initially set in the model for the UAV to spend more time for
the exploration. As the agent obtains more knowledge about the environment,
the $\epsilon$ value is gradually decreased to leverage the experience and
choose the best actions for the UAV, rather than continuing with the
exploration.

The details of the DQN-based algorithm used by the UAV agent for optimizing
the UE SNR and calculating the secrecy rates is presented in Algorithm 1. The
brief description of the pseudocode is as follows: The parameters of the
algorithm are initialized in lines $1$ to $4$. Line $5$ starts the first loop
for $K$ episodes. The environment is reset in line $6$ to initialize the
starting state. The second loop begins at line $7$, representing $T$ time
slots for the UAV to adjust its trajectory and power. Line $8$ denotes the
state in each time slot, and lines $9$ to $13$ apply the $\epsilon-$greedy
algorithm to balance the exploration versus exploitation. The UAV takes an
action in line $14$. Then it receives the reward and goes to the new state, as
denoted in line $15$. The replay memory collects the new experience in line
$16$. Using the mini-batch in line $17$, the training DNN is trained in line
$18$. The weights of the training DNN are updated in line $19$ using the
gradient descent algorithm on the loss function of (18). Line $20$ updates the
weights of the target DNN every $B$ time slots. Once the algorithm runs out of
time slots in each episode, it updates the value of $\epsilon$ (line $21$).
The rewards for each episode are stored for each episode according to line
$22$.

##  IV Numerical Analysis and Discussion

We numerically analyze the performance of the proposed DQN-based UAV
positioning and power control scheme in optimizing the UL SNR of the ground
user and its effect on the secrecy capacity in the presence of an
eavesdropping attack.

The simulation scenario consists of a single antenna ground UE, an ABS mounted
on the UAV, and a single antenna malicious node that is performing a passive
eavesdropping attack on the UL transmission (Fig. 1). The UE and the
eavesdropper are randomly distributed in a 2D area that has a $L_{x}$ length
and a $L_{y}$ width. The ABS is launched at a random location with a fixed
altitude and is equipped with an omnidirectional antenna to enable
communications with UEs.

Table I provides the simulation parameters and the hyperparameters for the
proposed DQN solution. The training and target DNN networks consist of $4$
layers where each DNN contains two fully connected hidden layers, one with
$24$ and the other with $32$ neurons. Each DNN has $3$ neurons at the input
layer and $12$ neurons at the output layer, corresponding to the number of
states and possible actions defined in (12) and (14), respectively. The
simulator is implemented in Python, using PyTorch to train the DQN.

For the performance evaluation of the proposed DQN algorithm, we compare the
resulting secrecy capacity of the DQN against the values resulting from
employing Q-learning, a greedy policy, and a random state selection scheme.
The random scheme selects the next position and power action at each time slot
randomly, whereas the greedy policy assigns the next action based on the
highest Q-value.

Figure 3: The reward (a), distance (b), and power (c) convergence of the DQN,
Q-learning and other base line techniques.

Figure 3(a) shows our accumulative reward function over the total number of
episodes that is defined as the legitimate UL SNR of the ground UE for the
proposed DQN and for the other techniques. The illustration of rewards informs
that the proposed solution has the fastest convergence rate to the highest
SNR. The SNR performance of the Q-learning is similar, but takes longer to
converge because of the nature of the problem that has a large number of
states and actions.

Since the optimization problem in (10) relies on the location of the ABS and
the transmission power, it is critical to study the convergence of these two
parameters. Figures 3(b) and 3(c) illustrate the convergence of the UAV
position and the transmission power over the number of episodes. Comparing the
converged position of the ABS and the transmission power for the DQN and
Q-learning, we observe that both techniques reach the same ABS position;
however, there is a slight increase of the converged transmission power of the
DQN over the converged Q-learning transmission power. This difference is the
main explanation behind the difference in the optimized legitimate SNR levels
presented in Fig. 3(a). The low SNR of the random approach results from low
the transmission power value of Fig. 3(c). Note that the ABS position of the
random scheme converges to the same value as the greedy algorithm.

Figure 4 shows the secrecy capacity of the DQN compared to the other
techniques over the number of episodes. The secrecy capacity is calculated
using the optimized ABS position and transmission power level after finalizing
the learning. The proposed DQN algorithm improves the secrecy capacity by as
much as 40%, 10%, and 5% when compared to the random, greedy, and the
Q-learning solutions, respectively. The DQN reaches a relatively stable
secrecy capacity value already after $2\times 10^{4}$ episodes. On the other
hand, the convergence of Q-learning occurs after $5\times 10^{4}$ episodes.
This proves the superiority of the DQN regarding the speed of convergence.

Figure 4: Secrecy capacity achieved by the DQN, Q-learning and baseline
techniques.

##  V Conclusions

This paper has presented a novel positioning and UL transmission power control
approach for ABSs that serve a ground user. We consider the legitimate user
capacity as the metric to optimize in eavesdropping scenarios without
knowledge of the eavesdropper location. This contributes to improving the
secrecy capacity of the ground user under attack. We have provided detailed
information about the designed DQN and the Q-learning algorithms. The obtained
results show that the highest capacity and secrecy capacity are achieved with
the proposed DQN when compared with Q-learning and two baseline techniques.

In future research, we will extend the simulation environment to cover
multiple legitimate and malicious nodes, analyze the performance of using a
multi-antenna systems, and consider both the uplink and the downlink.
Moreover, the presented technique can be implemented into a testbed, such as
AERPAW [17], which provides robust drones with modular software radio hardware
and software, collocated computers, and an experimental license for RF
radiation, enabling A2G wireless experiments. An AERPAW experiment can deploy
ground users and UAVs, where the UAV can implement an ABS that uses the
proposed method to position itself for serving a legitimate user and control
the transmission power in such a way to maximize the user rate in the presence
of an eavesdropper.

## Acknowledgement

The work of A. S. Abdalla and V. Marojevic was supported in part by the NSF
award CNS-1939334.

## References

  * [1] A. S. Abdalla and V. Marojevic, “Communications standards for unmanned aircraft systems: The 3GPP perspective and research drivers,” _IEEE Commun. Standards Mag._ , vol. 5, no. 1, pp. 70–77, 2021.
  * [2] A. S. Abdalla, K. Powell, V. Marojevic, and G. Geraci, “UAV-assisted attack prevention, detection, and recovery of 5G networks,” _IEEE Wireless Commun._ , vol. 27, no. 4, pp. 40–47, 2020.
  * [3] R. Amer, W. Saad, and N. Marchetti, “Mobility in the sky: Performance and mobility analysis for cellular-connected UAVs,” _IEEE Transactions on Communications_ , vol. 68, no. 5, pp. 3229–3246, 2020.
  * [4] B. Shang, V. Marojevic, Y. Yi, A. S. Abdalla, and L. Liu, “Spectrum sharing for UAV communications: Spatial spectrum sensing and open issues,” _IEEE Veh. Technol. Mag._ , vol. 15, no. 2, pp. 104–112, 2020.
  * [5] A. S. Abdalla _et al._ , “Performance evaluation of aerial relaying systems for improving secrecy in cellular networks,” in _2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall)_ , 2020, pp. 1–5.
  * [6] A. S. Abdalla, B. Shang, V. Marojevic, and L. Liu, “Securing mobile IoT with unmanned aerial systems,” in _2020 IEEE 6th World Forum on Internet of Things (WF-IoT)_ , 2020, pp. 1–6.
  * [7] A. S. Abdalla and V. Marojevic, “Securing mobile multiuser transmissions with UAVs in the presence of multiple eavesdroppers,” _IEEE Trans. Veh. Technol._ , vol. 70, no. 10, pp. 11 011–11 016, 2021.
  * [8] D. Wang, B. Bai, G. Zhang, and Z. Han, “Optimal placement of low-altitude aerial base station for securing communications,” _IEEE Wireless Communications Letters_ , vol. 8, no. 3, pp. 869–872, 2019.
  * [9] H. Kang _et al._ , “Secrecy-aware altitude optimization for quasi-static UAV base station without eavesdropper location information,” _IEEE Communications Letters_ , vol. 23, no. 5, pp. 851–854, 2019.
  * [10] M. T. Mamaghani and Y. Hong, “Intelligent trajectory design for secure full- duplex MIMO-UAV relaying against active eavesdroppers: A model-free reinforcement learning approach,” _IEEE Access_ , vol. 9, pp. 4447–4465, 2021.
  * [11] Y. Zhang _et al._ , “UAV-Enabled Secure Communications by Multi-Agent Deep Reinforcement Learning,” _IEEE Trans. Veh. Technol._ , vol. 69, no. 10, pp. 11 599–11 611, 2020.
  * [12] X. Lin _et al._ , “The sky is not the limit: LTE for unmanned aerial vehicles,” _IEEE Commun. Mag._ , vol. 56, no. 4, pp. 204–210, 2018.
  * [13] N. Yang, L. Wang, G. Geraci, M. Elkashlan, J. Yuan, and M. D. Renzo, “Safeguarding 5G wireless communication networks using physical layer security,” _IEEE Commun. Mag._ , vol. 53, no. 4, pp. 20–27, 2015.
  * [14] J. Li _et al._ , “Deep reinforcement learning based computation offloading and resource allocation for mec,” in _2018 IEEE Wireless Communications and Networking Conference (WCNC)_ , 2018, pp. 1–6.
  * [15] K. Arulkumaran _et al._ , “Deep reinforcement learning: A brief survey,” _IEEE Signal Processing Magazine_ , vol. 34, no. 6, pp. 26–38, 2017.
  * [16] V. Mnih _et al._ , “Human-level control through deep reinforcement learning,” _Nature_ , vol. 518, no. 7540, pp. 529–533, 2015.
  * [17] V. Marojevic, I. Guvenc, R. Dutta, and M. Sichitiu, “Aerial experimentation and research platform for mobile communications and computing,” in _2019 IEEE Globecom Workshops (GC Wkshps)_ , 2019, pp. 1–6.


* [38] H. H. Pennes, Analysis of tissue and arterial blood temperatures in the resting human forearm, Journal of applied physiology, 1 (1948), pp. 93–122.
  * [39] R. Racke and B. Said-Houari, Global well-posedness of the Cauchy problem for the Jordan–Moore–Gibson–Thompson equation, Communications in Contemporary Mathematics, 23 (2021), p. 2050069.
  * [40] B. Said-Houari, Global existence for the Jordan–Moore–Gibson–Thompson equation in Besov spaces, Journal of Evolution Equations, 22 (2022), p. 32.
  * [41] B. Said-Houari, Global well-posedness of the cauchy problem for the Jordan–Moore–Gibson–Thompson equation with arbitrarily large higher-order Sobolev norms, Discrete and Continuous Dynamical Systems, 42 (2022), pp. 4615–4635.
  * [42] S. Salsa, Partial differential equations in action: From modelling to theory, Springer, 2008.
  * [43] Z. C. W. Liu and D. Chen, New general decay results for a Moore–Gibson–Thompson equation with memory, Applicable Analysis., https://doi.org/10.1080/00036811.2019.1577390 (2019), pp. 1–19.
  * [44] P. J. Westervelt, Parametric acoustic array, The Journal of the Acoustical Society of America, 35 (1963), pp. 535–537.
  * [45] M. Wilke, ${L}\\_p$-${L}\\_q$-theory for a quasilinear non-isothermal Westervelt equation, arXiv preprint arXiv:2204.09630, (2022).
  * [46] F. Xu, K. Seffen, and T. Lu, Non-fourier analysis of skin biothermomechanics, International Journal of Heat and Mass Transfer, 51 (2008), pp. 2237–2259.