# Multiplicity distributions in the low x regime in a simple model

G. R. Germano<EMAIL_ADDRESS>Instituto de Física, Universidade de
São Paulo, Rua do Matão, 1371, CEP 05508-090 , Cidade Universitária, São
Paulo, SP, Brazil  F.S. Navarra<EMAIL_ADDRESS>Instituto de Física,
Universidade de São Paulo, Rua do Matão, 1371, CEP 05508-090 , Cidade
Universitária, São Paulo, SP, Brazil

###### Abstract

In this work we introduce small changes in the model proposed by E. Levin and
D. Kharzeev for multiplicity distributions of particles produced in proton-
proton collisions. We compare the predictions of the model with the available
experimental data from the LHC. We also consider the most recent version of
the model proposed by E. Gotsman and E. Levin. These two versions of the model
give a good description of the measured multiplicity distributions in the
central pseudo-rapidity region ($|\eta|<0.5$). The agreement with data is not
so good when we consider the particles measured over a wider pseudo-rapidity
interval ($|\eta|<2.4$). Interestingly, the agreement becomes better when one
selects events with higher transverse momentum ($p_{T}>500$ MeV).

##  I Introduction

The most recent review paper on multiplicity distributions was written ten
years ago fiete . Its last section presented a number of predictions for the
future collisions that would be performed at the LHC.

After ten years of operation, the LHC delivered an impressive amount of data
on multiplicity distributions atlas11 ; cms11 ; atlas16a ; atlas16b ; atlas16c
; alice17 ; cms18 . The collision energies range from 0.9 to 13 TeV. These
data allow us to study the energy evolution of multiplicity distributions and
to look for qualitative changes in the reaction dynamics. One of the expected
changes was a transition from the soft (non-perturbative) to the semi-hard
(perturbative) QCD regime ugo98 . Previous studies had concluded that this
transition was already happening fiete  in the pre-LHC era. It was conjectured
that semi-hard events would have larger multiplicities and this would induce a
“shoulder” in the large $n$ region of the multiplicity distribution $P(n)$. Up
to $\sqrt{s}=1$ TeV, $P(n)$ was well described by a negative binomial
distribution (NBD). At the Tevatron at $\sqrt{s}=1.8$ TeV the appearance of
the “shoulder” was confirmed and this led to the two-component model proposed
in ugo98  in which the measured data were described by a combination of two
NBDs, one representing the soft and one the semi-hard component. This
“shoulder” was also responsible for the violation of the Koba-Nielsen-Olsen
(KNO) scaling fiete . More recently, the data taken at $\sqrt{s}=$ 7, 8 and 13
TeV show that the best fit is obtained with the inclusion of a third negative
binomial distribution zbo ; bya .

At the LHC another transition was expected to be seen: the transition to the
low x regime and to the BFKL dynamics with the possible manifestation of gluon
saturation effects kole ; gel . Compared to the previous accelerators, the LHC
proton-proton collisions occur at a center of mass energy which is one order
of magnitude higher. Increasing the energy we have access to the low momentum
component of the proton wave function, i.e., we “see” more partons with
smaller momentum fraction $x$ of the proton. At the same time, we may have
collisions between partons which are more off-shell, i.e., which have a higher
virtuality $Q^{2}$. The changes in the parton distribution functions
$f(x,Q^{2})$ when they enter in the low $x$ and high $Q^{2}$ regimes are
described by the BFKL and DGLAP QCD evolution equations respectively. The
solutions of these equations show that the number of partons increases with
$1/x$ and with $Q^{2}$. We can picture this process thinking that when we
boost a proton its partons go through a branching process, with a copious
production of new partons. This has direct implications for the final particle
multiplicity. At lower collision energies particle production is dominated by
string fragmentation, which describes hadron formation and which is a non-
perturbative QCD process. At higher energies, and especially at the LHC,
particle production is dominated by parton branching, which is studied with
perturbative QCD, with string fragmentation playing a secondary role. The
measured multiplicity distributions at the LHC are thus an excellent testing
ground for the predictions of the evolution equations.

The best theory of strong interactions is QCD. The part of QCD which describes
dense partonic systems formed in high energy collisions is the Color Glass
Condensate (CGC) theory kole ; gel . In practical applications of the CGC it
is necessary to make approximations and/or additional assumptions to obtain
results which can be compared to data. In several works it has been shown that
with the CGC based models it is possible to obtain a reasonable description of
the global features of multiparticle production in high energy collisions. In
the case of multiplicity distributions there is a lot of work to be done. In
schenke  the authors used the IP-Glasma model and calculated the multiplicity
distribution of particles produced with $|\eta|<0.5$ in pp collisions at 7
TeV. They found discrepancies between the model predictions and data at large
multiplicities. In nara ; raju11  the authors used CGC models to study
particle production and $P(n)$ was assumed to be a negative binomial
distribution with the parameters $k$ and $\bar{n}$ given by the CGC models.
The data analysis was restricted to the lower LHC energies and to central
pseudo-rapidity intervals. In the present analysis we will include the most
recent data from higher energies and consider other rapidity and $p_{T}$
windows. Moreover we will test a $P(n)$ which is related to BFKL dynamics.

In glitter  it was shown that the number of particles produced from the glasma
follows a negative binomial distribution, which depends on the parameter $k$,
which turns out to be proportional to the saturation scale $k\propto
Q_{s}^{2}$. We know that the saturation scale grows with the collision energy
and hence so does $k$. As $k\to\infty$ the negative binomial distribution goes
to a Poisson distribution, which is a very narrow distribution. Therefore the
CGC-Glasma prediction is that at increasing energies the multiplicity
distribution should “shrink”. So far, the existing data have shown the
opposite trend. The observed broadening of the multiplicity distributions can
be quantified through the moments $C_{n}$ which are increasing functions of
the energy, implying broader multiplicity distributions. The non-observation
of the shrinkage of $P(n)$ is an indication that some ingredient is still
missing in the theory and it must be completed.

In theoretical studies of MD’s there are top-down and bottom-up approaches. In
the first group we find works glitter ; mumu ; giaca ; dumi ; shab , firmly
rooted in QCD, which discuss in detail the parton branching process using
different formalisms, different techniques and different approximations.
Special attention is given to gluon saturation, which takes place at very high
gluon densities, when gluon-gluon fusion becomes important and reduces the
growth of gluon number, having a direct impact on the final hadron
multiplicity. In a non-perturbative approach, it was suggested in khoze  that
the presence of instantons could affect the observed multiplicity
distributions.

In the second group we find works zbo ; bya ; beg ; agga ; sha ; gre18 ; gre19
; gre20  that try to extract the maximum amount of information from data,
using sophisticated statistical tools, paying attention to the subtleties of
the measured distributions and building up models, with assumptions that may
motivate an a posteriori theoretical study.

In the top-down approach most of the works did not reach the point where they
address the data and try to reproduce them. In the bottom-up approach most the
works did not reach the point where they establish a clear connection to QCD.

In a third group, we find works in which the authors try to build the bridge
between QCD and data. In this cathegory we include the studies performed with
more phenomenological models nara ; raju11 ; adrian ; duna  and with event
generators cms11 ; lund , which are built to reproduce all the observable
quantities in a high energy collision.

The accumulated experimental data were studied with bottom-up models and with
phenomenological models. The global features of the MD’s are captured by the
models but they fail to reproduce the data in one or another kinematical
region. In the compilation of the theoretical results presented in cms11 , the
event generators either underestimate or overestimate the large $n$ tail of
the distribution $P(n)$. Inspite of the recent improvements, such as, for
example, in lund , the discrepancies persist. There is not yet a complete and
satisfactory theoretical description of multiplicity distributions.

In this work we try to follow the approach of the third group: we choose a
simple model of the QCD branching process and perform a comprehensive
comparison with the existing data. This model was first proposed some time ago
lubi , it was used recently kale  to study entanglement entropy in particle
production and it was improved very recently in gole ; gole2 , where a
preliminary discussion of the LHC data was presented. In the next section we
briefly describe the model and in the last section we present our results.

##  II The model

###  II.1 The Kharzeev - Levin model

In Ref. lubi ; kale , the authors have developed a model for multiplicity
distributions based on the BFKL equation, which we will call KL model. They
propose the following evolution equation for the parton multiplicity
distribution $P_{n}$:

$\frac{dP_{n}(Y)}{dY}\,=\,-\,\Delta\,n\,P_{n}\,+\,(n-1)\,\Delta\,P_{n-1}(Y)$
(1)

which has the simple solution:

$P_{n}(Y)\,=\,P_{KL}(n)\,=\,e^{-\Delta Y}\left(1-e^{-\Delta Y}\right)^{n-1}$
(2)

where $Y=ln(1/x)$ and $\Delta$ is the BFKL Pomeron intercept,
$\Delta\,=\,4\,\,ln\,\,2\,\,\bar{\alpha}_{s}$ with
$\bar{\alpha}_{s}=\alpha_{s}N_{c}/\pi$. From the above expression we obtain
the mean multiplicity:

$\langle n\rangle=\sum_{n}n\,P(n)=e^{\Delta
Y}=\left(\frac{1}{x}\right)^{\Delta}$ (3)

The variable $x$ is defined here as in lelu ; babi :

$x=\frac{q_{0}^{2}}{s}$ (4)

where $q_{0}$ is a constant. Inserting (4) into (3) we obtain:

$\langle n\rangle=\left(\frac{s}{q_{0}^{2}}\right)^{\Delta}$ (5)

The energy scale $q_{0}$ can be a mass or the average transverse momentum and
hence it might be different for different data sets, but it should not depend
on the collision energy $\sqrt{s}$.

The above expression is a prediction of the model. Interestingly, it can be
derived from the CGC formalism. In fact, the first estimate of the mean
multiplicity of produced particles in the CGC framework was done in kn  and in
kln . According to kn  (see also adrian ) in the saturation regime the number
of produced partons is given by

$\langle n\rangle\,=C\,\frac{Q_{s}^{2}\,R^{2}}{\alpha_{s}(Q^{2}_{s})}$ (6)

where $C$ is a constant, $R$ is the transverse size of the projectile (proton
in our case) and $\alpha_{s}(Q^{2}_{s})$ is the strong coupling evaluated at
$Q^{2}_{s}$, which is the saturation scale given by

$Q_{s}^{2}=Q^{2}_{0}\left(\frac{x_{0}}{x}\right)^{\lambda}$ (7)

where $\lambda=0.2-0.3$, $Q_{0}^{2}$ and $x_{0}$ are constants. From the above
equations we conclude that:

$\langle
n\rangle=\,C\,Q_{0}^{2}\,(\frac{x_{0}}{q_{0}^{2}})^{\lambda}\,\frac{R^{2}(s)}{\alpha_{s}(s)}\,s^{\lambda}$
(8)

The radius of the proton, $R$, is related to confinement and can not be
estimated from CGC physics. It has some weak energy dependence which comes
from nonperturbative effects. In dosch  the proton radius was parametrized as

$R^{2}(s)=R^{2}_{0}\left(\frac{s}{s_{p}}\right)^{0.05}$ (9)

Assuming that $\alpha_{s}(s)\simeq const$ and substituting (9) into (8) we
find

$\langle n\rangle\,=\,\left(\frac{s}{s_{0}}\right)^{\lambda_{eff}}$ (10)

which is equivalent to (5) and where all the constants were packed in the
parameter $s_{0}$ and $\lambda_{eff}=\lambda+0.05$.

At lower energies, $\sqrt{s}=10-50$ GeV, particle production is dominated by
soft, non-perturbative dynamics, and the calculated quantities have typically
a weak (log) energy dependence. In the $\sqrt{s}\simeq 1$ TeV region it was
argued ugo98  that the mean multiplicity has a soft $\langle n_{s}\rangle$ and
a semi-hard $\langle n_{h}\rangle$ component which depend on the collision
energy $\sqrt{s}$ as $\langle n_{s}\rangle\propto lns$ and $\langle
n_{h}\rangle\propto ln^{2}s$. At the LHC energies it was shown in cms11  and
kln  that power law expressions such as (5) and (10) are able to reproduce the
data. However these data are also compatible with other parametrization forms,
such as $\langle n_{ch}\rangle\propto ln^{3}s$, as was shown in roy .

In the next section we will compare (2) with the available experimental data
from LHC. Eq. (2) depends on two parameters, $\Delta$ and $q_{0}$, which will
be adjusted to fit the data. Before doing this we will describe the
improvement of the KL model proposed in gole  and gole2 .

###  II.2 The Gotsman - Levin model

The formalism developed in kale  was further explored in gole . In gole  it
was pointed out that (2) yields multiplicity distributions which are too
broad. This behavior was attributed to the fact that (2) describes dilute
systems. According to the authors, a more accurate description of central
rapidity data, requires the study of the collision between dense systems. In
gole2 , Gotsman and Levin derived the following formula for the multiplicity
distribution:

$P_{n}(Y)=P_{GL}(n)\,=\,\frac{2}{\sqrt{\pi z}}\frac{e^{-z}}{N};$ (11)

where $z=\frac{n}{N}$ and

$N=e^{\Delta Y}=e^{\Delta
ln(\frac{1}{x})}=e^{ln(\frac{s}{q_{1}^{2}})^{\Delta}}=\left(\frac{s}{q_{1}^{2}}\right)^{\Delta}$
(12)

It is useful to write the simple relation between $P_{KL}(n)$ and $P_{GL}(n)$
gole2 :

$R=\frac{P_{GL}(n)}{P_{KL}(n)}=\sqrt{\frac{1}{\pi z}}=\sqrt{\frac{N}{\pi n}}$
(13)

This more singular behavior of $P_{GL}(n)$ at small $n$ will be visible in all
the figures shown below.

##  III Results and discussion

In this section we compare the results of the model with the experimental data
from LHC. Three data sets were used, each one with several center of mass
energies ($\sqrt{s}$) and each set corresponding to a different cut in the
minimum transverse momentum ($p_{T}$) of the measured particles and a
different pseudo-rapidity ($\eta$) window. They are:

  * •

Set I: $p_{T}>100$ MeV, $|\eta|<0.5$ and energies $\sqrt{s}=$ 900, 2360 and
7000 GeV. These data are from cms11 .

  * •

Set II: $p_{T}>100$ MeV, $|\eta|<2.4$, and energies $\sqrt{s}=$ 900, 7000 and
8000 GeV alice17  and 8000 atlas16a  and 13000 GeV atlas16b .

  * •

Set III: $p_{T}>500$ MeV, $|\eta|<2.4$ and energies $\sqrt{s}=$ 900, 2360 and
7000 GeV atlas11 , 8000 GeV ($|\eta|<2.5$) atlas16a  and 13000 GeV cms18  and
($|\eta|<2.5$) atlas16c .

In Figs. 1, 2 and 3 we show the the fits of (2) (solid lines) and (11) (dashed
lines) to the data on multiplicity distributions from the LHC. For each figure
we adjusted three numbers $\Delta$, $q_{0}$ and $q_{1}$. They are listed in
Table 1. As it can be seen both models overshoot the data at large $n$ in all
data sets. The model GL, as expected, overshoots the data at small $n$. In
Figs. 1 and 3 the theoretical curves have essentially the correct shape. In
contrast, in Fig. 2, in some cases, the data show a curvature which is absent
in the theoretical curves. A careful comparison between set I and set II was
done in cms11 . The conclusion then was that set I is compatible with KNO
scaling while it is violated in set II. There seemed to be something different
happening in set II. Now set II has been enlarged with new data points from
higher energies and the agreement with models KL and GL is better. From the
perspective of low x physics, set III is the most interesting data sample
because the cut $p_{T}>500$ MeV removes a (significant ?) part of the non-
perturbative events. These events are not related to parton branching or to
the evolution equations. As can be seen in Table 1, the value of $\Delta$ is
larger, indicating a stronger dependence on the energy $\sqrt{s}$, typical of
perturbative physics. Finally, the overall agreement between theory and data
is better.

In gole  and gole2  $P_{GL}(n)$ and $P_{KL}(n)$ were compared with some
experimental data but here we perform a systematic comparison to all relevant
available data. In those works the final goal was to compute the entanglement
entropy and the multiplicity distributions were a crucial ingredient. To this
end it is important to have an accurate description of data.

The model presented here can be improved in many ways. The multiplicity
distributions are affected by fluctuations in several quantitites. The most
obvious one is the impact parameter $b$ raju11 ; beg . Another one is the
fraction (called inelasticity $K$) of the c.m.s energy which is converted into
secondary particles. It is typically $0.5$, but it changes from event to event
and it depends on the energy according to a distribution $\chi[K(b,s)]$ beg ;
nuww . The inclusion of these fluctuations will certainly modify the shape of
$P(n)$. Another aspect to be considered is the number of sources which emit
particles. When we use $P_{GL}(n)$ or $P_{KL}(n)$ above it is understood that
there is only one emitting system. However, in high energy proton-proton
collisions there is a separation between particle production at low rapidities
(mostly from the gluon cloud of the projectiles) and at larger rapidities
(mostly from the valence quarks). So it is conceivable that the final,
observed, $P(n)$, is a sum of partial multiplicity distributions.

To summarize: multiplicity distributions deserve more attention from
theorists. In Refs. kale ; gole ; gole2  simple distributions were derived
from the QCD evolution equations. Here we have performed a comprehensive
comparison between these multiplicity distributions and LHC data. The results
are encouraging but there is still a lot to be done.

Data Set | $\Delta$ |  $q_{0}$ (GeV) |  $q_{1}$ (GeV)
---|---|---|---
I | 0.13 | 4.46 | 4.70
II | 0.13 | 0.02 | 0.01
III | 0.16 | 2.85 | 1.93
Table 1: Fit parameters. |
---|---
Figure 1: Multiplicity distributions compared with data set I. Data are from CMS (2011) cms11 . Left panel: $P(n)$ for different energies. Solid lines: $P_{KL}(n)$. Dashed lines: $P_{GL}(n)$. Sets of points were multiplied by powers of ten for clarity. Right panel: Ratio Theory ($P_{KL}(n)$) /Data. |
---|---
Figure 2: Multiplicity distributions compared with data set II. Data for $\sqrt{s}=$ 900, 7000 and 8000 GeV from ALICE (2017) alice17 . Data from ATLAS (2016) for 8000 atlas16a  and 13000 GeV atlas16b . Solid lines: $P_{KL}(n)$. Dashed lines: $P_{GL}(n)$. Left panel: $P(n)$ for different energies. Sets of points were multiplied by powers of ten for clarity. Right panel: Ratio Theory ($P_{KL}(n)$)/Data. |
---|---
Figure 3: Multiplicity distributions compared with data set III. Data for
energies $\sqrt{s}=$ 900, 2360 and 7000 GeV from ATLAS (2011) atlas11 , for
8000 GeV from ATLAS (2016) atlas16a . Data for 13000 GeV from CMS (2018) cms18
and from ATLAS (2016) atlas16c . Solid lines: $P_{KL}(n)$. Dashed lines:
$P_{GL}(n)$. Left panel: $P(n)$ for different energies. Sets of points were
multiplied by powers of ten for clarity. Right panel: Ratio Theory
($P_{KL}(n)$)/Data.

###### Acknowledgements.

The authors are grateful to G. Wilk for useful comments. This work was
partially financed by the Brazilian funding agencies CNPq and CAPES.

## References

  * (1) J. F. Grosse-Oetringhaus and K. Reygers, J. Phys. G 37, 083001 (2010).
  * (2) G. Aad et al. [ATLAS], New J. Phys. 13, 053033 (2011).
  * (3) V. Khachatryan et al. [CMS], JHEP 1101, 079 (2011).
  * (4) G. Aad et al. [ATLAS Collaboration], Eur. Phys. J. C 76, 403 (2016).
  * (5) G. Aad et al. [ATLAS Collaboration], Eur. Phys. J. C 76, 502 (2016).
  * (6) G. Aad et al. [ATLAS Collaboration], Phys. Lett. B 758, 67 (2016).
  * (7) S. Acharya et al. [ALICE Collaboration], Eur. Phys. J. C 77, 852 (2017).
  * (8) A. M. Sirunyan et al. [CMS Collaboration], Eur. Phys. J. C 78, 697 (2018).
  * (9) A. Giovannini and R. Ugoccioni, Phys. Rev. D 59, 094020 (1999); Erratum: [Phys. Rev. D 69, 059903 (2004)].
  * (10) I. Zborovsky, Eur. Phys. J. C 78, 816 (2018).
  * (11) M. Biyajima and T. Mizoguchi, Int. J. Mod. Phys. A 34, 1950203 (2019).
  * (12) Yuri V. Kovchegov and Eugene Levin, “ Quantum Chromodynamics at High Energies”, Cambridge Monographs on Particle Physics, Nuclear Physics and Cosmology, Cambridge University Press, 2012 .
  * (13) F. Gelis, E. Iancu, J. Jalilian-Marian and R. Venugopalan, Ann. Rev. Nucl. Part. Sci. 60, 463 (2010).
  * (14) B. Schenke, P. Tribedy and R. Venugopalan, Phys. Rev. C 89, 024901 (2014).
  * (15) A. Dumitru, D. E. Kharzeev, E. M. Levin and Y. Nara, Phys. Rev. C 85, 044920 (2012).
  * (16) P. Tribedy and R. Venugopalan, Nucl. Phys. A 850, 136 (2011). Erratum: [Nucl. Phys. A 859, 185 (2011)].
  * (17) F. Gelis, T. Lappi and L. McLerran, Nucl. Phys. A 828, 149 (2009).
  * (18) T. Liou, A. H. Mueller and S. Munier, Phys. Rev. D 95, 014001 (2017).
  * (19) L. Dominé, G. Giacalone, C. Lorcé, S. Munier and S. Pekar, Phys. Rev. D 98, 114032 (2018).
  * (20) A. Dumitru and V. Skokov, Phys. Rev. D 96, 056029 (2017).
  * (21) G. H. Arakelyan, Y. M. Shabelski and A. G. Shuvaev, Eur. Phys. J. C 80, 592 (2020).
  * (22) V. V. Khoze, F. Krauss and M. Schott, JHEP 2004, 201 (2020); V. V. Khoze, D. L. Milne and M. Spannowsky, arXiv:2010.02287 [hep-ph].
  * (23) P. C. Beggio and F. R. Coriolano, Eur. Phys. J. C 80, 437 (2020).
  * (24) R. Aggarwal and M. Kaur, Adv. High Energy Phys. 2020, 5464682 (2020).
  * (25) S. Sharma and M. Kaur, Phys. Rev. D 99, 096016 (2019).
  * (26) M. Rybczynski, G. Wilk and Z. Włodarczyk, Phys. Rev. D 99, 094045 (2019).
  * (27) M. Rybczyński, G. Wilk and Z. Włodarczyk, Ukr. J. Phys. 64, 738 (2019).
  * (28) H. W. Ang, A. H. Chan, M. Ghaffar, M. Rybczyński, G. Wilk and Z. Włodarczyk, Eur. Phys. J. A 56, 117 (2020).
  * (29) A. Dumitru and E. Petreska, arXiv:1209.4105.
  * (30) A. Dumitru and Y. Nara, Phys. Rev. C 85, 034907 (2012).
  * (31) C. Bierlich, S. Chakraborty, G. Gustafson and L. Lönnblad, arXiv:2010.07595 [hep-ph].
  * (32) E. Levin and M. Lublinsky, Nucl. Phys. A 730, 191 (2004).
  * (33) D. E. Kharzeev and E. M. Levin, Phys. Rev. D 95, 114008 (2017).
  * (34) E. Gotsman and E. Levin, Phys. Rev. D 102, 074008 (2020).
  * (35) E. Gotsman and E. Levin, arXiv:2008.10911 [hep-ph].
  * (36) J. Bartels, E. Gotsman, E. Levin, M. Lublinsky and U. Maor, Phys. Lett. B 556, 114 (2003).
  * (37) F. Carvalho, F. O. Duraes, V. P. Goncalves and F. S. Navarra, Mod. Phys. Lett. A 23, 2847 (2008).
  * (38) D. Kharzeev and M. Nardi, Phys. Lett. B 507, 121 (2001).
  * (39) D. Kharzeev, E. Levin and M. Nardi, Nucl. Phys. A 747, 609 (2005).
  * (40) H. G. Dosch, E. Ferreira and A. Kramer, Phys. Rev. D 50, 1992 (1994).
  * (41) R. A. Lacey, P. Liu, N. Magdy, M. Csanád, B. Schweid, N. N. Ajitanand, J. Alexander and R. Pak, Universe 4, 22 (2018).
  * (42) F. S. Navarra, O. V. Utyuzh, G. Wilk and Z. Wlodarczyk, Phys. Rev. D 67, 114002 (2003); G. N. Fowler et al., Phys. Rev. C 40, 1219 (1989).


# Practical Trustworthiness Model for DNN in Dedicated 6G Application
††thanks: This work was partially supported by the DFG Project Nr. 403579441,
”Meteracom: Metrology for parallel THz communication channels.”

Anouar Nechi1, Ahmed Mahmoudi1, Christoph Herold2, Daniel Widmer1, Thomas
Kürner2, Mladen Berekovic1,
and Saleh Mulhem1
1Institute of Computer Engineering, University of Lübeck, Lübeck, Germany
2Institute for Communications Technology, Technische Universität Braunschweig,
Braunschweig, Germany
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Artificial intelligence (AI) is considered an efficient response to several
challenges facing 6G technology. However, AI still suffers from a huge trust
issue due to its ambiguous way of making predictions. Therefore, there is a
need for a method to evaluate the AI’s trustworthiness in practice for future
6G applications. This paper presents a practical model to analyze the
trustworthiness of AI in a dedicated 6G application. In particular, we present
two customized deep neural networks (DNNs) to solve the automatic modulation
recognition (AMR) problem in Terahertz communications-based 6G technology.
Then, a specific trustworthiness model and its attributes, namely data
robustness, parameter sensitivity, and security covering adversarial examples,
are introduced. The evaluation results indicate that the proposed
trustworthiness attributes are crucial to evaluate the trustworthiness of DNN
for this 6G application.

###### Index Terms:

6G communication, Terahertz band, AI, Modulation recognition, Trustworthiness

##  I Introduction

The sixth-generation (6G) network technology aims to outperform the current
wireless standards by utilizing frequencies above 100 GHz [1]. Hence,
designing efficient communication systems at these frequencies is far more
complex than those at lower frequency systems. 6G technology increasingly
relies on two main pillars: Terahertz communications (THzCom) and Machine
Learning (ML). While 6G has made one further step toward THzCom according to
IEEE802.15.3d standard [2], ML is recommended as a novel solution for 6G
performance optimization [1]. In other words, the ultra-wide THz band ranging
from 0.1 to 10 THz is foreseen as an excellent candidate for 6G, whereas ML
has proved its efficiency in solving technical problems in communication
systems [3].

Furthermore, ML ambiguously solves several problems in wireless
communications. For instance, Deep Neural Networks (DNNs) as a subset of ML
have been used in a black-box manner to solve the automatic modulation
recognition problem [4]. Therefore, there is a huge need to understand the
risk of deploying such an Artificial Intelligence (AI) algorithm. According to
an Independent High-Level Expert Group on AI [5], the only possibility to
achieve the maximum benefits of AI is to ensure its trustworthiness during all
steps of development and use. The concept of trustworthy AI can be perceived
as a response to mitigate the risks of deploying AI [6]. Several works have
proposed definitions of system trustworthiness [7], [8] or specified
definitions of trustworthy AI [5], [6]. However, these definitions are still
general and introduce principles more than practical approaches [9]. In
practice, a model of trustworthiness evaluation for a dedicated application in
6G is still missing. To our knowledge, the literature comprises studies
investigating AI, especially DNN in communication, such as THzCom-based 6G
technology. Nevertheless, the open literature on applying DNN in the 6G domain
still needs to catch up to the problem of trustworthiness evaluation.

##  II Research Methodology & Background

Our proposed research methodology is carried out as follows:

  * •

We first define one of the 6G problems. In particular, we choose a THzCom-
based automatic modulation recognition (AMR) problem to demonstrate.

  * •

We propose two customized DNNs to solve this dedicated problem.

  * •

Then, we study trustworthiness attributes that need to be considered for this
problem, and we present the so-called trustworthiness model based on these
attributes.

  * •

Finally, we apply this model as a practical approach to evaluate the
trustworthiness of the customized DNNs.

The focus of this research methodology is not on developing DNNs to solve the
AMR problem but on using the customized DNNs as practical examples to evaluate
their trustworthiness in the 6G environment. In the following, we introduce
THzCom-based AMR as one of the 6G problems, and we review the available DNN-
based solutions for such a problem. Then, we investigate the available
trustworthiness models for DNNs.

###  II-A Deep Learning-based AMR for THz Communication

In modern communication systems, a transmitter can use a pool of modulation
schemes to control data rate and bandwidth usage. While the transmitter
adaptively selects the modulation type, the receiver may or may not know the
modulation type. This problem is usually perceived as a classification
problem, where the receiver aims at recognizing and classifying the
modulation. To solve such a problem, modulation information can be supplied in
each signal frame, allowing the receiver to identify the modulation type and
react accordingly. However, this approach has become more expensive since
modern wireless networks are very heterogeneous, and the number of users is
increasing significantly. Therefore, such an approach may not be efficient
enough in real-world scenarios as it degrades spectrum efficiency due to the
additional information in each signal frame [10].

AMR has been proposed to detect the modulation scheme of received signals
without any potential overhead in the network protocol. Ultimately, the
signals are demodulated, and the received data is recovered correctly.
Further, conventional AMR approaches require a huge amount of computation or
experts’ feature extraction experience [11]. To overcome these issues, deep
learning (DL) is considered a powerful tool that can be used for AMR to
provide high classification accuracy. DL does not require prior pre-processing
or feature extraction, making it more efficient than conventional approaches.
For instance, Convolutional Neural Networks (CNNs) were used in [12, 13, 14]
to extract features from raw I/Q data and perform classification. In [4],
Recurrent Neural Network (RNN)-based AMR has been proposed to extract
sequence-correlated features of I/Q signal components and amplitude/phase
signal components to recognize modulation schemes. Other works employed RNNs
to estimate signal parameters and correct signal distortions like Carrier
Frequency Offset (CFO) and multipath fading [3]. The results revealed that the
proposed RNN model provides not only good accuracy in signal distortion
estimation but also outperforms many DL methods in terms of classification
accuracy.

###  II-B AI Trustworthiness

Several works have investigated the concepts of trustworthiness and
dependability to determine their attributes. In _system design_ ,
availability, reliability, safety, integrity, and maintainability are defined
as dependability attributes [7]. Nevertheless, this definition does not cover
all security attributes in which the definition excludes confidentiality. In
[8], trustworthiness is defined as a twin of dependability that includes the
following attributes: reliability, safety, maintainability, availability,
integrity, and confidentiality. This definition considers security as one of
the dependability attributes. In _AI-based system design_ , the above
definitions of trustworthiness do not cover the recent AI requirements. AI is
highly data-dependent and needs dedicated attributes for its trustworthiness.
Therefore, new attributes of trustworthiness have been introduced, mainly
security, robustness, safety, transparency, and fairness [15, 9]. However,
these attributes are general and not specified for a dedicated AI application.
To determine the trustworthiness attributes of DNN regarding AMR in THzCom-
based 6G technology, the interaction between DNN and its host environment
needs to be carefully investigated and described.

###  II-C Paper’s Contribution

As 6G is crossing the primeval stage of its development, it is the right time
to consider the trustworthiness of DL deployed in this technology. This paper
proposes a trustworthiness model to analyze DNNs designed for recognizing
modulation schemes in THzCom-based 6G technology. To the best of our
knowledge, this work introduces the first practical approach to evaluating the
trustworthiness of DNNs designed for AMR in THzCom-based 6G technology.

##  III Deep learning-based Automatic Modulation Recognition

###  III-A Synthetic THz dataset

A dataset of transmitted I/Q samples has been used for the AMR task. The THz-
dataset contains seven modulation schemes: BPSK, QPSK, 8PSK, QAM16, QAM64,
8APSK, and OOK. Each modulation scheme consists of 26 Signal-to-Noise-Ratio
(SNR) levels with 4096 examples per level. The total number of samples in the
dataset is $745,472$. It was generated using the link-level simulation module
of the Simulator for Mobile Networks (SiMoNe) [16]. The link level simulation
module was developed to simulate point-to-point communication links under the
influence of realistic propagation effects in accordance with the
IEEE802.15.3d standard [2]. The simulated transmission was performed using a
Root-Raised-Cosine (RRC) transmit pulse and an AWGN channel. The Nyquist
Bandwidth is 880 MHz with an oversampling factor of 8, and it has not been
subjected to any channel coding technique. All samples have the exact
representation to make data processing easier. The THz dataset samples have a
1024 $\times$ 2 shape (I/Q representation).

###  III-B Two DNN Models for Automatic Modulation Classification

DNN consists of multiple layers that process input data and generate a set of
probabilities (classification). Each layer comprises a set of parameters
(weights and biases) used to process the final output in conjunction with the
activation function. In the following, two DNNs are adopted using the proposed
THz dataset to classify THz modulation schemes. The resulting DNN classifiers
have a 32-bit floating point (FP) parameter format.

TABLE I: CNN network layout

Layer | Output dimensions
---|---
Input layer | 1024 $\times$ 2 $\times$ 1
Conv2D + BN + ReLU | 1024 $\times$ 2 $\times$ 64
Max Pooling | 512 $\times$ 2 $\times$ 64
Conv2D + BN + ReLU | 512 $\times$ 2 $\times$ 32
Max Pooling | 256 $\times$ 2 $\times$ 32
Conv2D + BN + ReLU | 256 $\times$ 2 $\times$ 16
Max Pooling | 128 $\times$ 2 $\times$ 16
FC/SeLU | 128
FC/SeLU | 128
FC/Softmax | 7

####  III-B1 CNN for AMR

CNNs have been widely used for computer vision problems [17]. A CNN model can
learn directly from the raw data without prior expert feature extraction or
pre-processing of the raw data. To benefit from this property in AMR, we
construct a CNN including three convolutional layers. Each layer is followed
by a batch normalization layer, a ReLU activation function, and a MaxPooling
layer. We feed the raw I/Q samples of each radio signal into the CNN model.
The extracted feature maps are then forwarded to the fully connected region of
the network for classification, where we employ the Scaled exponential Linear
Unit (SeLU) activation function and an Alpha dropout. The proposed CNN
classifier has $555,287$ parameters, and its layout is shown in Table I.

The CNN classifier achieves, on average, $68.8\%$ accuracy across all SNR
levels. Fig. 1 shows the confusion matrix of CNN regarding each modulation
scheme. CNN classifier imprecisely predicts the higher-order modulation
schemes, namely 16QAM and 64QAM, and achieves only $56.2\%$ and $55.9\%$
correct predictions, respectively. In contrast, the low-order modulation
schemes appear to be the least confused, achieving $84.6\%$ for BPSK and
$93.1\%$ for OOK.

Figure 1: 7-Modulation Confusion Matrix for CNN Trained on THz Dataset.

####  III-B2 ResNet for AMR

Deep Residual Networks (ResNets) are enhanced versions of CNN. ResNet uses
skip connections to process features at multiple scales and depths through the
network. Moreover, it is possible to use wider layers, train effectively with
fewer epochs, and achieve better results compared to traditional CNN [18]. We
construct a ResNet layout similar to [19] for radio signal classification.
Fig. 2 shows the proposed ResNet architecture. It consists of six residual
units, each with two skip connections, followed by a fully connected region
with the same configuration as the proposed CNN but with only $159015$
parameters.

Figure 2: ResNet layout Figure 3: 7-Modulation Confusion Matrix for ResNet
Trained on THz Dataset.

The ResNet classifier achieves $70.8\%$ accuracy across all SNR levels. ResNet
classifier exhibits $2\%$ higher accuracy than CNN and has fewer parameters.
This result emphasizes the effectiveness of ResNets over conventional CNN
classifiers. Fig. 3 shows the confusion matrix of ResNet. $16.8\%$ confusion
between 16QAM and 64QAM is noted only. For the remaining modulation schemes,
we observe a slight improvement in accuracy.

##  IV Trustworthiness: Model & Attributes

To determine the trustworthiness attributes of DNN regarding AMR in THzCom-
based 6G technology, we first formulate DNN as a function of multiple inputs
and parameters, and we link this formulation and the trustworthiness
attributes as follows.

The layer $i$ of a DNN can be seen as an operation $f_{i}[p_{i}](x_{i-1})$,
where $p_{i}$ represents a set of layer $i$’s parameters
$p_{i}=(W^{j}_{i},b_{i})$ including $j$ weights and one bias, and $x_{i-1}$ is
the output of the previous layer. A composition of these operations defines
the DNN classifier $f_{\textrm{DNN}}$ as,

$f_{\textrm{DNN}}(x_{\textrm{in}};p)=f_{l}[p_{l}]\circ\cdots\circ
f_{2}[p_{2}]\circ f_{1}[p_{1}](x_{\textrm{in}})$ (1)

Where $x_{\textrm{in}}$ is an input signal, $p$ is a set of DNN parameters
$p=\\{p_{1},...,p_{l}\\}$, and $l$ is the number of DNN layers. The values of
DNN parameters and the model hyperparameters are given during the training
phase based on the THz dataset. In the prediction phase, the DNN classifier
$f_{\textrm{DNN}}(x_{\textrm{in}};p)$ can be perceived as a function of two
inputs: The trained parameters $p$ and the signal $x_{\textrm{in}}$ like an
input variable.

Therefore, the proposed trustworthiness model of such a DNN considers only the
input signals $x_{\textrm{in}}$ and the DNN parameters
$p=\\{p_{1},...,p_{l}\\}$. Other building blocks of DNN are considered
reliable and trustworthy such as activation functions etc. This model helps to
explain how DNN interacts with the THzCom environment and the user. Fig. 4
illustrates the three trustworthiness attributes that need to be considered in
DNN-based AMR, described as follows:

  1. 1.

Data robustness analysis helps to understand when the DNN classifiers exhibit
low accuracy due to environmental variation. It aims at evaluating such
variations and their impact on the quality of DNN classifiers [9]. Precisely,
DNN robustness analysis investigates a noisy environment effect on the input
signals $x_{\textrm{in}}$ and its impact on the DNN classification accuracy.
Here, different SNR levels are applied to input signal $x_{\textrm{in}}$, and
then, the drop in the DNN accuracy is observed and estimated.

  2. 2.

Parameters sensitivity analysis provides a deep understanding of DNN’s
reliability, especially the unreliable classification causes. The reliability
can be evaluated by analyzing the sensitivity of the DNN parameters
$p=\\{p_{1},...,p_{l}\\}$ for given signals. Reliability indicates the DNN
classifier should perform with the same accuracy as it’s intended without
failure (unreliable classification with less than 50% accuracy). The proposed
sensitivity analysis follows a random bit flipping model [20].

  3. 3.

Adversarial examples indicate the impact of deterministic signal changes on
the DNN classifiers introduced by an attacker. Adversarial example attacks can
be performed for Security evaluation. Here, the attacker chooses and generates
inputs $x_{\textrm{in}}$ to confuse DNN classifiers during the inference
phase, resulting in misclassification [21].

Figure 4: The Proposed Trustworthiness Model of DNN for AMR in 6G: (1) data
robustness analysis, (2) parameter sensitivity analysis, and (3) Adversarial
examples

Our trustworthiness model excludes the transparency of DNN from its attributes
as AMR doesn’t use private data, and it also ignores DNN fairness as the used
dataset is balanced. The misclassification leads to selecting an incorrect
scheme, and the received signals cannot be demodulated. This event and its
consequence are already involved in the reliability attribute. Therefore, DNN
safety can be seen as a subset of reliability in our application.

##  V Trustworthiness Analysis of DNN for AMR

In this section, we analyze the trustworthiness of the proposed CNN and ResNet
by using our trustworthiness model and its attributes.

###  V-A Data Robustness Analysis

The impact of environmental variation on the trained DNN model is considered a
significant factor of trustworthiness. In other words, the trained DNN model
should be aware of the diverse data distribution regarding different
environmental scenarios [9]. In this context, the impact of a noisy
environment on DL-based AMR is evaluated. This problem is critical as it
affects the data robustness of the DNN model.

Figure 5: Data Robustness Analysis of both CNN and ResNet on THz Dataset.

SNR is a crucial metric in any communication system. SNR quantifies the
environmental variation by indicating the signal quality concerning a
communication channel noise. To analyze the data robustness of DL-based AMR,
the following steps are carried out: (1) The dataset is split into a training
and testing set with consideration of the various SNR levels to maintain a
balanced dataset, (2) DNN models are trained based on the resulting dataset,
and (3) the accuracy of the proposed DNN models is evaluated considering the
various SNR levels.

Further, we apply the mentioned steps to the proposed CNN and ResNet models.
Fig. 5 shows that data samples with low SNR ranging from -20 to -4 dB are hard
to classify and score a maximum accuracy of 50%. With such a noise level, the
constellation of the received signals is random and does not form meaningful
clusters to distinguish between the different modulation schemes. It’s worth
noting that the model accuracy increases when the SNR increases from -2 dB to
10 dB. The model accuracy attains 99% as SNR approaches 10 dB. The highest
model accuracy is achieved starting from 10 dB. Moreover, the ResNet model
exhibits better accuracy than the CNN model in the SNR interval of -2 dB to 10
dB. The accuracies of both models are correlated out of this interval. As a
result, ResNet-based AMR is more robust than CNN-based AMR regarding the noisy
channel variation.

###  V-B Sensitivity Analysis

AI sensitivity analysis determines vulnerable bits that significantly decrease
the classification accuracy when flipped. Sensitivity analysis relies on a
bit-flipping model of AI parameters. Sensitivity analysis aims to provide a
deep understanding of AI’s behavior and gives some hints towards explaining
AI’s decision-making.

To conduct the sensitivity analysis of CNN and ResNet classifiers, a single-
bit flip is randomly introduced to the DNN’s parameters [20]. Both the bit
position and the targeted parameter are uniformly distributed. First, we
randomly inject single-bit faults 1000 times at different bit positions and
parameter locations of each layer of the CNN classifier. In the case of the
ResNet, we randomly inject single-bit faults in the residual block,
convolution, and dense layers. Nevertheless, the injected faults in the
convolution and dense layers are performed similarly to the CNN. However, the
faults in the residual block are randomly injected at different bit positions,
parameter locations, and layers. The above fault injection experiments are
conducted during inference.

Figure 6: Weight Sensitivity of (a) CNN, and (b) ResNet Classifiers

The single-bit faults injected in 32-bit FP parameters indicate that the
exponent (from 23 to 30-bit position) is more sensitive than the mantissa
(from 0 to 22-bit position). This emphasizes the well-known results in public
literature. To better understand the exponent sensitivity, we divide the
vulnerable exponent bits into two categories: the first category includes the
vulnerable bits resulting in misclassification (unreliable classifier with
accuracy lower than 50%), and the second covers the vulnerable bits resulting
in accuracy degradation.

Fig. 6-a illustrates the impact of single-bit faults on the CNN classifier
regarding convolution layers C1, C2, C3 and dense layers D1, D2, D3. The
unreliable classification is observed at 25 in C1 and 30 in C1, C2, C3, D2,
D3. However, the flipping of bit 30 in D1 results only in accuracy
degradation. It should be noted that the faults injected in the remaining
layers show insignificant accuracy degradation. Fig. 6-b shows the impact of
bit flipping on all layers of the ResNet classifier. The 30th bit (i.e. bit
31) is more sensitive than the others as it causes unreliable classification
across all layers. The remaining vulnerable bits cause only an accuracy drop.

As a result, flipping the vulnerable 30th-bit causes the misclassification of
both classifiers. Other vulnerable bits show only accuracy degradation.

###  V-C Security Analysis

In [21], several neural network models exhibit vulnerabilities to adversarial
examples, where the attacker generates some inputs that lead to
misclassification. These inputs are slightly different from the original
inputs that are classified correctly, yet they are likely to cause such
misclassification. The adversarial examples mainly occur due to some “linear
behavior in high-dimensional spaces” [22]. This observation introduces many
efficient adversarial example attacks such as Fast Gradient Method [22],
Projected Gradient Descend [23].

TABLE II: Results of Adversarial Examples Attacks

Attack | $\textbf{AER}_{\textbf{CNN}}$ | $\textbf{AER}_{\textbf{ResNet}}$
---|---|---
FGM [22] | $18.83\%$ | $22.12\%$
PGD [23] | $6.11\%$ | $5.60\%$
NewtonFool [24] | $26.84\%$ | $28.50\%$
DeepFool [25] | $26.07\%$ | $15.43\%$
HopSkipJump [26] | $14.92\%$ | $8.04\%$
Zoo [27] | $26.57\%$ | $19.25\%$
C&W [28] $L_{2}$ | $53.9\%$ | $59.25\%$
C&W [28] $L_{\infty}$ | $21.57\%$ | $14.57\%$

To analyze the impact of adversarial examples, we launch eight attacks to
generate adversarial examples against the investigated CNN and ResNet
classifiers using the Adversarial Robustness Toolbox (v1.2.0) [29]. We set up
each attack using the predefined sets of attack parameters. Then, we perform
the same attacks on both classifiers. Table II shows the attack results of
Fast Gradient Method (FGM) [22], Projected Gradient Descend (PGD) [23],
NewtonFool [24], DeepFool [25], HopSkipJump [26], Zeroth Order Optimization
(Zoo) [27], and Carlini & Wagner methods (C&W) [28] over $L_{2}$, and
$L_{\infty}$. The adversarial example resistance (AER) quantifies the model’s
correct classifications despite adversarial example generation. For instance,
both classifiers exhibit comparably high AER against C&W over $L_{2}$, while
the generated adversarial examples by PGD fundamentally devastate both.
Generally, both models lead to different AER w.r.t. the respective attacks.
Thus, the decision for the more-resistant model against adversarial examples
will remain dependent on the chosen attacks.

###  V-D Trustworthiness Evaluation

According to the above analysis, the trustworthiness level of the investigated
classifiers can be evaluated as follows. First, when the SNR ranges from 0 to
30 dB, both classifiers are robust against environmental variations. Here,
ResNet shows greater robustness than the CNN classifier. Second, the
sensitivity analysis of the classifier’s parameters indicates that flipping
the vulnerable 30th-bit results in unreliable classifications. Finally, both
classifiers show different levels of resistance against selected adversarial
example attacks without yielding a clear verdict for either DNN. Future work
may identify suitable attacks to provide the required metrics.

##  VI Conclusion

In this paper, we introduced a methodology to build a practical
trustworthiness model of Deep Neuronal Networks (DNN) dedicated to one of the
6G applications. The need for such a model is significant as 6G technology
requires higher levels of reliability and security compared to prior
generations.

Particularly, we constructed two DNN classifiers addressing the automatic
modulation recognition (AMR) problem in a THzCom-based 6G environment. Then,
we applied our trustworthiness model to analyze the classifiers w.r.t.
attributes chosen to meet this environment: Robustness, DNN parameter
reliability, and DNN adversarial example resistance. Based on our experiments
results, we conclude our trustworthiness model a suitable approach to analyze
the trustworthiness of the used DNN for AMR in THzCom-based 6G technology.

## References

  * [1] V. Petrov, T. Kurner, and I. Hosako, “Ieee 802.15. 3d: First standardization efforts for sub-terahertz band communications toward 6g,” _IEEE Communications Magazine_ , vol. 58, no. 11, pp. 28–33, 2020.
  * [2] “Ieee standard for high data rate wireless multi-media networks–amendment 2: 100 gb/s wireless switched point-to-point physical layer,” _IEEE Std 802.15.3d-2017 (Amendment to IEEE Std 802.15.3-2016 as amended by IEEE Std 802.15.3e-2017)_ , pp. 1–55, 2017.
  * [3] S. Hanna, C. Dick, and D. Cabric, “Combining deep learning and linear processing for modulation classification and symbol decoding,” in _GLOBECOM 2020-2020 IEEE Global Communications Conference_. IEEE, 2020, pp. 1–7.
  * [4] Y. Guo, H. Jiang, J. Wu, and J. Zhou, “Open set modulation recognition based on dual-channel lstm model,” _arXiv preprint arXiv:2002.12037_ , 2020.
  * [5] A. HLEG, “High-level expert group on artificial intelligence, ethics guidelines for trustworthy ai,” AI HLEG, Tech. Rep., 2019.
  * [6] S. Thiebes, S. Lins, and A. Sunyaev, “Trustworthy artificial intelligence,” _Electronic Markets_ , vol. 31, no. 2, pp. 447–464, 2021.
  * [7] A. Avizienis, J.-C. Laprie, B. Randell, and C. Landwehr, “Basic concepts and taxonomy of dependable and secure computing,” _IEEE Transactions on Dependable and Secure Computing_ , vol. 1, no. 1, pp. 11–33, 2004.
  * [8] B. Bauer, M. Ayache, S. Mulhem, M. Nitzan, J. Athavale, R. Buchty, and M. Berekovic, “On the dependability lifecycle of electrical/electronic product development: The dual-cone v-model,” _Computer_ , vol. 55, no. 9, pp. 99–106, 2022.
  * [9] B. Li, P. Qi, B. Liu, S. Di, J. Liu, J. Pei, J. Yi, and B. Zhou, “Trustworthy ai: From principles to practices,” _ACM Comput. Surv._ , aug 2022, just Accepted. [Online]. Available: https://doi.org/10.1145/3555803
  * [10] S. Chen, Y. Zhang, Z. He, J. Nie, and W. Zhang, “A novel attention cooperative framework for automatic modulation recognition,” _IEEE Access_ , vol. 8, pp. 15 673–15 686, 2020.
  * [11] W. Xiao, Z. Luo, and Q. Hu, “A review of research on signal modulation recognition based on deep learning,” _Electronics_ , vol. 11, no. 17, p. 2764, 2022.
  * [12] J. Shi, S. Hong, C. Cai, Y. Wang, H. Huang, and G. Gui, “Deep learning-based automatic modulation recognition method in the presence of phase offset,” _IEEE Access_ , vol. 8, pp. 42 841–42 847, 2020.
  * [13] H. Gu, Y. Wang, S. Hong, and G. Gui, “Blind channel identification aided generalized automatic modulation recognition based on deep learning,” _IEEE Access_ , vol. 7, pp. 110 722–110 729, 2019.
  * [14] Y. Wang, M. Liu, J. Yang, and G. Gui, “Data-driven deep learning for automatic modulation recognition in cognitive radios,” _IEEE Transactions on Vehicular Technology_ , vol. 68, no. 4, pp. 4074–4077, 2019.
  * [15] A. Jobin, M. Ienca, and E. Vayena, “The global landscape of ai ethics guidelines,” _Nature Machine Intelligence_ , vol. 1, no. 9, pp. 389–399, 2019.
  * [16] J. M. Eckhardt, C. Herold, B. K. Jung, N. Dreyer, and T. Kürner, “Modular link level simulator for the physical layer of beyond 5g wireless communication systems,” _Radio Science_ , vol. 57, no. 2, p. e2021RS007395, 2022, e2021RS007395 2021RS007395. [Online]. Available: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2021RS007395
  * [17] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” _Communications of the ACM_ , vol. 60, no. 6, pp. 84–90, 2017.
  * [18] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [19] T. J. O’Shea, T. Roy, and T. C. Clancy, “Over-the-air deep learning based radio signal classification,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 12, no. 1, pp. 168–179, 2018.
  * [20] G. Li, S. K. S. Hari, M. Sullivan, T. Tsai, K. Pattabiraman, J. Emer, and S. W. Keckler, “Understanding error propagation in deep learning neural network (dnn) accelerators and applications,” ser. SC ’17. New York, NY, USA: Association for Computing Machinery, 2017\. [Online]. Available: https://doi.org/10.1145/3126908.3126964
  * [21] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in _2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2014. [Online]. Available: http://arxiv.org/abs/1312.6199
  * [22] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in _3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings_ , Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6572
  * [23] A. Kurakin, I. J. Goodfellow, and S. Bengio, “Adversarial examples in the physical world,” in _Artificial intelligence safety and security_. Chapman and Hall/CRC, 2018, pp. 99–112.
  * [24] U. Jang, X. Wu, and S. Jha, “Objective metrics and gradient descent algorithms for adversarial examples in machine learning,” in _Proceedings of the 33rd Annual Computer Security Applications Conference_ , ser. ACSAC ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 262–277. [Online]. Available: https://doi.org/10.1145/3134600.3134635
  * [25] S. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: A simple and accurate method to fool deep neural networks,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016_. IEEE Computer Society, 2016, pp. 2574–2582. [Online]. Available: https://doi.org/10.1109/CVPR.2016.282
  * [26] J. Chen, M. I. Jordan, and M. J. Wainwright, “Hopskipjumpattack: A query-efficient decision-based attack,” in _2020 IEEE Symposium on Security and Privacy (SP)_ , 2020, pp. 1277–1294.
  * [27] P.-Y. Chen, H. Zhang, Y. Sharma, J. Yi, and C.-J. Hsieh, “Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models,” in _Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security_ , ser. AISec ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 15–26. [Online]. Available: https://doi.org/10.1145/3128572.3140448
  * [28] N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in _2017 IEEE Symposium on Security and Privacy (SP)_. Los Alamitos, CA, USA: IEEE Computer Society, may 2017, pp. 39–57. [Online]. Available: https://doi.ieeecomputersociety.org/10.1109/SP.2017.49
  * [29] M.-I. Nicolae, M. Sinn, M. N. Tran, B. Buesser, A. Rawat, M. Wistuba, V. Zantedeschi, N. Baracaldo, B. Chen, H. Ludwig, I. Molloy, and B. Edwards, “Adversarial robustness toolbox v1.2.0,” _CoRR_ , vol. 1807.01069, 2018\. [Online]. Available: https://arxiv.org/pdf/1807.01069