# NOMA-ISAC: Performance Analysis and Rate Region Characterization

Chongjun Ouyang, Yuanwei Liu, and Hongwen Yang C. Ouyang and H. Yang are with
the School of Information and Communication Engineering, Beijing University of
Posts and Telecommunications, Beijing, 100876, China (e-mail:
{DragonAim,yanghong}@bupt.edu.cn).Y. Liu is with the School of Electronic
Engineering and Computer Science, Queen Mary University of London, London, E1
4NS, U.K. (e-mail: yuanwei.liu@qmul.ac.uk).

###### Abstract

This paper analyzes the performance of a multiuser integrated sensing and
communications (ISAC) system, where nonorthogonal multiple access (NOMA) is
exploited to mitigate inter-user interference. Closed-form expressions are
derived to evaluate the outage probability, ergodic communication rate, and
sensing rate. Furthermore, asymptotic analyses are carried out to unveil
diversity orders and high signal-to-noise ratio (SNR) slopes of the considered
NOMA-ISAC system. As the further advance, the achievable sensing-communication
rate region of ISAC is characterized. It is proved that ISAC system is capable
of achieving a larger rate region than the conventional frequency-division
sensing and communications (FDSAC) system.

###### Index Terms:

Integrated sensing and communications (ISAC), nonorthogonal multiple access
(NOMA), performance analysis, rate region.

##  I Introduction

Next-generation wireless networks are envisioned to provide the capability of
simultaneous communications and sensing [1]. This gave birth to the concept of
integrated sensing and communications (ISAC), where wireless communications
and radar sensing are integrated together to share the same spectrum and
hardware platform [1]. Compared with the conventional frequency-division
sensing and communications (FDSAC) scheme, in which sensing and communications
require isolated frequency bands as well as hardware infrastructures, ISAC is
expected to be more spectrum- and hardware-efficient and has sparked a
significant burst of recent research attention [1, 5, 2, 3, 4, 6, 7, 8].

From an information-theoretic perspective, the performance of ISAC can be
evaluated by two fundamental metrics, i.e., sensing rate (SR) and
communication rate (CR) [5]. Specifically, SR measures how much environmental
information can be estimated, wheras CR measures how much data information can
be transmitted [5]. These two metrics were discussed in several studies for
the sake of demonstrating the information-theoretic limits of ISAC systems;
see [5, 6, 7, 8] and the references therein. However, we notice that these
studies would have made more practical sense if the authors had took account
of the influence of both inter-user interference (IUI) and channel fading.
Furthermore, despite its significance, a rigorous comparison between the
sensing-communication rate regions achieved by ISAC and FDSAC is still
missing.

In this letter, we investigate the performance of a multiuser ISAC system
where a nonorthogonal multiple access (NOMA) protocol is involved in the
communication procedure for IUI mitigation [9, 10]. For the sake of
convenience, this system is also termed “NOMA-ISAC”. It is worth mentioning
that the NOMA-ISAC framework was firstly established in [2]. Yet, this work
focused more on the waveform design and shed less light on the fundamental
properties of SR as well as CR. The main contributions of our work are
summarized as follows. Firstly, by taking the influence of IUI as well as
channel fading into consideration, we derive closed-form expressions for
outage probability (OP), ergodic CR (ECR), and SR, respectively. Secondly, by
letting the signal-to-noise ratio (SNR) go to infinity, we perform asymptotic
analyses to the aforementioned metrics in order to unveil the diversity order
as well as high-SNR slope of the NOMA-ISAC system. Finally, based on the
developed analytical results, we characterize the sensing-communication rate
region achieved by NOMA-ISAC and prove that ISAC is capable of achieving a
larger rate region that FDSAC.

Figure 1: Illustration of an NOMA-ISAC system

##  II System Model

There is a radar-communications (RadCom) base station (BS) equipped with a
single transmit antenna and $M$ receive antennas, which is serving a pair of
single-antenna communication users (CUs)111The developed analytical results
can be extended to systems with more than two CUs. We skip further details for
sake of brevity., i.e., one near user (NU) and one far user (FU), while
sensing the radar targets (RTs) in the surrounding environment, as depicted in
Figure 1. Let
$\mathbf{x}=\left[x_{1},\cdots,x_{L}\right]^{\mathsf{T}}\in{\mathbbmss{C}}^{L\times
1}$ be a dual-functional signal vector, with $L$ being the length of the
communication frame/radar pulse. From a sensing perspective, $x_{l}$ for
$l\in{\mathcal{L}}=\left\\{1,\cdots,L\right\\}$ represents the radar snapshot
transmitted at the $l$th time slot. For communications, $x_{l}$ is the $l$th
data symbol.

###  II-A Communication Model

By transmitting $\mathbf{x}$ to the two CUs, the received signal matrix at the
CU pair is given by

$\displaystyle{\mathbf{X}}_{\rm{c}}={\mathbf{h}}{{\mathbf{x}}^{\mathsf{T}}}+{\mathbf{N}}_{\rm{c}}\in{\mathbbmss{C}}^{2\times
L},$ (1)

where ${\mathbf{N}}_{\rm{c}}\in{\mathbbmss{C}}^{2\times L}$ is the additive
white Gaussian noise (AWGN) with variance of each entry being
$\sigma_{\rm{c}}^{2}$, and ${\mathbf{h}}\in{\mathbbmss{C}}^{2\times 1}$ is the
communication channel matrix, which is assumed to be known by the BS. In this
work, NOMA is employed at the BS for serving multiple CUs and the
communication channel matrix reads
${\mathbf{h}}=\left[h_{\rm{N}},h_{\rm{F}}\right]^{\mathsf{T}}$, where
$h_{\rm{N}}$ and $h_{\rm{F}}$ denote the channel coefficients of NU and FU,
respectively. We assume that all the communication links shown in Figure 1
suffer Rayleigh fading and the CUs are ordered based on their channel quality,
i.e., $\left|h_{\rm{N}}\right|>\left|h_{\rm{F}}\right|$ [9, 10]. Consequently,
we have $\left|h_{\rm{N}}\right|=\max\\{|h_{1}|,|h_{2}|\\}$ and
$\left|h_{\rm{F}}\right|=\min\\{|h_{1}|,|h_{2}|\\}$, where
$h_{1}\sim{\mathcal{CN}}\left(0,\varrho_{1}\right)$ and
$h_{2}\sim{\mathcal{CN}}\left(0,\varrho_{2}\right)$ are two mutually
independent Gaussian random variables with $\varrho_{1}>0$ and $\varrho_{2}>0$
indicating the fading severity. Under the framework of NOMA-ISAC, the signal
vector $\mathbf{x}$ is given by

$\displaystyle{\mathbf{x}}^{\mathsf{T}}=\sqrt{p}\left[\sqrt{\alpha_{\rm{N}}},\sqrt{\alpha_{\rm{F}}}\right]{\mathbf{S}},$
(2)

where $p$ is the power budget at each time slot, $\alpha_{u}$ for
$u\in\left\\{\rm{N},\rm{F}\right\\}$ denote power allocation factors with
$\alpha_{\rm{N}}+\alpha_{\rm{F}}=1$ and $\alpha_{\rm{N}}<\alpha_{\rm{F}}$, and
$\mathbf{S}\in{\mathbbmss{C}}^{2\times L}$ contains two unit-power data
streams intended for the two CUs. The data streams are assumed to be
independent with each other so that [4, 2, 3]

$\displaystyle{\mathbf{S}}{\mathbf{S}}^{\mathsf{H}}\approx L\mathbf{I}_{2}.$
(3)

###  II-B Sensing Model

By transmitting $\mathbf{x}$ to sense the RTs, the reflected echo signal
matrix at the receiver of the RadCom BS is given by

$\displaystyle{\mathbf{Y}}_{\rm{s}}={\mathbf{g}}{\mathbf{x}}^{\mathsf{T}}+{\mathbf{N}}_{\rm{s}}\in{\mathbbmss{C}}^{M\times
L},$ (4)

where ${\mathbf{N}}_{\rm{s}}\in{\mathbbmss{C}}^{M\times L}$ is the AWGN matrix
with variance of each entry being $\sigma_{\rm{s}}^{2}$, and
$\mathbf{g}=\left[g_{1},\cdots,g_{M}\right]^{\mathsf{T}}\in{\mathbbmss{C}}^{M\times
1}$ represents the target response vector with $g_{m}$ for
$m\in{\mathcal{M}}=\left\\{1,\cdots,M\right\\}$ representing the target
response from the transmit antenna to the $m$th receive antenna. The target
response vector can be written as [11, 4]

$\displaystyle{\mathbf{g}}=\sum\nolimits_{k}\beta_{k}{\mathbf{a}}\left(\theta_{k}\right),$
(5)

where $\beta_{k}\sim{\mathcal{CN}}\left(0,\sigma_{k}^{2}\right)$ is the
complex amplitude of the $k$th RT with $\sigma_{k}^{2}$ representing the
average strength,
${\mathbf{a}}\left(\theta_{k}\right)\in{\mathbbmss{C}}^{M\times 1}$ is the
associated receive array steering vector, and $\theta_{k}$ is its angle of
arrival (AoA). It is worth noting that the BS has no prior knowledge about the
number and the corresponding AoAs of RTs. Therefore, in essence, sensing the
RTs is equivalent to extracting environmental information contained in
$\mathbf{g}$, e.g., the AoA and reflection coefficient of each RT, from the
reflected echo signal presented in (4) [1, 4].

###  II-C Baseline Scheme

In this letter, we consider NOMA-FDSAC as a baseline scenario, where the total
bandwidth is partitioned into two sub-bands according to some
$\kappa\in\left[0,1\right]$, one for radar only and the other for
communications. Particularly, we assume $\kappa$ fraction of the total
bandwidth is used for communications with the NOMA CU pair. In addition to
this, the total power in FDSAC should also be partitioned into two parts
according to some $\mu\in\left[0,1\right]$, one for radar only and the other
for communications. Specifically, the powers for communications and sensing
are given by $\mu p$ and $\left(1-\mu\right)p$, respectively. Moreover, it is
worth noting that the superiority of NOMA over orthogonal multiple access
(OMA) was discussed by a considerable amount of literature [9, 10]. In light
of this as well as the page limitations, further discussions on OMA-ISAC or
OMA-FDSAC are skipped here and left as a potential direction for future work.

Given the NOMA-ISAC and NOMA-FDSAC frameworks, we intend to investigate the
performance of communications as well as sensing. Particularly, we rely on the
sensing rate for radar target sensing, which tells the information-theoretic
limit on how much environmental information can be extracted from the
reflected radar echoes, and employ the outage probability and ergodic
communication rate to measure the performance of communications.

##  III Performance Analysis

###  III-A Performance of Communications

In this part, we investigate the communication performance of the NOMA-ISAC
and NOMA-FDSAC systems. More specifically, we aim to derive exact expressions
as well as high-SNR approximations of the outage probability and the ergodic
communication rate, respectively.

Based on the NOMA technique, NU accomplishes the successive interference
cancellation (SIC) procedure to detect FU’s message and then remove the
message from its observation, in a successive manner. Hence, the signal-to-
interference-plus-noise ratio (SINR) of the SIC process for NU to decode FU’s
message is given by
$\gamma_{t}^{\rm{sic}}=\frac{\mu_{t}p\left|h_{\rm{N}}\right|^{2}\alpha_{\rm{F}}}{\kappa_{t}\sigma_{\rm{c}}^{2}+\mu_{t}p\left|h_{\rm{N}}\right|^{2}\alpha_{\rm{N}}}$
for $t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$, and the SNR for NU to decode its
own message is given by
$\gamma_{t}^{\rm{N}}=\frac{\mu_{t}p\left|h_{\rm{N}}\right|^{2}\alpha_{\rm{N}}}{\kappa_{t}\sigma_{\rm{c}}^{2}}$,
where the subscripts “${\rm{i}}$” and “${\rm{f}}$” indicate ISAC and FDSAC,
respectively. It is worth noting that $\kappa_{\rm{i}}=\mu_{\rm{i}}=1$,
$\kappa_{\rm{f}}=\kappa$, and $\mu_{\rm{f}}=\mu$. In contrast to NU, FU
directly decodes its signal by treating NU’s message as interference, and the
corresponding SINR is given by
$\gamma_{t}^{\rm{F}}=\frac{\mu_{t}p\left|h_{\rm{F}}\right|^{2}\alpha_{\rm{F}}}{\kappa_{t}\sigma_{\rm{c}}^{2}+\mu_{t}p\left|h_{\rm{F}}\right|^{2}\alpha_{\rm{N}}}$
for $t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$. Bearing in mind the above
results, we analyze the communication performance of NOMA-FDSAC and NOMA-ISAC
in two major steps which are illustrated in the sequel.

####  III-A1 Outage Probability

At the first step, we investigate the outage performance of the two CUs. By
definition, the outage probabilities of NU and FU for
$t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$ can be written as

$\displaystyle\mathcal{P}_{t}^{\rm{N}}$
$\displaystyle=1-\Pr\left(\gamma_{t}^{\rm{sic}}>\bar{\gamma}_{t}^{\rm{F}},\gamma_{t}^{\rm{N}}>\bar{\gamma}_{t}^{\rm{N}}\right),$
(6) $\displaystyle\mathcal{P}_{t}^{\rm{F}}$
$\displaystyle=\Pr\left(\gamma_{t}^{\rm{F}}<\bar{\gamma}_{t}^{\rm{F}}\right),$
(7)

respectively, where
$\bar{\gamma}_{t}^{\rm{N}}=2^{\overline{\mathcal{R}}_{\rm{N}}/\kappa_{t}}-1$
and
$\bar{\gamma}_{t}^{\rm{F}}=2^{\overline{\mathcal{R}}_{\rm{F}}/\kappa_{t}}-1$
with $\overline{\mathcal{R}}_{\rm{N}}$ and $\overline{\mathcal{R}}_{\rm{F}}$
being the target rates of NU and FU, respectively. The following theorem
provides exact expressions for these OPs as well as their high-SNR
approximations.

######  Theorem 1.

When ${\alpha_{\rm{F}}\leq\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}}$, the OPs
of NU and FU for $t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$ are both given by
$1$. When ${\alpha_{\rm{F}}>\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}}$, the
OPs of NU and FU for $t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$ are given by

$\displaystyle\mathcal{P}_{t}^{\rm{N}}=\left(1-{\rm{e}}^{-\chi_{t,1}{\theta_{t}}{p^{-1}}}\right)\left(1-{\rm{e}}^{-\chi_{t,2}{\theta_{t}}{p^{-1}}}\right),$
(8a)
$\displaystyle\mathcal{P}_{t}^{\rm{F}}=1-{\rm{e}}^{-\chi_{t,3}{\vartheta_{t}}{p^{-1}}},$
(8b)

respectively, where
$\theta_{t}=\max\left\\{\frac{\bar{\gamma}_{t}^{\rm{N}}}{\alpha_{\rm{N}}},\vartheta_{t}\right\\}$
with
$\vartheta_{t}=\frac{\bar{\gamma}_{t}^{\rm{F}}}{\alpha_{\rm{F}}-\alpha_{\rm{N}}\bar{\gamma}_{t}^{\rm{F}}}$,
and $\chi_{t,b}=\frac{\kappa_{t}\sigma_{\rm{c}}^{2}}{\mu_{t}\varrho_{b}}$ for
$b\in\\{1,2,3\\}$ with
$\varrho_{3}=\frac{\varrho_{1}\varrho_{2}}{\varrho_{1}+\varrho_{2}}$. Let
${\alpha_{\rm{F}}>\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}}$ and
$p\rightarrow\infty$, the OPs satisfy:

$\displaystyle\mathcal{P}_{t}^{{\rm{N}}}\approx{\chi_{t,1}\chi_{t,2}}{\theta_{t}^{2}}{p^{-2}},\quad\mathcal{P}_{t}^{{\rm{F}}}\approx{\chi_{t,3}{\vartheta_{t}}{p^{-1}}}.$
(9)

###### Proof:

The proof is given in Appendix A. ∎

######  Remark 1.

The results in (9) suggest that when
$\alpha_{\rm{F}}>\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}$, the diversity
orders of NU and FU are given by $2$ and $1$, respectively, in both NOMA-ISAC
and NOMA-FDSAC.

####  III-A2 Ergodic Rate

At the second step, we discuss the ergodic communication rates of the two CUs.
Particularly, the ergodic rates of NU and FU for
$t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$ are defined as

$\displaystyle\mathcal{R}_{t}^{\rm{N}}$
$\displaystyle={\mathbbmss{E}}\left\\{{\kappa_{t}}\log_{2}\left(1+\gamma_{t}^{\rm{N}}\right)\right\\},$
(10) $\displaystyle\mathcal{R}_{t}^{\rm{F}}$
$\displaystyle={\mathbbmss{E}}\left\\{{\kappa_{t}}\log_{2}\left(1+\gamma_{t}^{\rm{F}}\right)\right\\},$
(11)

respectively. Theorem 2 provides exact expressions for these ECRs as well as
their high-SNR approximations.

######  Theorem 2.

The ECRs $\mathcal{R}_{t}^{\rm{N}}$ and $\mathcal{R}_{t}^{\rm{F}}$ are given
by

$\displaystyle\mathcal{R}_{t}^{{\rm{N}}}=\frac{\kappa_{t}}{\ln{2}}\left(\psi_{t,3}-\psi_{t,2}-\psi_{t,1}\right),$
(12a)
$\displaystyle\mathcal{R}_{t}^{{\rm{F}}}=\frac{\kappa_{t}}{\ln{2}}\psi_{t,3}-\frac{\kappa_{t}}{\ln{2}}{{\rm{Ei}}\left({{-\chi_{t,3}}/{p}}\right)}{\rm{e}}^{{\frac{\chi_{t,3}}{p}}},$
(12b)

respectively, where ${\rm{Ei}}\left(\cdot\right)$ is the exponential integral
and
$\psi_{t,b}={{\rm{Ei}}\left(-\frac{\chi_{t,b}}{\alpha_{\rm{N}}p}\right)}{{\rm{e}}^{\frac{\chi_{t,b}}{\alpha_{\rm{N}}p}}}$.
When $p\rightarrow\infty$, the ECRs satisfy:

$\displaystyle\mathcal{R}_{t}^{{\rm{N}}}\approx\kappa_{t}\log_{2}{p}-\frac{\kappa_{t}\gamma}{\ln{2}}-\kappa_{t}\log_{2}{\frac{\kappa_{t}\sigma_{\rm{c}}^{2}}{\mu_{t}{\alpha_{\rm{N}}}({\varrho_{1}+\varrho_{2}})}},$
(13a)
$\displaystyle\mathcal{R}_{t}^{{\rm{F}}}\approx-\kappa_{t}\log_{2}{{\alpha_{\rm{N}}}},$
(13b)

where $\gamma$ is the Euler–Mascheroni constant.

###### Proof:

The proof is given in Appendix B. ∎

######  Remark 2.

The results in (13) indicate that the high-SNR slopes of NU and FU in NOMA-
FDSAC are given by $\kappa$ and $0$, respectively, whereas the high-SNR slopes
of NU and FU in NOMA-ISAC are given by $1$ and $0$, respectively. This also
means that the high-SNR slopes of the sum ECR of the CU pair, i.e.,
$\mathcal{R}_{t}^{\rm{c}}\triangleq\mathcal{R}_{t}^{{\rm{N}}}+\mathcal{R}_{t}^{{\rm{F}}}$
($t\in\left\\{{\rm{i}},{\rm{f}}\right\\}$), in NOMA-FDSAC and NOMA-ISAC are
given by $\kappa$ and $1$, respectively.

###  III-B Performance of Sensing

Having characterized the performance of communications, we move on to
analyzing the performance of sensing.

Let us first consider the NOMA-ISAC case. As mentioned previously, sensing the
RTs is equivalent to extracting environmental information contained in the
target response vector, i.e., $\mathbf{g}$ in (5), from the reflected radar
echo signal matrix, i.e., ${\mathbf{Y}}_{\rm{s}}$ in (4). To define the
information-theoretic limit on how much environmental information can be
extracted, we resort to the sensing rate for sensing performance evaluation.
More specifically, the sensing rate is defined as the sensing mutual
information (MI) per unit time and the sensing MI is defined as the MI between
${\mathbf{Y}}_{\rm{s}}$ and $\mathbf{g}$ conditioned on the radar waveform
$\mathbf{x}$ [8, 11]. Assuming that each waveform symbol in the ISAC system
lasts $1$ unit time, we write the SR as follows:

$\displaystyle\mathcal{R}_{\rm{i}}^{\rm{s}}={L}^{-1}I\left({\mathbf{Y}}_{\rm{s}};{\mathbf{g}}|{\mathbf{x}}\right),$
(14)

where $I\left(X;Y|Z\right)$ denotes the MI between $X$ and $Y$ conditioned on
$Z$ [8, 11]. Theorem 3 provides an exact expression for the SR as well as its
high-SNR approximation.

######  Theorem 3.

The SR in the NOMA-ISAC system is given by

$\mathcal{R}_{\rm{i}}^{\rm{s}}={L^{-1}}\log_{2}\det\left({\mathbf{I}}_{M}+{pL}{\sigma_{\rm{s}}^{-2}}{\mathbf{R}}\right),$
(15)

where
${\mathbf{R}}={\mathbbmss{E}}\left\\{{\mathbf{g}}{\mathbf{g}}^{\mathsf{H}}\right\\}\in{\mathbbmss{C}}^{M\times
M}$ is the correlation matrix of the target response vector $\mathbf{g}$. When
$p\rightarrow\infty$, the SR satisfies:

$\mathcal{R}_{\rm{i}}^{\rm{s}}\approx\frac{\mathsf{r}}{L}\log_{2}{p}+\frac{1}{L}\sum\nolimits_{a=1}^{\mathsf{r}}\log_{2}\left(\frac{L\lambda_{a}}{\sigma_{\rm{s}}^{2}}\right),$
(16)

where ${\mathsf{r}}={\mathsf{rank}\left({\mathbf{R}}\right)}$ is the rank of
matrix ${\mathbf{R}}$ and $\left\\{\lambda_{a}\right\\}_{a=1}^{\mathsf{r}}$
are the associated positive eigenvalues.

###### Proof:

The proof is given in Appendix C. ∎

Following similar steps as those outlined in Appendix C, we obtain the SR of
the FDSAC system as follows.

######  Corollary 1.

The SR of NOMA-FDSAC can be written as

$\mathcal{R}_{\rm{f}}^{\rm{s}}=\frac{1-\kappa}{L}\log_{2}\det\left({\mathbf{I}}_{M}+\frac{\left(1-\mu\right)pL}{\left(1-\kappa\right)\sigma_{\rm{s}}^{2}}{\mathbf{R}}\right).$
(17)

When $p\rightarrow\infty$, the SR satisfies:

$\mathcal{R}_{\rm{f}}^{\rm{s}}\\!\approx\\!\frac{\mathsf{r}(1\\!-\\!\kappa)}{L}\log_{2}{p}+\frac{1\\!-\\!\kappa}{L}\sum\nolimits_{a=1}^{{\mathsf{r}}}\\!\\!\log_{2}{\frac{\left(1-\mu\right)\lambda_{a}L}{\left(1-\kappa\right)\sigma_{\rm{s}}^{2}}}.$
(18)

######  Remark 3.

The results in (16) and (18) indicate that the high-SNR slopes of
$\mathcal{R}_{\rm{i}}^{\rm{s}}$ and $\mathcal{R}_{\rm{f}}^{\rm{s}}$ are given
by $\frac{\mathsf{r}}{L}$ and $\frac{\left(1-\kappa\right)\mathsf{r}}{L}$,
respectively.

Last but not least, although the sensing rate defines the information-
theoretic limit on how much environmental information can be extracted, how to
extract this information relies on specific estimation or detection algorithms
[1]. Yet, this is beyond the scope of this paper.

###  III-C Summary of the Analytical Results

After completing all analyses, we summarize the results related to diversity
order and high-SNR slope in Table I for ease of reference. Since
$\kappa\in[0,1]$, it is observed that NOMA-ISAC achieves larger high-SNR
slopes than NOMA-FDSAC in terms of the SR as well as the sum ECR of the CU
pair. This reflects that NOMA-ISAC can provide more degrees of freedom for
both communications and sensing than NOMA-FDSAC. In fact, this superiority
mainly originates from ISAC’s integrated exploitation of spectrum and power
resources [1].

System | N | F | CU Pair | Sensing
---|---|---|---|---
$\mathcal{D}$ | $\mathcal{S}$ | $\mathcal{D}$ | $\mathcal{S}$ | $\mathcal{S}$ | $\mathcal{S}$
NOMA-FDSAC | 2 | $\kappa$ | 1 | 0 | $\kappa$ | $\frac{\left(1-\kappa\right)\mathsf{r}}{L}$
NOMA-ISAC | 2 | 1 | 1 | 0 | 1 | $\frac{\mathsf{r}}{L}$
TABLE I: Diversity Order ($\mathcal{D}$) and High-SNR Slope ($\mathcal{S}$)

##  IV Rate Region Characterization

The previous part of this paper has analyzed the basic performances of the
NOMA-ISAC and NOMA-FDSAC systems. It is now necessary to characterize the
sensing-communication rate region of these two systems as well as providing a
full and complete comparison between them.

###  IV-A Achievable Sensing-Communication Rate Region

In this subsection, we aim to characterize the achievable sensing-
communication rate region of the NOMA-ISAC and NOMA-FDSAC systems. To this
end, let ${\mathcal{R}}^{\rm{s}}$ and ${\mathcal{R}}^{\rm{c}}$ denote the
achievable SR and sum ECR, respectively. Then, the rate regions achieved by
ISAC and FDSAC are given by

$\displaystyle\mathcal{C}_{\rm{isac}}=\left\\{\left({\mathcal{R}}^{\rm{s}},{\mathcal{R}}^{\rm{c}}\right)|{\mathcal{R}}^{\rm{s}}\in\left[0,\mathcal{R}_{\rm{i}}^{\rm{s}}\right],{\mathcal{R}}^{\rm{c}}\in\left[0,\mathcal{R}_{\rm{i}}^{\rm{c}}\right]\right\\},$
(19)
$\displaystyle\mathcal{C}_{\rm{fdsac}}=\left\\{\left({\mathcal{R}}^{\rm{s}},{\mathcal{R}}^{\rm{c}}\right)\left|\begin{aligned}
&{\mathcal{R}}^{\rm{s}}\in\left[0,\mathcal{R}_{\rm{f}}^{\rm{s}}\right],{\mathcal{R}}^{\rm{c}}\in\left[0,\mathcal{R}_{\rm{f}}^{\rm{c}}\right],\\\
&\kappa\in\left[0,1\right],\mu\in\left[0,1\right]\end{aligned}\right.\right\\},$
(20)

respectively. This subsection has presented the rate regions of NOMA-ISAC and
NOMA-FDSAC. The next part of this section will estimate the relationship
between $\mathcal{C}_{\rm{isac}}$ and $\mathcal{C}_{\rm{fdsac}}$.

###  IV-B Comparison Between NOMA-ISAC and NOMA-FDSAC

Let us introduce some preliminary results that will be useful in discussing
the relationship between $\mathcal{C}_{\rm{isac}}$ and
$\mathcal{C}_{\rm{fdsac}}$.

######  Lemma 1.

Given $\kappa\in\left[0,1\right]$ and $\mu\in\left[0,1\right]$. The sum ECRs
of the CU pair in NOMA-ISAC and NOMA-FDSAC satisfy
$\mathcal{R}_{\rm{i}}^{\rm{c}}\geq\mathcal{R}_{\rm{f}}^{\rm{c}}$, where the
equality holds for $\kappa=\mu=1$.

###### Proof:

The proof is given in Appendix D. ∎

######  Lemma 2.

Given $\kappa\in\left[0,1\right]$ and $\mu\in\left[0,1\right]$. The SRs of the
NOMA-ISAC and NOMA-FDSAC systems satisfy
$\mathcal{R}_{\rm{i}}^{\rm{s}}\geq\mathcal{R}_{\rm{f}}^{\rm{s}}$, where the
equality holds for $\kappa=\mu=0$.

###### Proof:

The proof is given in Appendix D. ∎

Leveraging the above two lemmas, we present the relationship between
$\mathcal{C}_{\rm{isac}}$ and $\mathcal{C}_{\rm{fdsac}}$ as follows.

######  Theorem 4.

The achievable rate regions of NOMA-ISAC and NOMA-FDSAC satisfy
$\mathcal{C}_{\rm{fdsac}}\subseteq\mathcal{C}_{\rm{isac}}$.

###### Proof:

By Lemmas 1 and 2, for any rate pair
$\left({\mathcal{R}}^{\rm{s}},{\mathcal{R}}^{\rm{c}}\right)\in\mathcal{C}_{\rm{fdsac}}$,
it satisfies
${\mathcal{R}}^{\rm{s}}\leq\mathcal{R}_{\rm{f}}^{\rm{s}}\leq\mathcal{R}_{\rm{i}}^{\rm{s}}$
and
${\mathcal{R}}^{\rm{c}}\leq\mathcal{R}_{\rm{f}}^{\rm{c}}\leq\mathcal{R}_{\rm{i}}^{\rm{c}}$,
which yields
$\left({\mathcal{R}}^{\rm{s}},{\mathcal{R}}^{\rm{c}}\right)\in\mathcal{C}_{\rm{isac}}$.
It follows that $\mathcal{C}_{\rm{fdsac}}\subseteq\mathcal{C}_{\rm{isac}}$. ∎

######  Remark 4.

The results in Theorem 4 suggest that the rate region achieved by FDSAC can be
entirely covered by that achieved by ISAC, which highlights the superiority of
ISAC.

##  V Numerical Results

In this section, computer simulations will be employed to demonstrate the
performance of the considered NOMA-ISAC system and also verify the accuracy of
the developed analytical results. The parameters used for simulation are
listed as follows: $M=\mathsf{r}=8$,
$\left\\{\lambda_{a}\right\\}_{a=1}^{\mathsf{r}}=\left\\{5,3,3.5,2.5,1.5,2,1,0.5\right\\}$,
$L=30$, $\varrho_{1}=0.9$, $\varrho_{2}=0.2$, $\alpha_{\rm{F}}=0.8$, and
$\alpha_{\rm{N}}=0.2$.

(a) Outage probability.

(b) Ergodic communication rate.

Figure 2: Performance of communications.
$\overline{\mathcal{R}}_{\rm{N}}=\overline{\mathcal{R}}_{\rm{F}}=0.8$ bps/Hz
and $\kappa=\mu=0.5$. Figure 3: Performance of sensing. $\kappa=\mu=0.5$.

Figure 2(a) and Figure 2(b) plot the OP and ECR versus the SNR $p$,
respectively, where the simulation results (denoted by symbols) match the
analytical results developed at (8) and (12). It is worth pointing out that
the asymptotic results derived at (9) and (13) can track the provided
simulation results in Figure 2(a) and Figure 2(b) accurately in the high-SNR
regime. As can be observed from Figure 2(a), although FDSAC can provide the
same diversity order as ISAC, ISAC yields lower OPs for both NU and FU than
FDSAC. Moreover, it can be seen from Figure 2(b) that ISAC outperforms FDSAC
in terms of NU’s ECR, NU’s high-SNR slope, and FU’s ECR, which is consistent
with the conclusions drawn in Remark 2 and Appendix D. In Figure 3, the
sensing rate is shown as a function of the SNR $p$, where the asymptotic
results are calculated by (16) and (18). As can be seen from this graph, the
asymptotic results track the provided simulation results accurately in the
high-SNR regime. From the data in Figure 3, it is apparent that the ISAC
outperforms FDSAC in terms of both sensing rate and high-SNR slope, which
agrees with the results presented in Remark 3 and Theorem 4. Taken together,
the above numerical studies reveal that NOMA-ISAC is capable of providing
better performance of communications as well as sensing than NOMA-FDSAC.

Figure 4: Rate region comparison. $p=5$ dB.

Figure 4 compares the sensing-communication rate regions achieved by the ISAC
system (presented in (19)) and the baseline FDSAC system (presented in (20)).
In particular, the rate region of the FDSAC system is plotted by changing the
bandwidth allocation factor $\kappa$ as well as the power allocation factor
$\mu$ in (20) from $0$ to $1$. What stands out in this graph is that the rate
region achieved by FDSAC is completely included in that achieved by ISAC,
which, therefore, verifies the correctness of Theorem 4. This, together with
the numerical results in Figure 2 and Figure 3, highlights the superiority of
NOMA-ISAC over the conventional NOMA-FDSAC scheme.

##  VI Conclusion

In this letter, we have analyzed the performance and characterized the
achievable sensing-communication rate region of NOMA-ISAC systems. Simulation
results have verified the accuracy of the developed analytical results.
Theoretical analyses have demonstrated that NOMA-ISAC is capable of providing
more degrees of freedom as well as achieving a larger rate region compared
with NOMA-FDSAC.

##  Appendix A Proof of Theorem 1

The probability density functions (PDFs) of $\left|h_{\rm{N}}\right|^{2}$ and
$\left|h_{\rm{F}}\right|^{2}$ can be written as
$f_{\rm{N}}(x)=\frac{1}{\varrho_{1}}{\rm{e}}^{-\frac{x}{\varrho_{1}}}+\frac{1}{\varrho_{2}}{\rm{e}}^{-\frac{x}{\varrho_{2}}}-\frac{1}{\varrho}{\rm{e}}^{-\frac{x}{\varrho}}$
and $f_{\rm{F}}(x)=\frac{1}{\varrho}{\rm{e}}^{-\frac{x}{\varrho}}$,
respectively. The cumulative distribution functions (CDFs) of
$\left|h_{\rm{N}}\right|^{2}$ and $\left|h_{\rm{F}}\right|^{2}$ can be written
as
$F_{\rm{N}}\left(x\right)=(1-{\rm{e}}^{-{x}/{\varrho_{1}}})(1-{\rm{e}}^{-{x}/{\varrho_{2}}})$
and $F_{\rm{F}}\left(x\right)=1-{\rm{e}}^{-{x}/{\varrho}}$, respectively. Note
that
$\gamma_{t}^{\rm{sic}}=\frac{\mu_{t}p\left|h_{\rm{N}}\right|^{2}\alpha_{\rm{F}}}{\kappa_{t}\sigma_{\rm{c}}^{2}+\mu_{t}p\left|h_{\rm{N}}\right|^{2}\alpha_{\rm{N}}}$.
Hence, when ${\alpha_{\rm{F}}<\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}}$, we
get
$\Pr\left(\gamma_{t}^{\rm{sic}}>\bar{\gamma}_{t}^{\rm{F}}\right)=\Pr\left(\left|h_{\rm{N}}\right|<0\right)=1$,
whereas when ${\alpha_{\rm{F}}=\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}}$, we
get
$\Pr\left(\gamma_{t}^{\rm{sic}}>\bar{\gamma}_{t}^{\rm{F}}\right)=\Pr\left(0>\bar{\gamma}_{t}^{\rm{F}}\right)=1$.
As a result, the OP of NU is given by ${\mathcal{P}}_{t}^{\rm{N}}=1$ when
${\alpha_{\rm{F}}<\bar{\gamma}_{t}^{\rm{F}}\alpha_{\rm{N}}}$. Moreover, by the
definition of ${\mathcal{P}}_{t}^{\rm{N}}$, the OP of NU can be calculated as
${\mathcal{P}}_{t}^{\rm{N}}=\Pr\left(\left|h_{\rm{N}}\right|^{2}>\frac{\kappa_{t}\bar{\gamma}_{t}^{\rm{F}}\sigma_{\rm{c}}^{2}}{\mu_{t}p\varrho_{\rm{N}}^{2}\alpha_{\rm{F}}-\mu_{t}p\varrho_{\rm{N}}^{2}\alpha_{\rm{N}}\bar{\gamma}_{t}^{\rm{F}}},\left|h_{\rm{N}}\right|^{2}>\frac{\kappa_{t}\bar{\gamma}_{t}^{\rm{N}}\sigma_{\rm{c}}^{2}}{\mu_{t}\alpha_{\rm{N}}p\varrho_{\rm{N}}^{2}}\right)$.
Leveraging the PDF of $\left|h_{\rm{N}}\right|^{2}$, we get (8a) after some
basic mathematical manipulations. Then, following similar steps as listed
above, we also get (8b). As $p\rightarrow\infty$, we obtain the asymptotic
results in (9) based on the fact of $\lim_{x\rightarrow
0}\left(1-{\rm{e}}^{-x}\right)\approx x$.

##  Appendix B Proof of Theorem 2

Inserting the obtained PDF expressions into the expressions of the ECRs and
calculating the resultant integrals with the aid of [12, Eq. (4.337.2)], we
obtain the results in (12). By continuously using the fact of
$\lim_{x\rightarrow 0}{\rm{Ei}}(-x)\approx\gamma+\ln{x}$ [12, Eq. (8.214.2)],
we get the approximated results in (13).

##  Appendix C Proof of Theorem 3

Define
$\mathbf{y}={\mathsf{vec}}\left({\mathbf{Y}}_{\rm{s}}^{\mathsf{T}}\right)\in{\mathbbmss{C}}^{LM\times
1}$,
$\mathbf{n}={\mathsf{vec}}\left({\mathbf{N}}_{\rm{s}}^{\mathsf{T}}\right)\sim{\mathcal{CN}}\left(\mathbf{0},\sigma_{\rm{s}}^{2}\mathbf{I}_{LM}\right)$,
and $\mathbf{X}=\mathbf{I}_{M}\otimes\mathbf{x}\in{\mathbbmss{C}}^{LM\times
M}$, where the symbol $\otimes$ represents the Kronecker product. Then, we can
rewrite (4) as $\mathbf{y}=\mathbf{X}\mathbf{g}+\mathbf{n}$. Therefore, the
sensing MI satisfies
$I\left({\mathbf{Y}}_{\rm{s}};{\mathbf{g}}|{\mathbf{x}}\right)=I\left({\mathbf{y}};{\mathbf{g}}|{\mathbf{X}}\right)=\log_{2}\det({\mathbf{I}}_{LM}+{\sigma_{\rm{s}}^{-2}}{\mathbf{X}}\mathbf{R}{\mathbf{X}}^{\mathsf{H}})$,
which together with the Sylvester’s identity, yields
$I\left({\mathbf{Y}}_{\rm{s}};{\mathbf{g}}|{\mathbf{x}}\right)=\log_{2}\det({\mathbf{I}}_{M}+{\sigma_{\rm{s}}^{-2}}\mathbf{R}{\mathbf{X}}^{\mathsf{H}}{\mathbf{X}})$.
We note that
${\mathbf{X}}^{\mathsf{H}}{\mathbf{X}}={\mathbf{x}}^{\mathsf{H}}{\mathbf{x}}{\mathbf{I}}_{M}$.
Based on (2) and (3), we get ${\mathbf{x}}^{\mathsf{H}}{\mathbf{x}}=pL$ and
the results in (15) follow immediately. Using the eigendecomposition of
$\mathbf{R}$, the sensing rate in (15) can be simplified to
$\mathcal{R}_{\rm{i}}^{\rm{s}}=\sum\nolimits_{a=1}^{\mathsf{r}}\log_{2}(1+{pL}{\sigma_{\rm{s}}^{-2}}\lambda_{a})$.
By setting $p\rightarrow\infty$, the results in (16) follow immediately.

##  Appendix D Proof of Lemmas 1 and 2

Before starting the proof, we introduce the following preliminary results that
will be used.

######  Lemma 3.

Given $c\geq 0$. The function $g_{c}(x)=\ln\left(x+c\right)+\frac{c}{x+c}$
increases with $x\in\left[0,\infty\right)$ monotonically.

###### Proof:

The derivative of $g_{c}(x)$ with respect to $x$ is given by
$\frac{{\rm{d}}}{{\rm{d}}x}g_{c}(x)=\frac{x}{(x+c)^{2}}\geq 0$, where the
equality only holds for $x=0$ and $c>0$. Thus, this lemma is proved. ∎

######  Lemma 4.

Given $a>0$ and $b\geq 0$. The function $f(x)=x\ln(1+\frac{a}{x+b})$ increases
with $x\in\left[0,1\right]$ monotonically.

###### Proof:

The derivative of $f(x)$ with respect to $x$ is given by
$\frac{{\rm{d}}}{{\rm{d}}x}f(x)=g_{x}(a+b)-g_{x}(b)$, which together with the
fact of $a+b>b\geq 0$ and Lemma 3, suggests that
$\frac{{\rm{d}}}{{\rm{d}}x}f(x)>0$ holds for $x\geq 0$. Thus, this lemma is
proved. ∎

We now intend to prove Lemmas 1 and 2 with the aid of Lemma 4. Particularly,
using Lemma 4 and the fact of $\kappa\in[0,1]$, we can get
$\kappa\log_{2}(1+\frac{y_{1}}{\kappa+y_{2}})\leq\log_{2}(1+\frac{y_{1}}{1+y_{2}})$
($y_{1}>0$ and $y_{2}\geq 0$), where the equality holds for $\kappa=1$.
Moreover, using the monotonicity of $q(t)=\frac{y_{1}t}{1+y_{2}t}$ for $t\geq
0$ and the fact of $\mu\in[0,1]$, we obtain
$\log_{2}(1+\frac{y_{1}}{1+y_{2}})\leq\log_{2}(1+\frac{y_{1}/\mu}{1+y_{2}/\mu})$,
where the equality holds for $\mu=1$. Taken together, we get
$\kappa\log_{2}(1+\frac{y_{1}}{\kappa+y_{2}})\leq\log_{2}(1+\frac{y_{1}/\mu}{1+y_{2}/\mu})$.
By setting $y_{1}={\mu
p\left|h_{\rm{F}}\right|^{2}\alpha_{\rm{F}}}/{\sigma_{\rm{c}}^{2}}$ and
$y_{2}={\mu
p\left|h_{\rm{F}}\right|^{2}\alpha_{\rm{N}}}/{\sigma_{\rm{c}}^{2}}$ and taking
the expectation with respect to $\left|h_{\rm{F}}\right|^{2}$ on each side of
this inequality, we have
$\mathcal{R}_{\rm{f}}^{{\rm{F}}}\leq\mathcal{R}_{\rm{i}}^{{\rm{F}}}$. Besides,
by setting $y_{1}={\mu
p\left|h_{\rm{N}}\right|^{2}\alpha_{\rm{N}}}/{\sigma_{\rm{c}}^{2}}$ and
$y_{2}=0$, we have
$\mathcal{R}_{\rm{f}}^{{\rm{N}}}\leq\mathcal{R}_{\rm{i}}^{{\rm{N}}}$. In
summary, we obtain
$\mathcal{R}_{\rm{i}}^{\rm{c}}=\mathcal{R}_{\rm{i}}^{\rm{N}}+\mathcal{R}_{\rm{i}}^{\rm{F}}\geq\mathcal{R}_{\rm{f}}^{\rm{N}}+\mathcal{R}_{\rm{f}}^{\rm{F}}=\mathcal{R}_{\rm{f}}^{\rm{c}}$,
and thus Lemma 1 is proved. Following the way we obtain
$\mathcal{R}_{\rm{f}}^{{\rm{N}}}\leq\mathcal{R}_{\rm{i}}^{{\rm{N}}}$, we can
get $\mathcal{R}_{\rm{i}}^{\rm{s}}\geq\mathcal{R}_{\rm{f}}^{\rm{s}}$. Thus,
Lemma 2 is proved.

## References

  * [1] F. Liu, _et al._ , “Integrated sensing and communications: Towards dual-functional wireless networks for 6G and beyond,” _IEEE J. Sel. Areas Commun._ , vol. 40, no. 6, pp. 1728–1767, Jun. 2022.
  * [2] Z. Wang, _et al._ , “NOMA empowered integrated sensing and communication,” _IEEE Commun. Lett._ , vol. 26, no. 3, pp. 677–681, Mar. 2022.
  * [3] X. Mu, _et al._ , “NOMA-aided joint radar and multicast-unicast communication systems,” _IEEE J. Sel. Areas Commun._ , vol. 40, no. 6, pp. 1978–1992, Jun. 2022.
  * [4] F. Liu, _et al._ , “Cramér-rao bound optimization for joint radar-communication beamforming,” _IEEE Trans. Signal Process._ , vol. 70, pp. 240–253, 2022.
  * [5] A. Liu, _et al._ , “A survey on fundamental limits of integrated sensing and communication,” _IEEE Commun. Surveys Tuts._ , Early Access, 2022.
  * [6] A. R. Chiriyath, B. Paul, G. M. Jacyna, and D. W. Bliss, “Inner bounds on performance of radar and communications co-existence,” _IEEE Trans. Signal Process._ , vol. 64, no. 2, pp. 464–474, Jan. 2016.
  * [7] Y. Rong, _et al._ , “MIMO radar and communications spectrum sharing: A multiple-access perspective,” in _Proc. IEEE 10th Sensor Array Multichannel Signal Process. Workshop_ , 2018, pp. 272–276.
  * [8] X. Yuan, _et al._ , “Spatio-temporal power optimization for MIMO joint communication and radio sensing systems with training overhead,” _IEEE Trans. Veh. Technol._ , vol. 70, no. 1, pp. 514–528, Jan. 2021.
  * [9] Y. Liu, _et al._ , “Non-orthogonal multiple access for 5G and beyond,” _Proc. IEEE_ , vol. 105, no. 12, pp. 2347–2381, Dec. 2017.
  * [10] Y. Liu, _et al._ , “Evolution of NOMA toward next generation multiple access (NGMA) for 6G,” _IEEE J. Sel. Areas Commun._ , vol. 40, no. 4, pp. 1037–1071, Apr. 2022.
  * [11] B. Tang and J. Li, “Spectrally constrained MIMO radar waveform design based on mutual information,” _IEEE Trans. Signal Process._ , vol. 67, no. 3, pp. 821–834, Feb. 2019.
  * [12] I. S. Gradshteyn and I. M. Ryzhik, _Table of Integrals, Series and Products_ , 7th ed., Academic, San Diego, C.A., 2007.


# A Panoramic Survey of
Natural Language Processing in the Arab World

Kareem Darwish, Nizar Habash,
Mourad Abbas, Hend Al-Khalifa, Samhaa R. El-Beltagy,
Houda Bouamor,Karim Bouzoubaa, Violetta Cavalli-Sforza,
Wassim El-Hajj, Mustafa Jarrar,
Huseein T. Al-Natsheh, and Hamdy Mubarak

(September 2020)

##  1 Introduction

##  2 What is Natural Language Processing?

Written language could be defined as a thought communication script for human
beings. It has been a way to store knowledge and information for millenniums
[DBLP:phd/hal/AlNatsheh19]. People can naturally understand and “decode” back
the written language into thoughts and meanings. However, natural language is
considered to be very complex for a machine with classical computations to
process and understand. The science that studies the computational methods and
syntactical approaches to automatically process and semantically understand
the language is what the community defines as Natural Language Processing
(NLP). Automating language processing and understanding is very important for
many applications like search engines, dialogue systems, recommender systems,
question-answering systems, and big data analytics.

There are many overlapping scientific disciplines that actively contribute to
NLP, i.e, computational linguistics, machine learning, conversational AI,
natural language understanding (NLU), text representation, voice recognition,
cognitive science, and logic and symbolic learning [DBLP:phd/hal/AlNatsheh19].
A sample of NLP main tasks are; Named Entity Recognition (NER), morphological
segmentation, Part of Speech (POS), syntactical parsing, and text
preprocessing and normalization like tokenization and numeration. Recently
with the big advancements in machine learning, most of the state-of-the-art
NLP techniques are based on unsupervised machine learning text embedding, or
dense vector representation, that encodes not only syntactic but also
contextual semantics information [DBLP:conf/naacl/PagliardiniGJ18,
DBLP:conf/naacl/DevlinCLT19].

##  3 A Short History of NLP in the Arab World

Arabic is one of the most widely used languages in the world. It is one of the
six official languages in the United Nations and also the language of the Holy
Quran, thus it is not only the language of people in Arab countries but also
it is the second language for millions of Muslims around the world.

Historically, we can say that Arabic natural language processing went through
three waves, the first wave was in the early 1980’s with the introduction of
Microsoft MS-DOS 3.3 with Arabic language support [rippin1991arabic]. In 1985
the first Arabic morphological analyzer was developed by Sakhr company. During
the same year there was a symposium in Kuwait for Arabic and computers, where
most of the presented research focused on morphological analysis of Arabic
text and the use of rule-based approach in Arabic NLP. Sakhr has also
continued leading research and development in Arabic computational linguistics
by developing the first syntactic and semantic analyzer in 1992 and Arabic
optical character recognition in 1995. Sakhr has also produced many commercial
products and solutions including Arabic to English machine translation, Arabic
text-to-speech (TTS), and Arabic search engine.

The second wave was during the years 2000-2010. Arabic NLP has gained
increasing importance in the Western world to enable non-Arabic speakers make
sense of Arabic text especially after September 11. The need to identify
Arabic names at airport security and analyze Arabic documents on the spot
became very essential; thus, countries such as the USA went funding companies
and research centers to develop Arabic NLP tools including: machine
translation, speech synthesis and recognition, information retrieval and
extraction, text to speech and named entity recognition [farghaly2009arabic].
Most of the developed systems in that era used machine learning because it
does not require deep linguistic knowledge, and it is fast and inexpensive
[farghaly2009arabic]. The leading universities, companies and consortium at
the time were: Columbia University, University of Maryland, IBM, and LDC.
Also, the MEDAR consortium was established 2008 for cooperation between Arabic
and European Union countries for developing Arabic language resources.

The third wave started 2010, when the research focus on Arabic NLP has came
back to the Arab world and it witnessed the proliferation of Arab researchers
and graduate students interested in ANLP. Universities such as: NYAUD (UAE),
AUB (Lebanon), KSU (SA), Cairo University (Egypt), etc. and research centers
such as: QCRI (Qatar), KACST (SA), etc. have shown increase publications in
this area. There research has focused more on Arabic dialects and social media
and the used techniques have developed to neural and deep learning approaches.
We also witnessed increase in Arabic language resources and pre-processing
tools.

##  4 Arabic NLP Challenges

Arabic has its own characteristics. These characteristics cause many
difficulties and ambiguities to process it. The remarkable difference between
Arabic and remaining languages, particularly Latin ones, can be at first
glance noticed through its RTL writing system and its letters that change
shape according to their position in the word. This makes Arabic OCR
challenges more serious. Arabic is a pro-drop language which allows subject
pronouns to drop [farghalyshaalan]. Arabic is also an agglutinative language,
having a rich and non concatenative morphology. Most of texts don’t contain
diacritics which increases lexical ambiguity. Contrary to Latin languages,
There is no capitalization in Arabic, one can imagine how difficult is to
identify named entities without capital letters.
In addition, Arabic is a VSO language but it has the ability to switch to the
five remaining orders: VOS, SVO, SOV, OVS, OSV, which is not the case for many
other languages. This can be a source of additional issues for Arabic NLP,
knowing that the majority of Arabic textual content does not contain
diacritics.
One of the most serious problems that face Arabic processing is the lack of
significant annotated corpora for different NLP tasks, as lemmatization,
diacritization, transliteration of named entities, etc.
The emerging of dialects in the social media due to the advent of the Internet
and the new technologies allowed an important slice of people, particularly
those who have low intellectual levels, to write texts in their own dialects
instead of MSA, which highly contributed in increasing dialectal textual
resources, but without respecting orthographic and grammar rules. This can be
considred as an additional serious issue that face Arabic NLP researchers.

##  5 Arabic Tools and Resources

We would like people to write about different topics (individually or in small
groups).

  * •

what is the task?

  * •

why is it important?

  * •

why is it challenging for Arabic?

  * •

what tools/sota right now?

  * •

what are future directions?

This section can be broken into its three main subsections.

###  5.1 Arabic Data Resources

####  5.1.1 Introduction

This section presents Arabic NLP resources, focusing on the most relevant and
commonly used ones. While NLP tools are software tools, resources include
linguistic data in an electronic and efficiently computer-readable format.
These resources, required for the preparation and execution of NLP tools, are
roughly divided into those whose unit of organization is the word (or multi-
word expression) and those whose unit of organization is the sentence or
sentence fragment. The former category includes dictionaries, thesauri,
wordnets and ontologies, and, for speech applications, pronunciation
dictionaries. They are normally constructed manually or semi-automatically and
revised by language experts. The latter contain large, sometimes very large
amounts of text, obtained from various sources and can be either raw, with no
more processing that division into sentence or sentence fragments, or
annotated in several different ways.

Especially these days, when so much of NLP practice is based on tools trained
using machine learning algorithms and extensive data resources, the
performance of NLP applications directly depends on the quality of the
computational resources used to construct them. High quality resources are
therefore a key for NLP improvement. Frequently however, they do not receive
sufficient attention or support because resource construction is a lengthy and
costly task that requires significant teamwork among linguists,
lexicographers, database producers, and publishers over an extended period of
time. While many new digital data resources have been constructed for Arabic
in the last two decades, construction has frequently been driven by an agenda
that mostly ignored certain applications to the benefit of others. It is worth
remarking that, although resource construction is now well underway the Arab
world itself in both the commercial and sectors, much of the work on the
development of Arabic resources took place in the United States, in Europe
and, more recently, through collaborations between organizations located in
those countries and partner institutions in the MENA region. At present, many
data resources for Arabic can be found in known international repositories
(namely ELRA/ELDA, LDC, and CLARIN), from which they can be licensed at costs
that vary depending on the origin of the resource and its intended use. Open
resources are released directly from their authors’ websites.

While perhaps there does not appear to be a shortage in Arabic resources, a
2014 survey by Zaghouani of freely available Arabic corpora
[zaghouani2014critical] open indicated that there were significantly fewer
than one hundred. A small additional number has been constructed and released
since that time or is still under construction. This number gives a somewhat
misleading picture of the landscape of Arabic NLP resources because, of the
ones that do exist, many are not interoperable, having been built using
different tools and assumptions, few are comprehensive. Moreover, many
resources are proprietary and only available at prohibitively high prices.
Serious, well-planned, and well-coordinated investment in resources will be
instrumental for the future of Arabic NLP.

####  5.1.2 Arabic lexicons, wordnets, and ontologies

Although there are many Arabic lexicons in paper format, covering most domains
and aspects of the language, few resources are available in digital form or
usable in NLP tasks. A recent initiative to build a large Arabic lexicographic
database was proposed in [JA19], which includes 150 Arabic-multilingual
lexicons that were digitized, restructured, normalized and stored in one
database. This database is accessible through a set of RESTFULL web services
(API) as well as through a lexicographic search engine [ADJ19]. One of the
early stem databases was developed to support the BAMA morphological analyzer
[Buckwalter2002]. An extended version of BAMA, called SAMA, includes more
stems, tags, and solutions [Graff09]. There are also other initiatives to
represent Arabic lexicons in international standard formats, such as the W3C
lemon model [JAM19, khalfi16] and ISO’s LMF [aida16, sou14]. In addition,
[namly20] developed one of the most comprehensive lexicons with more than 7M
stems and corresponding lemmas.

Arabic WordNet [awn06] is a semantic lexicon consisting of about 11k synsets,
created by manually translating a small subset of Princeton WordNet, the
Common Base Concepts. These are common concepts in several wordnets, thus they
are assumed to also exist in Arabic. This initial set was extended downwards
with more specific concepts, mostly, and upwards with more general concepts,
to improve the maximal connectivity of those base concepts. Later, it was
enriched with more verbs, plurals, and thousands of named entities
[abouenour2013evaluation]. An Arabic VerbNet [Mousser2013] was also developed
to classify verbs that have the same syntactic descriptions and argument
structure.

The Arabic Ontology [jarrarAraOnto] is a formal Arabic wordnet that has the
same the structure as WordNet, but its content was built with ontological
analysis in mind. It was benchmarked, as much as possible, against scientific
and rigorous knowledge sources, rather than being based just on speakers’
beliefs, as lexicons and wordnets typically are. The ontology consists
currently of about 1.5k well-investigated concepts, in addition to about 20k
concepts that are partially validated but usable for NLP tasks. In addition,
the ontology is mapped to a large set of lexical concepts in the Arabic
Lexicographic Database [JAM19], and fully mapped to Princeton WordNet,
Wikidata, and other resources.

The above-mentioned lexicons and semantic resources have targeted exclusively
MSA. Work on comparable resources for dialectal Arabic, and particularly for
the Maghrebi (western) dialects of Morocco, Algeria, and Tunisia, has been
sparser and very recent. Maltese Arabic, though closely related to Tunisian,
is frequently not considered a dialect of Arabic because of the heavy
influence of both Romance languages (through Italian) and English.

Construction of lexicons for various dialects of Arabic has followed different
approaches. As a multi-dialectal lexicon, the MADAR lexicon [Bouamor-
etal-2018] is a collection of 1,045 concepts extracted from the MADAR Corpus
(mentioned below) defined in terms of triplets of words and phrases from
English, French and MSA, along with multiple equivalent dialectal forms
covering 25 cities from the Arab world. For the Algerian dialect, authors of
[abidi2018automatic] developed an iterative multilingual word embeddings
approach, which allowed to construct an Algerian dialect corpus, from which an
Algerian lexicon of 6947 entries was built. Each of its words is associated to
its different transliterations. Since the dialect is evolving every day, one
of the advantages of this approach is that the lexicon can be updated easily
by harvesting new data. For Tunisian, a dialect to MSA bilingual lexicon
[sghaier2017tunisian] containing almost 14K of manually approved words was
developed from different sources. Data was collected from social networks,
blogs, and forums.

A different approach was used to develop the Moroccan Dialect Electronic
Dictionary (MDED) containing 12K entries [tachicart2014building]. One major
MDED feature is its annotation111http://arabic.emi.ac.ma:8080/mded/ with
useful metadata such as the MSA translation, the POS, the origin (MSA, French,
Amazigh, Spanish or English) and the dialectal root. The lexicon has been
semi-automatically developed with a manual linguistic validation. First,
Almaknaz MSA dictionary was translated to Moroccan and then the Moroccan old
Colin [zakia1994dictionnaire] dictionary was translated to MSA. Lastly, the
integration of the two translations was augmented to include more current
Moroccan words from forums and from speech used in daily conversations between
young Moroccan people.

There are also two relatively recent efforts to create WordNets for Arabic
dialects. An Iraqi Wordnet [c-s2013aiccsa] was bootstrapped from Arabic
WordNet, assuming that the node and linkage structure could start out being
the same for the two WordNets and automatically proposing Iraqi words for
synsets with the aid an Iraqi Arabic-English bidirectional dictionary. Where
no candidate words could be found, the MSA word was left as an option, to be
accepted or rejected (as all proposals) by an Iraqi native speaker. A very
different approach was followed to create TunDiaWN, a WordNet for the Tunisian
dialect [bouchlaghem-etal-2014-tunisian]. The authors used a corpus-based
method to extract from a dialectal corpus they collected and used clustering
techniques to group them, leveraging both human expertise and English WordNet
and Arabic WordNet structures. Finally, it also appears that some work has
been done towards the creation of an Egyptian Arabic
Wordnet222https://catalog.ldc.upenn.edu/LDC2018T23.

####  5.1.3 Corpora

According to EAGLES (the Expert Advisory Group on Language Engineering
Standards) [sinclair2004intuition], a corpus is a collection of naturally
occurring samples of a language. These samples are selected and stored in
electronic format. It is worth mentioning that, historically, Al-Khalil bnu
Ahmed Al-Farahidi, was among the first to assemble a large collection of texts
in the 8th century [versteegh2014arabic].

Over the past few years, Arabic corpus building has seen a tremendous growth
in interest and activity. Extensive lists of available corpora can be found
in, [habash2010book], [Zaghouani2017CriticalSO], and [zeroual2018thesis]. Some
corpora were built for general use, while others were designed and built for
specific purposes, such as language learning, Quranic studies, spellchecking,
and others. We briefly outline below different types of corpora and provide
well-known examples of each, distinguishing between monolingual and
multilingual corpora and covering different kinds of Arabic, namely Classical
Arabic (CA), Modern Standard Arabic (MSA), and varieties of Dialectal Arabic
(DA). Corpora developed specifically for pedagogical or closely related
applications will be discussed in a later section.

CA corpora are fundamentally monolingual. The Al-Mus’haf corpus [zeroual2016]
is considered both a classical and a Quranic corpus, and includes morpho-
syntactic annotations such as stem, stem’s pattern, lemma, lemma pattern, and
root. Shamela is another large-scale CA corpus (1B words) covering the past 14
centuries [belinkov16]. The Tashkeela corpus [zerrouki17] was extracted from
Shamela, and contains about 75M fully vocalized words.

Many corpus-building efforts have targeted MSA, the language of the Arabic
media. MSA monolingual corpora are probably the most common and easiest to
construct, by collecting text from online news sites, newswire and other
sources across different Arab countries. The earliest of the very large Arabic
raw (non-annotated) monolingual corpora was the ArabicGigaword corpus
[parker11], initially produced by
LDC333https://catalog.ldc.upenn.edu/LDC2003T12 in 2003 and last revised in
2011444https://catalog.ldc.upenn.edu/LDC2011T11. The International Corpus of
Arabic was developed by Bibliotheca Alexandrina and contains 100 million words
that were collected from the press, net articles, books, and academic text
sources [alansary07]. Many more and freely available corpora have been
developed since then, as general-purpose corpora or destined for specific
tasks, but often usable beyond their original purpose. A recent listing and
description of such corpora is provided by [zeroual2018thesis].

A special category of monolingual corpora are treebanks, containing text that
has been syntactically annotated according to specific grammatical theories.
For Arabic, different treebanks have been developed. The Penn Arabic Treebank
(PATB) [Maamouri2004ThePA, maamouri2010] is an MSA corpus and consists of
about 1.6M words of morphologically and syntactically annotated data (mostly
newswire). The annotation scheme was modeled on the Penn Treebank for English,
while taking into account specificities of Arabic syntax. In the Prague Arabic
Dependency Treebank555http://ufal.mff.cuni.cz/padt/PADT_1.0/docs/index.html
data similar to the PATB data is annotated using a dependency grammar
theoretical framework [smrz08]. The Columbia Arabic Treebank [habash-
roth-2009-catib] sought to create a simpler resource, sacrificing some
linguistic richness for the sake of speed of annotation and using
representations and terminology inspired by traditional Arabic syntax.

Multilingual corpora represent an important category of resources that are
particularly useful for machine translation, but not only. MSA has been
included in relevant pioneer multilingual corpora such as the open source
parallel corpus (OPUS) [tiedemann2012parallel], the largest collection of
freely available parallel corpora in more than 90 languages, comprising over
40 billion tokens, and including data from several domains. OPUS draws
primarily on legislative and administrative texts, translated movie subtitles,
newspapers, and localization data from open-source software projects. One
other widely-known resource is the MultiUN corpus (Multilingual Corpus from
United Nation) [chen2012multiun], available in all six official languages of
the UN plus German, and comprising 271.5 million Arabic tokens. The MulTed
corpus [zeroual2020multed] is a multilingual and PoS tagged parallel corpus
with bilingual sentence-alignment considering English as a pivot language. The
corpus has been derived from the TEDx talks and contains subtitles that cover
1,100 talks, including a variety of domains and topics.

Dialectal corpus construction has also received some attention recently. MADAR
is a multi-dialect corpus in the travel domain covering 25 Arabic cities
[bouamor18]. PADIC [meftouh2015machine] is another one composed of 6400
sentences in five dialects from both the Maghreb and the Middle-East (Syria
and Palestine) that have been aligned with MSA. The Shami corpus
[kwaik2018shami], constructed using a combination of manual and automatic
methods, specializes on the four Levantine dialects with 117K sentences.
Curras [jarrar2017curras] is a general-purpose annotated corpus for the
Palestinian Levant dialect, consisting of about 60K tokens; it was manually
annotated with about 16 morphological features and linked with MSA using LDC
tags. An Emirati Arabic corpus, called Gumar, was collected from Emirati
novels and consists of about 200k words [khalifa18]. A monolingual dialectal
corpus of Moroccan Arabic was developed and collected from different sources
such as social media, TV scripts, and recorded conversations between Moroccan
people [tachicart2019empirical]. It was manually validated by 30 native
speakers’ annotators, resulting in a corpus containing 34k Moroccan dialectal
sentences666http://arabic.emi.ac.ma:8080/MCAP/faces/corpus_tei.xhtml.

Finally, it should be mentioned that there are also speech-oriented corpora,
such as the CALLHOME Egyptian corpus [canavan97] and the Orientel corpus
[siemund02], which covers several dialects.

###  5.2 Arabic Enabling Technologies

Proposed areas:

####  5.2.1 Morphological Processing

Tokenization, analysis, disambiguation, diacritization

####  5.2.2 Syntactic Processing

Part of Speech Tagging (POS) is the process of assigning parts of speech, such
as noun, verb, or adjective, to each word (or token) given its context within
a sentence. POS tagging is often required for downstream NLP tasks such as
parsing. There are three approaches typically used for part of speech tagging,
namely: (1) rule-based; (2) statistical; and (3) deep learning.

There are several Arabic POS tag sets that have been developed for different
purposes based upon the tagging objective [alqrainy2020towards]. Some of the
most widely known tag sets include the Penn Arabic Treebank (PATB), CATiB and
Khoja tag sets [habash2010book]. There are several Arabic POS taggers from
commercial companies, such as Sakhr and RDI, and research labs and
universities such as Farasa, Stanford, and MADAMIRA [darwish2017arabic].

Syntactic parsing is the process of generating a parse tree for a sentence.
Parse trees have two main representations, namely constituency and dependency
trees. The constituency-based syntactic trees, also known as a phrase-
structure trees, partition sentences into sub-constituents (phrases), where
non-terminals are constituents (ex. prepositional phrase) and terminals are
words. As for dependency trees, a sentence is represented in a tree with an
independent node as a root and each of the other words depends on exactly one
word as its syntactic head. The arcs between the head and the dependents have
relation or function labels. For example, in the sentence “he left”, the head
is the verb “left” and “he” is dependent on the head with a “subject of”
relationship. The most popular Arabic syntactic parsers for Arabic are:
Stanford, Farasa, and Camel. Stanford is a statistical parser from the
Stanford Natural Language Processing Group that can parse English, German,
Arabic and Chinese. For Arabic, it uses a probabilistic context free grammar
that was developed based on pATB [Green:2010:better]. Updated versions of the
parser use a shift-reduce algorithm [zhu2013fast] and neural transition-based
dependency parsing [chen2014fast]. Farasa is an Arabic NLP toolkit that
provides syntactic constituency and dependency parsing [zhang2015randomized].
Similar to Stanford, Farasa was tuned for the news domain in MSA. Farasa POS
tagger uses 16 tags of PATB simplified tags set [darwish2017arabic]. Camel is
a dependency parser trained on CATiB using MaltParser, which is a language-
independent and data-driven dependency parser [Shahrour:2016:camelparser].

####  5.2.3 Named Entity Recognition

Named Entity recognition(NER) is the task of identifying one or more
consecutive words in text that refer to objects that exist in the real-world
(named entities), such as organizations, persons, locations, brands, products,
foods, etc. Named entity recognition is essential for extracting structured
data from an unstructured text, relationship extraction, ontology population,
classification, machine translation, question answering, and other
applications. Unlike English, where named entities, such as person names, are
capitalized, Arabic lacks casing, which complicates Arabic NER. Further,
Arabic named entities often correspond to Arabic adjectives or nouns. For
example, the Arabic names ’Jameela’ and ’Adel’ are also the common adjectives
‘beautiful’ and ‘just’. This inherent ambiguity means that matching entries in
named entity gazetteers is highly error prone. A comprehensive list of other
challenges can be found in [shalaan1]. Traditionally, Arabic NER approaches
included the use of hand-crafted heuristics, machine learning, and a hybrid of
both [shalaan1] with heavy reliance on gazetteers [benajiba2008arabic]. Recent
approaches exploited cross-lingual links between Arabic and English knowledge
bases to expand Arabic gazetteers and to carry over capitalization from
English to Arabic [darwish2013named]. Much of the earlier work on Arabic NER
focused on formal text, typically written in MSA, and applying models that are
trained on MSA on social media text, which becoming more ubiquitous and much
of which is dialectical, led to unsatisfactory results [darwish2014simple].
The aforementioned Arabic NER challenges suggest that understanding the
context in which a word appears can address linguistic ambiguity and
significantly improve NER. The recent rise to of contextualized embeddings,
such as BERT [bert], revealed promising possibilities for improving Arabic
NER. At the time of writing this article, a handful of contextualized
embedding models are known to support Arabic including BERT
Multilingual[bert], Arabic BERT[safaya2020kuisail],
AraBERT[antoun2020arabert], and more recently GigaBert [lan2020gigabert].
However, none of these models has wide Arabic dialect coverage. Other deep
learning approaches such as sequence to sequence models and convolutional
neural networks have led to improved results for Arabic NER [ali2019boosting,
DBLP:journals/csl/KhalifaS19, liu-etal-2019-arabic]. It is expected that the
use of contextualized embeddings trained on larger corpora of varying Arabic
dialects, coupled with the use of deep learning models is likely to contribute
positively to Arabic NER.

####  5.2.4 Text classification

Arabic dialect identification (DID) is the task of automatically identifying
the dialect of a particular segment of speech or text of any size (i.e., word,
sentence, or document). This task has attracted increasing attention in the
NLP research field. For instance, several evaluation campaigns were dedicated
to discriminating between language varieties [malmasi-EtAl:2016:VarDial3,
zampieri-EtAl:2017:VarDial, zampieri-EtAl:2018:VarDial]. This is not
surprising considering the importance of automatic DID for several NLP tasks,
where prior knowledge about the dialect of an input text can be helpful for
applications, such as machine translation [salloum-EtAl:2014:P14-2], sentiment
analysis [altwairesh-alkhalifa-alsalman:2016:P16-1], or author profiling
[sadat-kazemi-farzindar:2014:SocialNLP].

A majority of Dialectal Arabic computational efforts have targeted creating
resources for country or regionally specific dialects [Gadalla:1997:callhome,
diab2010colaba, al2012yadac, Salama:2014:youdacc, sadat-kazemi-
farzindar:2014:SocialNLP, Smaili:2014:building, Jarrar:2016:curras,
Khalifa:2016:large, Al-Twairesh:2018:suar, el2020habibi]. The expansion into
multi-dialectal data sets and models to identify them was initially done at
the regional level [Biadsy:2009:using, zaidan2011arabic, Elfardy:2014:aida,
Bouamor:2014:multidialectal, darwish-sajjad-mubarak:2014:EMNLP2014,
Meftouh:2015:machine, 7982330].

The Multi Arabic Dialects Application and Resources (MADAR) project aimed to
create finer grained dialectal corpus and lexicon [Bouamor:2018:madar]. The
data was used for dialectal identification at the city level
[Salameh:2018:fine-grained, obeid-etal-2019-adida] of 25 Arab cities. The main
issue with that data is that it was commissioned and not naturally occurring.
Concurrently, larger Twitter-based datasets covering 10-21 countries were also
introduced [abdelali2020arabic, Abdul-Mageed:2018:you, Mubarak:2014:using,
Zaghouani:2018:araptweet]. The Nuanced Arabic Dialect Identification (NADI)
Shared Task [mageed-etal-2020-nadi] followed earlier pioneering works by
providing country-level dialect data for 21 Arab countries, and introduced a
province-level identification task aiming at exploring a total of 100
provinces across these countries.

####  5.2.5 Infrastructures

Arabic NLP has made significant progress with the development of several tools
and resources, such as POS taggers, morphological analyzers, text classifiers,
and parsers. However, there is a limited number of homogeneous and flexible
Arabic toolkits that gathers these components.

The need for such toolkits led to the development of multi-lingual
infrastructures such as GATE777https://gate.ac.uk, Stanford CoreNLP
888https://stanfordnlp.github.io/CoreNLP/ and
UIMA999https://uima.apache.org/d/uimaj-current/index.html. Such offer
researchers easy access to several tools, eliminating the need to develop them
from scratch every time. In the case of Arabic, some available infrastructures
include, among others, ATKS101010https://www.microsoft.com/en-
us/research/project/arabic-toolkit-service-atks/, AraNLP
[althobaiti2014aranlp], MADAMIRA [pasha2014madamira], Farasa, and
ArabiTools111111https://www.arabitools.com/.

CAMeL Tools121212https://github.com/CAMeL-Lab, is a collection of open-source
tools, developed in Python, that supports both MSA and Arabic dialects. It
currently provides utilities for pre-processing, morphological modeling,
dialect identification, named entity recognition and sentiment analysis. CAMeL
Tools provides command-line interfaces (CLIs) and application programming
interfaces (APIs) covering these utilities.

Farasa131313http://qatsdemo.cloudapp.net/farasa/ is a collection of Java
libraries and CLIs for MSA. These include separate tools for diacritization,
segmentation, POS tagging, parsing, and NER. This allows for a more flexible
use of components compared to MADAMIRA.

SAFAR141414http://arabic.emi.ac.ma/safar/ is a Java-based framework bringing
together all layers of ANLP: resources, pre-processing, morphology, syntax and
semantics. The general idea behind SAFAR is to gather, within a single
homogeneous infrastructure, the already developed and available Arabic tools.
In addition to facilitating access to such tools, the platform enables
comparing tools’ performance. All tools within SAFAR are standardized
according to several Java interfaces, therefore users can add new
implementations and also easily create customized pipelines.

A feature comparison of some Arabic infrastructures can be found in
[obeid2020camel] while a more detailed survey and a software engineering
comparative study can be found in [jaafar2018survey]. Whatever the specific
choice a researcher might make, the trend to use infrastructures more
extensively will significantly save time and change the way Arabic
applications are designed and developed.

###  5.3 Arabic Applications

Proposed areas:

####  5.3.1 Sentiment and Emotion Analysis

Sentiment analysis is a very powerful tool to use when trying to understand
customer satisfaction, carrying out competition analysis, and more generally
when gauging public opinion towards a specific issue, topic, product, ad,
decision, etc. Research in this area has attracted much attention in the
Arabic research community during the last decade, and the applications of this
work have been valuable to a multitude of stakeholders from governments to
companies and public figures. Rise in the interest in this research area came
with the almost explosive increase of Arabic social media usage which in turn
resulted in the availability of large volumes of opinionated and sentiment
reflecting data. When work on this area first started, Arabic was considered
as an under resourced language with many resources needed for carrying out
sentiment analysis, such as sentiment lexicons, large training datasets, and
sentiment treebanks. With time, these resources were constructed [Abdul-
Mageed2012, baly2017sentiment, Badaro2014, Eskander2015, El-Beltagy2016,
ElSahar2015, Refaee2014, salameh-mohammad-kiritchenko:2015:NAACL-HLT,
Shoukry2015, Khalil2015, Mourad2013], and SemEval conducted shared tasks that
included Arabic sentiment analysis in 2017 [SemEval:2017:task4] and its affect
(emotion) detection in 2018 [SemEval2018Task1]. All such efforts contributed
to resources and benchmarks for the task. Early approaches for Arabic
sentiment analysis explored the use of rules [El-Beltagy2013], but it quickly
became apparent that machine learning approaches are better suited for this
task, and the focus shifted to the best pre-preprocessing steps and features
to use to produce the best results. The success of deep learning approach on
English sentiment analysis and the prevalence of the use of word embedding
models such as word2vec [Mikolov2013] and GloVe [Pennington2014], quickly had
researchers doing work on Arabic sentiment analysis investigate similar models
for Arabic [baly2017sentiment, al2015deep]. The efforts were initially held
back due to the lack of Arabic pre-trained word embeddings, but this again was
quickly mitigated by the appearance of several models such as AraVec
[Soliman2017] and Mazajak [farha2019mazajak]. This new direction, resulted in
improved performance for the task with little or no effort being directed to
feature engineering. Another shift is currently in the making with the
appearance of Arabic contextualized embeddings [antoun2020arabert,
safaya2020kuisail, lan2020gigabert]. Sentiment analysis gave rise to Aspect
Based Sentiment Analysis which takes into account the terms related to the
aspects and identifies the sentiment associated with each aspect
[al2019enhancing, areed2020aspect, trigui2020hybrid]. Although a lot of
progress has been achieved in the area of sentiment analysis in Arabic, there
remains a lot to be done for dialectal Arabic. Also there is still a need to
develop large scale Arabic annotated data to enable training of robust
sentiment classification models. [badaro2019survey] presents a comprehensive
survey on the status of sentiment analysis in Arabic and the future
directions.

A closely related topic that recently attracted the attention of researchers
is Emotion Recognition in Arabic [alswaidan2020hybrid, baly2019arsentd,
badaro2018arsel, hifny2019efficient, shahin2019emotion]. Similar to how
sentiment analysis resources and models started maturing, a lot of work still
needs to be done in emotion recognition.

####  5.3.2 Machine Translation

NOTE: need to say something on work on Arabic sign language:
[al2001recognition, luqman2019automatic]

One of the earliest works on Arabic Machine translation has been addressed in
1991 in [ibrahim1991MA] where the authors presented the problem of translating
the embedded idioms and proverb expressions from English into Arabic. In 1992,
the problem of aligning sentences in bilingual texts for the two couples of
languages French–English and French–Arabic has been tackled in
[debili1992aligning]. In the same year, an English-Arabic MT system has been
developed for translating political news sentences [rafea1992mutargem]. In
1995, a machine translation system from Arabic to English and French has been
implemented [mankai1995machine] based on Fillmore’s case frames that the
authors found to be well adapted to the analysis of Arabic. For the sake of
facilitating the access to the English web content for Arab people, the
authors of [guessoum2001methodology] introduced in 2001 a system that
translates web pages from English to Arabic using a commercial machine
translation system. The same authors presented in [guessoum2004methodology] a
statistical methodology for evaluating four English–Arabic commercial (MT)
systems, based on lexical, and grammatical coverage, and, semantic and pronoun
resolution correctness. Stating that the majority of the content of scientific
documents contain noun phrases, the authors of [shaalan2004machine]
investigated machine translation of noun phrases from English into Arabic. The
same task has been achieved for the opposite direction, i.e. from Arabic into
English using transfer-based approach [hatem2010syntactic]. Though, the most
of MT systems have been performed mainly for Arabic and English, we can find
works on how to improve statistical machine translation from Arabic into
Chinese using English as pivot language [habash2009improving]. There are many
other works dealing with SMT from and to Arabic that have been carried out,
for instance in [habash2006arabic, hasan2006creating, elming2009syntactic,
hatem2008modified, nguyen2008context, carpuat2010improving,
tahir2010knowledge]. Recently, Deep Neural Networks have been adopted for
Arabic Machine Translation as in [gashaw2019amharicarabic,
DBLP:journals/mt/AmeurGM19, al2020neural, 2016arXiv160602680A]. The task
becomes more complicated when dealing with Arabic dialects that suffer of lack
of resources. Indeed, a lot of research works have been conducted to achieve
SMT and NMT tasks for Arabic dialects. Indeed, authors of [sawaf2010arabic]
presented a hybrid MT system from Arabic dialects into English. While in
[salloum2011dialectal], the authors improved an Arabic-English SMT system. In
[salloum2014sentence], the impact of sentence-level dialect identification on
machine translation performance has been explored.
In addition to works on Machine translation between Arabic dialects and other
languages, many trials to achieve Machine translation between Arabic (MSA) and
its dialects have been done covering Egyptian [bakr2008hybrid,
salloum2012elissa], Maghrebi [meftouh2015machine], Gulf and Levantine
[salloum2012elissa]. In spite of all these contributions, much research work
is still needed to improve the performance of Machine translation for Arabic.

####  5.3.3 Pedagogical Applications

Pedagogical applications of Arabic NLP can be divided into three general
areas: computer-assisted language learning systems based on NLP techniques,
readability assessment of Arabic texts, and corpus-building efforts in support
of language learning for learners of Arabic as a first language (L1) or as a
foreign or second language (L2). Pedagogical applications focus on four major
skills, namely reading, writing, listening, and speaking. For Arabic, such
applications have solely focused on the Modern Standard variety of Arabic
(MSA) with no application targeting dialectal varieties.

Computer-Assisted Language Learning (CALL) Systems

The earliest CALL system can probably be attributed to
[shaalan2005intelligent] [shaalan2006error]. The software included
morphological, syntax, and error analyzers to issue feedback to the learner.
The system used grammar and linguistic analysis rules to help learners
identify and correct a variety of errors, based on the instructor entering a
single correct answer to an exercise.

More recent work by [mahmoud2016intelligent] used production rules to provide
tutoring on Arabic grammar. The work was extended [mahmoud2018multiagents] to
use a multi-agent tutoring system that simulate the instructor, the student,
the learning strategy, a log book to monitor progress, and a learning
interface. Another approach focused on reading [maamouri2012developing] where
the reading experience is enriched with concordances, text-to-speech (TTS),
morpho-syntactic analysis, and auto-generated quiz questions.

##### Readability Assessment of MSA Texts

Text readability can be defined as the ability of a text to be read and
understood by its reader in an acceptable amount of time and effort given the
characteristics of the reader (skills, knowledge, interests), text
characteristics (lexical richness, grammatical complexity, genre, etc.), and
reader-text interaction. Learning to read MSA is somewhat akin to reading a
foreign tongue due to its lexical and syntactic divergence from people’s
native dialects.

For L1 readability, There has been multiple attempts to formulate readability
scores using easy-to-extract text characteristics and statistically estimated
parameters based on reference corpora [c-s2018review]. One striking
characteristic of these measures is that they used a limited number of
language-independent variables such as text length, average word length, and
average sentence length. Further measures, incorporated other parameters such
as the number of syllables in words, the relative rarity or absence of
dialectal alternatives, and the presence of less common letters [el-haj-
rayson-2016-osman].

An alternative approach, particularly for L2, entails using supervised
learning to automatically classify the readability levels of texts using
supervised classification methods such as Support Vector Machines (SVMs)
[alkhalifa2010arability, alTamimi2014AARI] and decision trees
[c-s2014elmezouar]. The classifiers would look at different features, such as
surface, morphological, lexical, and syntactic features, of the text along
with the reader’s exposure to the vocabulary in earlier readings.

Although clear progress has been made for both L1 and L2 readability
assessment, machine learning techniques are still not able to give fine-
grained discrimination of the difficulty of texts. We believe that there are a
few fundamental reasons for this can be attributed to the dearth of reliable
computational tools and corpora – compared to English – with available ones
being strongly task dependent. Most the current work has focused on L1 readers
with particular emphasis on grade school curricula [Al-Ajlan2008,
alkhalifa2010arability, alTamimi2014AARI]. There are unfortunately questions
about the suitability of these curricula to student [usaid2015], and there is
a push to inform the enhancement of curricula using pedagogical tools and to
compare curricula across Arab countries [al-sulaiti-etal-2016-compilation,
mubarak2020curricula]. The L2 pedagogical applications are even more limited
with limited corpora [maamouri2012developing] with disproportionate focus on
beginners151515https://learning.aljazeera.net/en. There is a definite need for
augmenting these corpora in a reasoned way, taking into consideration
different text features and learners, both young and old, beefing up the
sparsely populated levels with authentic material, and exploiting technologies
such as text simplification and text error analysis and correction. Learner
corpora, which as the name suggests are produced by learners of Arabic can
inform the creation of tools and corpora [Alfaifi2014ArabicLC,
alkanhal2012kacstc, alotaibi2017parallel, habash2017qalb, rytting-
etal-2014-arcade].

####  5.3.4 Dialogue Systems

Automated dialog systems capable of sustaining a smooth and natural
conversation with users have attracted considerable interest from both
research and industry in the past few years. This technology is changing how
companies engage with their customers among many other applications. While
commercial dialog systems such as Amazon Alexa, Google Home and Apple Siri
support many international languages, only Apple Siri supports Arabic with
limited performance. On the other hand, there is an important body of research
work on English language dialog systems. Current NLP methods for Arabic
language dialogue, action classification or semantic decoding is mostly based
on handcrafted rule-based systems and methods that use feature engineering
[alhumoud2018arabic]. Indeed, as explained in previous sections, Arabic
language texts have a lot of challenges, such as the complexity of Arabic
grammar, a high degree of ambiguity, orthographic variations, and the
existence of multiple dialects.

Among the earliest research works on Arabic dialog application is the Quran
chatbot [shawar2004arabic] where the conversation length is short since the
system answers a user inputer with a single response. It uses a retrieval-
based model as the dataset is limited by the content of the Quran. More
recently, and following current trends, [bashir2018implementation] presented
an approach that uses deep learning techniques for text classification and
Named Entity Recognition to build a natural language understanding module –
the core component of any dialogue system – for the domain of home automation
in Arabic. Other works have focused on developing dialog systems for the case
of Arabic dialects. For instance, [ali2016botta] developed an Egyptian
chatbot, named BOTTA, that is publicly available for users who want to chat
with it online. Also, [alnabiha] built “Nabiha” a chatbot that can carry out
conversation with Information Technology students in the Saudi Arabic dialect.
As a whole, with these limited efforts, Arabic dialogue systems technology is
still in an infancy stage. There is a great need to develop new advanced NLP
understanding and speech recognition modules for Arabic language users.

####  5.3.5 Content Moderation on Social Media

Typically, news websites and online forums do not allow posting user comments
that contain offensive language, hate speech, cyber-bullying, and spam among
other types of inappropriate or dangerous content161616ex:
https://www.bbc.co.uk/usingthebbc/terms/what-are-the-rules-for-commenting/.
Although social media platforms like Facebook and Twitter have similar rules
to protect users from harmful content, many posts violate these rules. Such
content cannot be easily detected given the huge volume of posts, dialectal
variations, creative spelling on social media, and the scarcity of available
data and detection tools. This area is relatively new for Arabic. One of the
more active areas has to do with the detection of offensive language, which
covers targeted attacks, vulgar and pornographic language, and hate speech.
Initial work was performed on comments from a news site and limited numbers of
tweets (1.1k) [mubarak2017abusive] and YouTube comments [alakrot2018towards].
Some works focused on adult content [Alshehri2018ThinkBY] and others on hate
speech [albadi2018they, al2019detection]. SemEval2020 and the workshop on
Open-Source Arabic Corpora and Processing Tools included shared tasks on the
automatic detection of such language using a labeled set of 10k tweets
[mubarak-etal-2020-overview]. A myriad of classifiers were used for this task
including SVMs, recurrent neural networks, and transformer models. Work on
spam detection on Twitter is nascent and much work is required
[mubarak2020spam].

###  5.4 Arabic Information Retrieval (IR)

With the increasing volume of Arabic content, information retrieval has become
a necessity for many domains such as medical records, digital libraries, web
content, and news. The main research interests have focused on retrieval of
formal language, mostly in the news domain, with ad hoc retrieval, OCR
document retrieval, and cross-language retrieval. The literature on other
aspects of retrieval continues to be sparse or non-existent, though some of
these aspects have been investigated by industry. Others aspects of Arabic
retrieval that have received some attention include document image retrieval,
speech search, social media and web search, and filtering [darwish2014arabic].
However, efforts on different aspects of Arabic retrieval continue to be
deficient and severely lacking behind efforts in other languages. Examples of
unexplored problems include searching Wikipedia, which contains semi-
structured content, religious text, which often contain semi-structured data
such chains of narrations, rulings, and commentaries, Arabic forums, which are
very popular in the Arab world and constitute a significant portion of the
Arabic web, and poetry. To properly develop algorithms and methods to retrieve
such content, standard test sets and clear usage scenarios are required. We
also opine that recent improvements in contextual embeddings can positively
impact the effectiveness of many retrieval tasks.

Another IR related problem is question answering, which comes in many flavors,
the most common of which is trying to attempting to identify a passage or a
sentence that answers a question. Performing such a task may employ a large
set of NLP tools such as parsing, NER, co-reference resolution, and text
semantic representation. There has been limited research work on this problem
[IEEE:mawdoo3/deep2019, romeo2019language] and existing commercial solutions
such as Ujeeb.com are rudimentary.

##  6 Future Directions

  * •

An association for Arabic Language Technologies?

  * •

A hub for data and resources

  * •

NLP education in the Arab world

  * •

Attracting Arab talent to work on Arabic

  * •

More basic common frameworks for development