#  A Novel Multi-Task Learning Empowered Codebook Design for Downlink SCMA
Networks

Qu Luo, Zilong Liu, Gaojie Chen,
Yi Ma, Pei Xiao  Qu Luo, Gaojie Chen, Yi Ma and Pei Xiao are with 5G & 6G
Innovation Centre, University of Surrey, UK, email:{q.u.luo, gaojie.chen,
y.ma<EMAIL_ADDRESS>Zilong Liu is with the School of Computer Science
and Electronics Engineering, University of Essex, UK. email:
<EMAIL_ADDRESS>This work was supported in part by the UK Engineering
and Physical Sciences Research Council under Grant EP/P03456X/1 and the Key
Project of Science and Technology of Hainan (N0. ZDKJ2019003).

###### Abstract

Sparse code multiple access (SCMA) is a promising code-domain non-orthogonal
multiple access (NOMA) scheme for the enabling of massive machine-type
communication. In SCMA, the design of good sparse codebooks and efficient
multiuser decoding have attracted tremendous research attention in the past
few years. This paper aims to leverage deep learning to jointly design the
downlink SCMA encoder and decoder with the aid of autoencoder. We introduce a
novel end-to-end learning based SCMA (E2E-SCMA) design framework, under which
improved sparse codebooks and low-complexity decoder are obtained. Compared to
conventional SCMA schemes, our numerical results show that the proposed
E2E-SCMA leads to significant improvements in terms of error rate and
computational complexity.

###### Index Terms:

SCMA, codebook design, deep neural network, autoencoder, multi-task learning.

##  I Introduction

T he wireless networks are rapidly evolving towards a paradigm shift from
connecting people to networking everything. A pressing challenge of future
wireless network design is how to develop a highly efficient multiple access
scheme to meet various stringent requirements such as lower access latency,
and higher spectral efficiency. A disruptive technique for addressing such a
challenge is called non-orthogonal multiple access (NOMA). In a NOMA system,
multiple users are able to communicate simultaneously to achieve overloading
factor larger than 1. Existing NOMA techniques can be largely categorized into
two classes: power-domain NOMA and code-domain NOMA (CD-NOMA) [1, 2]. In this
paper, we focus on an emerging CD-NOMA scheme called sparse code multiple
access (SCMA) in which multiple users are separated by adopting different
sparse codebooks [3, 4]. Over the past decade, SCMA has attracted tremendous
research attention from both academia and industry [5, 6, 7, 8].

In SCMA, two fundamental research problems are the design of good sparse
codebooks and efficient multi-user decoding [9, 10, 5, 6]. Existing known SCMA
codebook constructions mostly follow a multi-stage sub-optimal design for
rapid generation [10, 6, 5], albeit it is unclear how far the obtained SCMA
codebooks are from the optimal ones. By taking advantage of the codebook
sparsity, low-complexity MPA has been developed for SCMA decoding. For a
downlink SCMA system where multiple user devices (e.g., sensors, tablets,
machines) are constrained by their limited computation capability and battery
life, however, the current MPA may not be affordable, especially when a large
number of MPA iterations is needed [9, 11].

In recent years, deep learning (DL) has been extensively studied in wireless
networks, thanks to its capability in solving very complicated optimization
problem [12]. A comprehensive introduction on autoencoder for end-to-end
communication system was contributed by O’shea and Hoydis in [12]. Following
[12], a denoising autoencoder (DAE) for SCMA was reported in [13]. The core
idea of [13] is to model the entire SCMA system as a DAE by implementing both
the encoder and decoder with fully connected neural networks (NNs).
Subsequently, a similar structure was studied in [11, 14] by jointly
considering the sparse and dense mapping of CD-NOMA. It is noted that [13, 11,
14] considered the decoder as a single learning task implemented with fully
connected layers. However, the bit error performances of these systems may not
beat an SCMA system with the aforementioned sparse codebooks that are obtained
from a multi-stage sub-optimal design. Very recently, a deep neural network
(DNN) with multi-task structure was proposed in [15] for SCMA detection.
However, [15] has not touched the sparse codebook design with the aid of DNN,
and hence a good error rate performance may not be guaranteed.

In this letter, we introduce a novel multi-task learning empowered end-to-end
SCMA (E2E-SCMA) design framework. The main novelty of this work stems from the
proposed architecture of E2E-SCMA and the unique training scheme. Building
upon a new SCMA mapping design with linear encoding, we first propose an
efficient SCMA encoder, which can reduce the depth of the network and thereby
helping prevent the gradient from vanishing. Unlike existing works [13, 11,
14], where the decoding is conducted by viewing $J$ users as a single learning
task, we view each user as a single learning task and then design the decoder
in a task-specific fashion. The advantages of using the multi-task learning
structure are twofold: 1) it can improve learning efficiency and reduce over-
fitting [16]; 2) it can avoid the curse of dimensionality while using one-hot
encoding. Specifically, for an multi-task learning structure of $J$ tasks, if
each task has a $M$-dimensional input vector, the corresponding input
dimension of single task learning structure will increase to $M^{J}$. Finally,
we propose to train the E2E-SCMA in a range of signal-to-noise ratios (SNRs)
instead of over a fixed SNR. Consequently, this enables the proposed E2E-SCMA
to work over a wide range of SNR values with a low error rate performance. The
remainder of the letter is organized as follows. Section II briefly describes
the system model of SCMA. We present the proposed E2E-SCMA framework in
Section III. The numerical results and conclusion are presented in Sections IV
and V, respectively.

##  II System Model

In this paper, we consider a downlink SCMA system with $J$ users communicating
over the $K$ orthogonal resources, where $J>K$. Let us define the overloading
factor as $\lambda=\frac{J}{K}>1$. At the transmitter side, the SCMA encoder
maps $\log_{2}\left(M\right)$ binary bits toa length-$K$ codeword drawn from
codebook $\mathcal{X}_{j}\in\mathbb{C}^{K}$ with size $M$. The mapping process
is defined as
$f_{j}:\mathbb{B}^{\log_{2}M}\rightarrow{\mathcal{X}}_{j}\in\mathbb{C}^{K}$,
where
$\mathcal{X}_{j}=\\{\mathbf{x}_{j,1},\mathbf{x}_{j,2},\ldots,\mathbf{x}_{j,m}\\}$
is the codebook set for the $j$th user with cardinality of $M$. All the
$K$-dimensional complex codewords of each SCMA codebook are sparse vectors
with $N$ non-zero elements111For user $j$, the $N$ non-zero element positions
remain unchanged from one codeword to another. and $N<K$. Let $\mathbf{c}_{j}$
be a length-$N$ vector drawn from ${\mathcal{C}}_{j}\subset\mathbb{C}^{N}$,
where ${\mathcal{C}}_{j}$ is obtained by removing all the zero elements in
${\mathcal{X}}_{j}$. We further define the mapping from
$\mathbb{B}^{\log_{2}M}$ to ${\mathcal{C}}_{j}$ as

$\small g_{j}:\mathbb{B}^{\log_{2}M\times
1}\mapsto\boldsymbol{{\mathcal{C}}}_{j},\quad{~{}\text{i.e.,
}}\mathbf{c}_{j}=g_{j}(\mathbf{b}_{j}),$ (1)

where
$\mathbf{b}_{j}=[b_{j,1},b_{j,2},\ldots,b_{j,\log_{2}M}]^{T}\in\\{1,-1\\}^{\log_{2}M}$
stands for $j$th user’s instantaneous input binary message vector. By
collecting all the $\mathbf{b}_{j}$ according to their corresponding integer
values in ascending order, we form a $\log_{2}(M)\times M$ binary matrix
$\mathbf{B}$. For example, when $M=4$, we have

$\displaystyle\mathbf{B}=\left[{\begin{matrix}-1&+1&-1&+1\\\
-1&-1&+1&+1\end{matrix}}\right].$ (2)

Thus, the corresponding SCMA mapping $f_{j}$ can be expressed as

$\small f_{j}:\equiv\mathbf{V}_{j}g_{j},\quad{~{}\text{i.e.,
}}\mathbf{x}_{j}=\mathbf{V}_{j}g_{j}(\mathbf{b}_{j}),$ (3)

where $\mathbf{V}_{j}\in\mathbb{B}^{K\times N}$ is an mapping matrix that maps
the $N$-dimensional vector to a $K$-dimensional sparse SCMA codeword. The
sparse structure of the $J$ SCMA codebooks can be represented by the indicator
(sparse) matrix
$\mathbf{F}=\left[\mathbf{f}_{1},\ldots,\mathbf{f}_{J}\right]\subset\mathbb{B}^{K\times
J}$ where $\mathbf{f}_{j}=\text{diag}(\mathbf{V}_{j}\mathbf{V}_{j}^{T})$.

For a fixed $\mathbf{V}_{j}$, the task of SCMA codebook design is to find the
$J$ mapping functions $g_{j},j=1,2,\ldots,J$, according to certain criteria,
such as minimum Euclidean distance (MED). Specifically, by viewing the mapping
function $g_{j}$ as a ${N\times\text{log}_{2}M}$ complex codebook generator
matrix times the $j$th user’s bit vector $\mathbf{b}_{j}$, we have

$\small\mathbf{c}_{j}={{\mathbf{G}}_{j}}{{\mathbf{b}}_{j}},$ (4)

where ${{\mathbf{G}}_{j}}\in{{\mathbb{C}}^{N\times\text{log}_{2}M}}$ is the
codebook generator matrix of the $j$th user. Therefore, the codebook for user
$j$ is $\mathcal{X}_{j}=\mathbf{V}_{j}{{\mathbf{G}}_{j}}{{\mathbf{B}}}$.

The received signal of user $j$ in downlink channel after the multiplexing can
be expressed as

$\displaystyle\mathbf{r}_{j}=\operatorname{diag}\left(\mathbf{h}_{j}\right)\sum_{u=1}^{J}\mathbf{V}_{u}\mathbf{G}_{u}\mathbf{b}_{u}+\mathbf{n}_{j},$
(5)

where
${{\mathbf{h}}_{u}}={{\left[{{h}_{j,1}},{{h}_{j,2}},\ldots,{{h}_{j,K}}\right]}^{T}}\in{{\mathbb{C}}^{K\times
1}}$ is the channel coefficient vector between the base station and the $j$th
user, diag$(\cdot)$ denotes the diagonalization of a matrix and
${{\mathbf{n}_{j}}=\left[{{n_{j,1},{n_{j,2}},\ldots,{n_{j,K}}}}\right]}^{T}$
is the complex Gaussian vector with the variance with zero mean and variance
$N_{0}$, i.e., ${{n}_{j,k}}\sim\mathcal{CN}\left({0,{N_{0}}}\right)$.

In the next section, we will design the near optimal generating matrices
$\mathbf{G}_{j},j=1,2,\ldots,J$ to improve the error rate performance with the
proposed novel autoencoder.

##  III Proposed novel autoencoder

In this section, a novel autoencoder is presented for downlink SCMA systems.
We first present the SCMA mapping, i.e., the signal model in (4), inspired
encoder designed. Then, the multi-user detection with deep multi-task learning
is elaborated. In addition, training procedure and complexity analysis will be
discussed.

###  III-A Autoencoder

Autoencoder is a special class of neural networks which is trained to produce
an output data that matches with their input data. It is composed of a basic
DNN unit formed of multiple repetitive hidden layers. Each hidden layer is an
affine mapping followed by a nonlinearlity operator. The output of the $l$th
hidden layer is given by

$\mathbf{x}_{l}=\sigma_{l}\left(\mathbf{W}_{l}\mathbf{x}_{l-1}+\mathbf{z}_{l}\right),$
(6)

where $\boldsymbol{W}_{l}$, $\mathbf{z}_{l}$, and $\sigma_{l}$ denote the
weight matrix, bias vector and the activation function for the $l$th layer,
respectively. The encoder first transforms the input vector $\mathbf{x}$ into
hidden representation $\mathbf{y}$ through a deterministic mapping
$e_{\boldsymbol{\theta}}$, i.e.,
$\mathbf{y}=e_{\boldsymbol{\theta}}\left(\mathbf{x}\right)$, where
$\boldsymbol{\theta}$ denotes the parameter set with all the weight matrices
and bias vectors. The resulting representation $\mathbf{y}$ is then mapped
back to reconstruct the input vector, i.e.,
$\mathbf{\hat{x}}=d_{\boldsymbol{\theta}^{\prime}}\left(\mathbf{y}\right)$.
The mapping $d_{{\boldsymbol{\theta}}}$ is called decoder and
$\mathbf{{\boldsymbol{\theta}^{\prime}}}$ is the corresponding parameter set.
The DAE is a type of autoencoder that learns to produce original denoised
samples from the inputs contaminated by noise. In an DAE, the parameter set
$\theta$ and $\boldsymbol{\theta}^{\prime}$ are trained to minimize the
reconstruction error [16]

$\small{\boldsymbol{\theta}}^{*},{\boldsymbol{\theta}^{\prime}}^{*}=\underset{{\boldsymbol{\theta}},{\boldsymbol{\theta}^{\prime}}}{\operatorname{argmin}L}\left(\mathbf{x},d_{{\boldsymbol{\theta}^{\prime}}}\left(e_{{\boldsymbol{\theta}}}\left(\mathbf{x}\right)\right)\right),$
(7)

where $L$ is a loss function, such as the squared error loss
$L\left(\mathbf{x},\mathbf{\hat{x}}\right)={\parallel\mathbf{x}-\mathbf{\hat{x}}\parallel}^{2}$.
Another commonly used loss function is the cross-entropy loss
${{L_{\text{CE}}}}\left(\mathbf{x},\mathbf{\hat{x}}\right)=-\sum_{d=1}^{D}x_{d}\log(\hat{x}_{d})$,
where $D$ is the length of the output vector, $x_{d}\in\mathbf{x}$ and
$\hat{x}_{d}\in\mathbf{\hat{x}}$. Note that for cross-entropy loss,
$\mathbf{x}$ and $\mathbf{\hat{x}}$ are in the form of the bit vector and bit
probability, respectively.

Figure 1: The system structure of the proposed E2E-SCMA.

###  III-B Signal Model Inspired Encoder Design

In our proposed E2E-SCMA, the mapping from the $j$th data stream to the $j$th
user’s constellation, i.e., ${\mathbf{c}}_{j}=g_{j}(\mathbf{b}_{j})$ is
implemented with neural networks. Note that the SCMA encoding in (4) has the
same expression with neural network in (6) when the activation function is
linear with basis $\boldsymbol{z}=\boldsymbol{0}^{T}$. Therefore, the codebook
generation process, i.e., $g_{j}$, can be implemented with a simple neural
network, which only consists of the input layer and output layer. The weight
matrix in the neural network is equivalent to the generator matrix
$\mathbf{G}_{j}$. Since the proposed network operates in real domain, the
output is separated into real and imaginary parts. Hence, (4) is re-written as

$\small\mathbf{\bar{c}}_{j}={{\mathbf{\bar{G}}}_{j}}{{\mathbf{b}}_{j}},$ (8)

with

$\mathbf{\bar{c}}_{j}={\left[\begin{array}[]{c}\Re({\mathbf{c}_{j}})\\\
\Im({\mathbf{c}_{j}})\end{array}\right]},\mathbf{\bar{G}}_{j}={\left[\begin{array}[]{cc}({\mathbf{G}_{j}^{\text{
R}}})^{T}&({\mathbf{G}_{j}^{\text{I}}})^{T}\end{array}\right]^{T}},$ (9)

where ${\mathbf{G}_{j}^{\text{R}}}$ and ${\mathbf{G}_{j}^{\text{I}}}$ are the
generator matrices of the real and imaginary parts, respectively. Based on the
above analysis, the proposed model based E2E-SCMA with $J$ users is shown in
Fig. 1, where the proposed E2E-SCMA is composed of $J$ codebook generators, a
signature mapping module, a channel module, and a multi-user detection module.
The structure of codebook generator is inspired by the signal model and only
consists two layers, i.e., the input layer and the output layer. In addition,
the number of nodes for input layer and output layer are $\text{log}_{2}(M)$
and $2N$, respectively.

In the forward-propagation phase, source message vector $\mathbf{{b}}_{j}$
first flows through codebook generator network, parameterized by
$\mathbf{\bar{G}}_{j}$ to derive the multi-dimensional complex symbol
$\mathbf{\bar{c}}_{j}$, and then the symbols are mapped to SCMA resources
according to $\mathbf{V}_{j}$. After that, $J$ users’ data symbols are
superimposed before passing through a Gaussian channel222In this paper, we
focus on the Gaussian channel case as in [13, 11, 14] in order to give a clear
comparison with other benchmarks. The fading channel will be investigated in
future work.. Finally, the superimposed signal is decoupled to accurately
recover source messages based on task-specific sub-networks in the decoder,
which will be elaborated in the next subsection.

###  III-C Decoder Design with Multi-task Learning

At the decoder part, deep multi-task learning is adopted to design the multi-
user detector. The proposed decoder consists of one shared network and $J$
user specific sub-networks, where the shared network is designed for
exchanging the information between the subcarriers and the $j$th task is
responsible for recovering the $j$th user’s data. We employ one-hot vector to
represent the input binary message vector $\mathbf{b}_{j}$, namely, each
message $\mathbf{b}_{j,m},m\in\\{1,2,\ldots,M\\}$ is represented by an
$M$-dimensional one-hot vector ${\boldsymbol{\mathfrak{m}}}_{j}^{m}$, which is
the $m$th column of the identity matrix $\mathbf{I}_{M}$. For example, for
$M=4$, the one-hot mapping is defined as

$\displaystyle\mathbf{b}_{j,1}=\left[-1,-1\right]^{T}\leftrightarrow$
$\displaystyle\boldsymbol{\mathfrak{m}}_{j}^{1}=[1,0,0,0],$ (10)
$\displaystyle\mathbf{b}_{j,2}=\left[-1,+1\right]^{T}\leftrightarrow$
$\displaystyle\boldsymbol{\mathfrak{m}}_{j}^{2}=[0,1,0,0],$
$\displaystyle\mathbf{b}_{j,3}=\left[+1,-1\right]^{T}\leftrightarrow$
$\displaystyle\boldsymbol{\mathfrak{m}}_{j}^{3}=[0,0,1,0],$
$\displaystyle\mathbf{b}_{j,4}=\left[+1,+1\right]^{T}\leftrightarrow$
$\displaystyle\boldsymbol{\mathfrak{m}}_{j}^{4}=[0,0,0,1].$

The decoder can be expressed as
$d_{{\overline{{\boldsymbol{\theta}}}}}^{\text{P}}d_{{{\boldsymbol{\theta}_{j}}}}^{\text{U}}:{\boldsymbol{r}_{j}}\to{{\boldsymbol{{p}}}}_{j}$,
where $d_{{\overline{{\boldsymbol{\theta}}}}}^{\text{P}}$ and
$d_{{{\boldsymbol{\theta}_{j}}}}^{\text{U}}$ are the non-linear mapping of the
forward DNN for the shared network and the $j$th user’ sub-network,
respectively. ${{{{\boldsymbol{p}}}}}_{j}$ is the output messages,
$\overline{{\boldsymbol{\theta}}}$ and ${\boldsymbol{\theta}}_{j}$ are the
parameter sets of the shared network and the $j$th user’ sub-network,
respectively. In our implementation, we choose fully-connected DNN with
$L_{\text{P}}$ and $L_{\text{U}}$ layers for both shared network and user sub-
network. The above process can be expressed as

$\displaystyle{{\boldsymbol{{\mathbf{p}}}}}_{j}$
$\displaystyle=d_{{{\boldsymbol{\theta}_{j}}}}^{\text{U}}\left({{\mathbf{x}}_{\text{P}}}\right)=\sigma_{j,L_{\text{U}}}^{\text{U}}\Big{(}{\mathbf{W}}_{j}^{(L_{\text{U}})}\Big{(}\sigma_{j,L_{\text{U}}-1}^{\text{U}}\cdots$
(11)
$\displaystyle\quad{\sigma_{j,1}^{\text{U}}\left({{{\mathbf{W}_{j}}^{(1)}}{{\mathbf{x}}_{\text{P}}}+{{\mathbf{z}_{j}}^{(1)}}}\right)\cdots+{\mathbf{z}}_{j}^{(L_{\text{U}}-1)}\Big{)}+{{\mathbf{z}_{j}}^{\left({L_{\text{U}}}\right)}}\Big{)},}$
$\displaystyle{{\mathbf{x}}}_{\text{P}}$
$\displaystyle=d_{{\overline{{\boldsymbol{\theta}}}}}^{\text{P}}\left({{{\mathbf{r}}}}\right)=\sigma_{L_{\text{P}}}^{\text{P}}\Big{(}{\overline{\mathbf{W}}}^{(L_{\text{P}})}\Big{(}\sigma_{L_{\text{P}}-1}^{\text{P}}\cdots$
$\displaystyle\quad{\sigma_{1}^{\text{P}}\left({{\overline{{\mathbf{W}}}^{(1)}}{\mathbf{r}_{j}}+{\overline{{\mathbf{z}}}^{(1)}}}\right)\cdots+{\overline{\mathbf{z}}}^{(L_{\text{P}}-1)}\Big{)}+{\overline{{\mathbf{z}}}^{\left({L_{\text{P}}}\right)}}\Big{)},}$

where ${\mathbf{x}}_{\text{P}}$ is the output of the shared layer,
$\sigma_{l}^{\text{P}}$ and $\sigma_{j,l}^{\text{U}}$ denote the activation
function of the $l$th layer of shared network and the $j$th sub-network,
respectively.
${{{\boldsymbol{\theta}}}_{j}}=\Big{\\{}{\mathbf{W}}_{j}^{(1)},{\mathbf{z}}_{j}^{(1)},\ldots,{\mathbf{W}}_{j}^{(L_{\text{U}})},{\mathbf{z}}_{j}^{(L_{\text{U}})}\Big{\\}},$
and
${\overline{{\boldsymbol{\theta}}}}=\Big{\\{}{\overline{\mathbf{W}}}^{(1)},$
${\overline{\mathbf{z}}}^{(1)},\ldots,{\overline{\mathbf{W}}}^{(L_{\text{\text{P}}})},{\overline{\mathbf{z}}}^{(L_{\text{P}})}\Big{\\}}$
are the parameters to be learned.

Observing that the task of SCMA detection is to recover the source messages in
a limited search space, such a problem is equivalent to a typical
classification problem in the machine learning field. Hence, this motivates us
to employ the widely used softmax activation for output layer. To facilitate
the network convergence, ReLU activation function is adopted for hidden
layers. Assume that the input of softmax is a vector $\mathbf{w}_{j}$ of
dimension $M$, and $w_{j,m}$ is the $m$th entry of $\mathbf{w}_{j}$. Then, the
softmax activation function takes the following expression:

$p_{j,m}=\frac{\exp(w_{j,m})}{\sum_{m^{\prime}=1}^{M}\exp(w_{j,m^{\prime}})},$
(12)

where $p_{j,m}$ is the $m$th entry of the output ${{\mathbf{p}}}_{j}$ with
$\sum_{m=1}^{M}p_{j,m}=1$. All hidden layers adopt ReLU activation function,
which can facilitate the network convergence during the training process. As
for the loss function, we consider the corresponding softmax cross-entropy
loss for each user. Let
${\mathbf{p}}=[{\mathbf{p}}_{1}^{T},{{\mathbf{p}}}_{2}^{T},\dots,{{\mathbf{p}}}_{J}^{T}]^{T}$
and
$\boldsymbol{\mathfrak{m}}=[\boldsymbol{\mathfrak{m}}_{1}^{T},\boldsymbol{\mathfrak{m}}_{2}^{T},\dots,\boldsymbol{\mathfrak{m}}_{J}^{T}]^{T}$,
where $\boldsymbol{\mathfrak{m}}_{j}$ is the one hot representation of
$\mathbf{b}_{j}$. The overall loss function is the summation over $J$ users,
which can be expressed as

$\small{{L}}^{\text{E2E-SCMA}}({{\mathbf{p}}},\boldsymbol{\mathfrak{m}})=-\sum_{j=1}^{J}\sum_{m=1}^{M}q_{j,m}\log(p_{j,m}),$
(13)

where $q_{j,m}$ denotes the $m$th entry of $\boldsymbol{\mathfrak{m}}_{j}$.
The loss function measures the difference between predicted probability
$\mathbf{p}$ diverges from the actual label $\boldsymbol{\mathfrak{m}}$.
Therefore, we aim to seek the model parameters
$\overline{\mathbf{G}}_{j},\overline{{\boldsymbol{\theta}}},{\boldsymbol{\theta}}_{j}$
to minimize the overall loss:

$\small\\{{\overline{\mathbf{G}}_{j}}^{*},\overline{{\boldsymbol{\theta}}}^{*},{{{\boldsymbol{\theta}}}_{j}^{*}}\\}=\underset{{\left[{\overline{\mathbf{G}}_{j}}\right]_{j=1}^{J}},\overline{{\boldsymbol{\theta}}},{\left[{{{\boldsymbol{\theta}}}_{j}}\right]_{j=1}^{J}}}{\mathop{\arg\min}}\,{{L}}^{\text{E2E-SCMA}}({{\mathbf{p}}},\boldsymbol{\mathfrak{m}}).$
(14)

Algorithm 1 Training of E2E-SCMA.

0: Set $J$, $K$, $\mathbf{V}_{j}$, $\alpha_{0},$ $\beta,$ $D$,
${{E}_{b}/{N}_{0}}_{\min},$ ${{E}_{b}/{N}_{0}}_{\max}$, $I_{T}$ and initialize
the network parameters
${{\overline{\mathbf{G}}_{j}}},\overline{{\boldsymbol{\theta}}},$
${{\boldsymbol{\theta}_{j}}},j\in\\{1,2,\dots,J\\}$.

1: repeat

2: $t\leftarrow 1$

3: Randomly generate training samples $\mathbf{b}_{j}$ and transfer
$\mathbf{b}_{j}$ to one-hot vector ${\boldsymbol{\mathfrak{m}}}_{j}$

4: Froward Propagation

5:
$\text{SNR}\leftarrow\mathcal{U}\left({{E}_{b}/{N}_{0}}_{\min},{{E}_{b}/{N}_{0}}_{\max}\right)$,
$\alpha_{t}\leftarrow\alpha_{0}\beta^{\left(t/D\right)}$

6: $\mathbf{\bar{c}}_{j}\leftarrow\mathbf{b}_{j}$ according to (8) and (9)

7: $\mathbf{r}\leftarrow$ Obtain $\mathbf{r}$ after resource mapping and pass
channel

8: ${{\boldsymbol{{p}}}}_{j}\leftarrow
d_{{\overline{{\boldsymbol{\theta}}}}}^{\text{P}}{\cdot}d_{{{\boldsymbol{\theta}_{j}}}}^{\text{U}}\left({\boldsymbol{r}}\right)$
according to (11)

9:
${{L}^{t}}_{batch}\leftarrow{{L}}^{\text{E2E-SCMA}}({{\mathbf{p}}},\boldsymbol{\mathfrak{m}})$
according to (13)

10: Backward Propagation

11: ${\overline{\mathbf{G}}_{j}}$,
$\boldsymbol{\theta}_{j},\boldsymbol{{\overline{\theta}}}\leftarrow$ Update
$\boldsymbol{\theta}_{j},\boldsymbol{{\overline{\theta}}}$ with $\alpha_{t},$
$\nabla_{{\boldsymbol{\theta}}_{j},\mathbf{\overline{\boldsymbol{\theta}}}}{{L}^{t}}_{batch}$,
and ${\overline{\mathbf{G}}_{j}}$ with $\alpha_{t}$ and
$\nabla_{{\overline{\mathbf{G}}_{j}},\overline{\boldsymbol{\theta}},{\boldsymbol{\theta}}_{j}}{{L}^{t}}_{batch}$
with gradient-based optimizer

12: $t\leftarrow t+1$

13: until reaching the maximum iteration number $I_{T}$

###  III-D Training Algorithm

The encoder and decoder are jointly optimized with gradient decent based
method using forward and backward propagation, such as adaptive moment
estimation (ADAM). Algorithm 1 demonstrates the detailed training of the
proposed E2E-SCMA system. $\mathbf{\bar{G}}_{j},j=1,2,\ldots,J$ are first
initialized with Huawei codebook [17]. Specifically, we first obtain
$\mathbf{{G}}_{j}=\mathbf{C}_{j}\mathbf{B}^{T}(\mathbf{B}\mathbf{B}^{T})^{-1}$,
where $\mathbf{C}_{j}$ denotes the $j$th user’s codebook in [17] by removing
the zero dimensions. Then, $\mathbf{\bar{G}}_{j}$ is obtained by concatenating
the real and imaginary parts of $\mathbf{{G}}_{j}$. The weights of the
decoder, i.e., $\boldsymbol{\bar{\theta}}$, and $\boldsymbol{\theta}_{j}$, are
initialized with a normal distribution with mean $0$ and variance $1$. In the
forward propagation, the randomly generated input data first flows through the
encoder and decoder to obtain an estimation of the input message. Then, during
the backward propagation, the parameters
${{\overline{\mathbf{G}}_{j}}},\overline{{\boldsymbol{\theta}}},$ and
${{\boldsymbol{\theta}_{j}}}$ are updated by minimising the total loss. In
addition, the learning rate $\alpha_{t}$ decays exponentially at each
iteration $t$ with a decay factor of $\beta$ and decay step of $D$. With
respect to the training ${E}_{b}/{N}_{0}$, the authors in [13, 11, 14, 15]
obtained SCMA codebooks by training the system at a fixed ${E}_{b}/{N}_{0}$.
However, in our implementation, the training SNR for each iteration was
randomly generated so that the SNR will be uniformly distributed on
$\mathcal{U}\left({{E}_{b}/{N}_{0}}_{\min},{{E}_{b}/{N}_{0}}_{\max}\right)$.
This approach allows us to train an SCMA system to work over a wide range of
SNR values while maintaining a low error rate performance.

###  III-E Complexity Analysis

The main differences between E2E-SCMA and convention SCMA in terms of
complexity is the decoder part, i.e., DNN decoder and MPA. Hence, we main
focus on analyze the complexity of DNN decoder and MPA. The complexity of MPA
is given by $\mathcal{O}\left({{N}_{iter}}Kd_{f}^{2}{{M}^{df}}\right)$ [9],
where ${{N}_{itr}}$ is defined as the iteration number of MPA. For E2E-SCMA,
we are concerned about the complexity of online deployment. The main
computation in E2E-SCMA is matrix multiplication, which is dominated by the
two consecutive layers with the largest number of neural nodes. Therefore, we
can simply the computation complexity as $\mathcal{O}\left(L_{1}L_{2}\right)$,
where $L_{1}$ and $L_{2}$ are the largest number of neural nodes of two
consecutive layers.

##  IV Numerical results

In this section, we evaluate the error rate performance of the proposed
E2E-SCMA system in Gaussian channel. The following indicating matrix with
$J=6,K=4,N=2$ is given by

$\displaystyle\mathbf{F_{4\times 6}}=\left[{\begin{matrix}0&\quad 1&\quad
1&\quad 0&\quad 1&\quad 0\\\ 1&\quad 0&\quad 1&\quad 0&\quad 0&\quad 1\\\
0&\quad 1&\quad 0&\quad 1&\quad 0&\quad 1\\\ 1&\quad 0&\quad 0&\quad 1&\quad
1&\quad 0\\\ \end{matrix}}\right].$ (15)

The initial learning rate, decay step and decay factor are set to be
$\alpha_{0}=0.001$, $D=500$ and $\beta=0.9$, respectively. The batch size for
each iteration is set to be $1000$ for a trade-off between convergence rate
and computational efficiency. The maximum iteration number is $I_{T}=2000$.
Therefore, the total number of training samples is $2\times 10^{6}$. We choose
a wide range of training ${{E}_{b}/{N}_{0}}$, specifically, we set
${{E}_{b}/{N}_{0}}_{\min}=5$ dB and ${{E}_{b}/{N}_{0}}_{\max}=11$ dB. The
codebook generator is implemented with $\log_{2}(M)$ input nodes and $2\times
K$ output nodes. For the decoder, the number of nodes and hidden layers for
shared network are $\\{128,64\\}$ and $L_{P}=2$, respectively, whereas the two
parameters for user sub-network are $\\{64,32,16\\}$ and $L_{U}=3$,
respectively. Therefore, the complexity of the E2E-SCMA is
$\mathcal{O}\left(L_{1}L_{2}\right)$, where $L_{1}=128$ and $L_{2}=64$.

Since the values of $E_{b}/N_{0}$ in training influence the BER performance,
we investigate how training samples generated by different $E_{b}/N_{0}$ can
affect the system performance in Fig. 4. We first train the system at the
fixed $E_{b}/N_{0}$ values, which were set to be $E_{b}/N_{0}=2$ dB, $7$ dB
and $10$ dB, respectively. Then, the system was also trained in the
$E_{b}/N_{0}$ range $\mathcal{U}\left(5,11\right)$ dB. It is clearly shown
that the low $E_{b}/N_{0}$ trained network only performs well in the low
$E_{b}/N_{0}$ range, whereas the high $E_{b}/N_{0}$ trained network will
degrade the performance in the low $E_{b}/N_{0}$ range. A better way is to
train the network in a wide $E_{b}/N_{0}$ range, thus the trained system can
harvest the good performance over a wide range $E_{b}/N_{0}$ .

Figure 2: System performance of E2E-SCMA trained by various $E_{b}/N_{0}$.

Figure 3: BER performance comparison with different decoders.

Figure 4: The BER performance of different codebooks with MPA decoder.

In Fig. 4, we compare the BER performance of the proposed E2E-SCMA scheme with
the AE-SCMA scheme [11], the D-SCMA scheme [13], and the conventional SCMA
scheme with Huawei codebook [17]. The MPA decoder is employed for conventional
SCMA scheme to compare with deep learning designed SCMA system. The results
show that the proposed scheme significantly outperforms all conventional SCMA
schemes. Specifically, the proposed E2E-SCMA achieves $3.5$ dB gain and $1.8$
dB gain over D-SCMA, AE-SCMA scheme at SER $=10^{-5}$, respectively.

To evaluate the codebook obtained by E2E-SCMA scheme, we compare the MED and
corresponding BER performance with MPA decoder with the state of art
codebooks. The MED is obtain by calculating
${{{M}^{J}}\left({{M}^{J}}-1\right)}/{2}$ mutual distances between ${{M}^{J}}$
superimposed codewords, which constitute a superimposed constellation
${{\Phi}}$. Hence, the MED can be expressed as

$\min\left\\{{\parallel\mathbf{v}_{n}-\mathbf{v}_{m}\parallel}^{2},\\!\forall{{\mathbf{v}}_{n}},{{\mathbf{v}}_{m}}\in\Phi,\\!\forall
m,n\in{{Z}_{{{M}^{J}}}},m\neq n\right\\},$

where ${{Z}_{{{M}^{J}}}}$ stands for the integer set
$\left\\{1,2,\ldots,{{M}^{J}}\right\\}$. Specifically, the MED of learned
codebook is compared with Huawei codebook [17], Chen codebook [5] and Yu
codebook[6]. The results are presented in Table I. It can be seen that the
learned codebook owns MED = $1.17$ and is higher than other codebooks. Then,
BER comparisons of different codebooks with MPA decoder are shown in Fig. 4.
The proposed codebook achieves $4.8$ dB gain over the Huawei codebook at BER
$=10^{-5}$, about $1.8$ dB gain over the Yu codebook, and $1$ dB gain over the
Chen codebook at BER $=10^{-5}$. The proposed codebook and the codebooks
employed for comparison are all available at our GuitHub
project333https://github.com/ethanlq/SCMA-codebook/tree/main/CB_autoencoder .

##  V Conclusion

In this paper, we have proposed an E2E-SCMA by joint optimization of SCMA
encoder and decoder with the aid of DAE. Our key idea is to design the SCMA
encoder by taking into account of the mapping procedure and then optimize the
decoder with multi-task learning approach. Simulation results showed that the
use of multi-task learning technique enables efficient derivation of codebook
and decoding strategy for a sparse and multidimensional superimposed signal.
In addition, our proposed scheme outperforms conventional schemes and existing
autoencoder SCMA in terms of both error rate and computational complexity.

TABLE I:  A comparison of MEDs of different codebooks  Codebook | MED
---|---
Huawei [17] | 0.56
Yu [6] | 0.90
Chen [5] | 1.07
Learned codebook | 1.17

## References

  * [1] Z. Liu and L.-L. Yang, “Sparse or dense: A comparative study of code-domain NOMA systems,” _IEEE Trans. Wireless Commun._ , vol. 20, no. 8, pp. 4768–4780, Aug. 2021.
  * [2] Q. Luo _et al._ , “An error rate comparison of power domain non-orthogonal multiple access and sparse code multiple access,” _IEEE Open J. Commun. Soc._ , vol. 2, no. 4, pp. 500–511, Mar. 2021.
  * [3] H. Nikopour and H. Baligh, “Sparse code multiple access,” in _Proc. IEEE 24th Int. Symp. Pers. Indoor Mobile Radio Commun. (PIMRC)_ , London, U.K., Sep. 2013, pp. 332–336.
  * [4] R. Hoshyar, F. P. Wathan, and R. Tafazolli, “Novel low-density signature for synchronous cdma systems over awgn channel,” _IEEE Trans. Signal Process._ , vol. 56, no. 4, pp. 1616–1626, Apr. 2008.
  * [5] Y.-M. Chen and J.-W. Chen, “On the design of near-optimal sparse code multiple access codebooks,” _IEEE Trans. Commun._ , vol. 68, no. 5, pp. 2950–2962, May 2020.
  * [6] L. Yu, P. Fan, D. Cai, and Z. Ma, “Design and analysis of SCMA codebook based on star-QAM signaling constellations,” _IEEE Trans. Veh. Technol._ , vol. 67, no. 11, pp. 10 543–10 553, Nov. 2018.
  * [7] K. Lai _et al._ , “Analyzing uplink grant-free sparse code multiple access system in massive IoT networks,” _IEEE Internet Things J._ , Sep. 2021\.
  * [8] L. Yu _et al._ , “Sparse code multiple access for 6G wireless communication networks: Recent advances and future directions,” _IEEE Commun. Stand. Mag._ , vol. 5, no. 2, pp. 92–99, Apr. 2021.
  * [9] L. Yang, Y. Liu, and Y. Siu, “Low complexity message passing algorithm for SCMA system,” _IEEE Commun. Lett._ , vol. 20, no. 12, pp. 2466–2469, Dec. 2016.
  * [10] Z. Mheich, L. Wen, P. Xiao, and A. Maaref, “Design of SCMA codebooks based on golden angle modulation,” _IEEE Trans. Veh. Technol._ , vol. 68, no. 2, pp. 1501–1509, Feb. 2018.
  * [11] J. Lin _et al._ , “A novel deep neural network based approach for sparse code multiple access,” _Neurocomputing_ , vol. 382, no. 8, pp. 52–63, Mar. 2020.
  * [12] T. O’shea and J. Hoydis, “An introduction to deep learning for the physical layer,” _IEEE Trans. Cogn. Commun. Netw._ , vol. 3, no. 4, pp. 563–575, Dec. 2017.
  * [13] M. Kim, N.-I. Kim, W. Lee, and D.-H. Cho, “Deep learning-aided SCMA,” _IEEE Commun. Lett._ , vol. 22, no. 4, pp. 720–723, Jun. 2018.
  * [14] M. Han, H. Seo, A. T. Abebe, and C. G. . Kang, “Deep learning-based multi-user multi-dimensional constellation design in code domain non-orthogonal multiple access,” in _Proc. IEEE Int. Conf. Commun. (ICC)_ , Dublin, Ireland, Jun. 2020, pp. 1–6.
  * [15] C.-P. Wei _et al._ , “SCMA decoding via deep learning,” _IEEE Wireless Commun. Lett._ , vol. 10, no. 4, pp. 878–881, Apr. 2021.
  * [16] Y. Zhang and Q. Yang, “A survey on multi-task learning,” 2017, arXiv: 1707.08114. [Online]. Available: https://arxiv.org/abs/1707.08114
  * [17] Altera Innovate Asia website, Presentation, “1st 5G algorithm innovation competition-env1.0-SCMA.” [Online]. Available: http://www.innovateasia.com/5G/en/gp2.html.