Component Training of Turbo Autoencoders

Jannis Clausius, Marvin Geiselhart and Stephan ten Brink

Institute of Telecommunications, University of Stuttgart, Pfaffenwaldring 47, 70659 Stuttgart, Germany

This work is supported by the German Federal Ministry of Education and Research (BMBF) within the project Open6GHub under grant 16KISK019 and the project FunKI  under grant 16KIS1187.

Isolated TGP of the component autoencoders of turbo-autoencoder architectures  enables faster, more consistent training and better generalization to arbitrary decoding iterations than training based on deep unfolding.
We propose fitting the components via EXIT charts to a desired behavior which enables scaling to larger message lengths ($k \approx 1000$) while retaining competitive performance.
To the best of our knowledge, this is the first autoencoder that performs close to classical codes in this regime.
Although the BCE loss function optimizes the BER of the components, the design via EXIT charts enables to focus on the BLER.
In serially concatenated systems the component-wise TGP approach is well known for inner components with a fixed outer binary interface, e.g., a learned inner code or equalizer, with an outer binary error correcting code.
In this paper we extend the component training to structures with an inner and outer autoencoder, where we propose a new 1-bit quantization strategy for the encoder outputs based on the underlying communication problem.
Finally, we discuss the model complexity of the learned components during design time (training) and inference and show that the number of weights in the encoder can be reduced by $99.96\,\%$.

§ INTRODUCTION

Concatenation of blocks, or modularization, is a crucial concept in engineering that breaks down complex systems into smaller, more manageable components to simplify the design process.
In communications engineering, iterating between the concatenated blocks of a receiver gives rise to an even more powerful concept: the turbo principle <cit.>. Here, two or more serially concatenated blocks iteratively exchange extrinsic information.
Since its introduction in turbo decoders <cit.>, the principle found application in many different scenarios, such as turbo equalization of multipath channels, MIMO detection, BICM and LDPC decoding <cit.>.
These systems show outstanding performance close to the theoretical limits in scenarios closely matching the models assumed in the design, e.g., the channel model.
However, in more complex situations or scenarios with hardware impairments or when no suitable channel models is known, machine learning-aided communication systems that are trained to optimize the end-to-end performance can outperform classical systems <cit.>. Still, the well-understood AWGN scenario serves as a valuable benchmark and can be seen as the “worst case” scenario for the learning-based systems.
Like in conventional communications, the turbo principle has been successfully applied in form of the turbo-autoencoder, called TurboAE, to scale deep-learning based transceivers to more practical block lengths <cit.>.
The TurboAE consists of parallel or serially concatenated CNN at the transmitter. At the receiver, information is passed iteratively between another set of two CNN. It is typically trained in a deep unfolded fashion, i.e., the iterations of the receiver are unrolled into a deep NN, optimally adapting the constituent NN to the iterative algorithm.
However, this comes at the cost of increased training complexity which scales linearly with the number of iterations.
To overcome this issue, component TGP has been proposed in <cit.>.

In this paper we consider component-wise autoencoder training for the serial TurboAE.
The main contributions of this paper are as follows:

* We apply EXIT charts <cit.> to analyze and optimize the serial TurboAE for BLER or large block lengths by finetuning the inner component to the outer autoencoder <cit.>.

* A Gaussian prior training framework for the serial TurboAE is proposed and demonstrated to drastically reduce the training complexity.

* A new binarization strategy for an encoder output layer (BPSK modulation) is proposed based on the underlying communication problem.

* The encoder networks are distilled down to just 148 weights by a student-teacher method without performance degradation, enabling practical implementations.

§ PRELIMINARIES

§.§ Densely Connected Convolutional Layers

DCCNN <cit.> allow the training of deeper structures with fewer weights than plain CNN.
A DCCNN consists of blocks of convolutional layers with an increased amount of connections, and transition layers.
The input to each densely connected convolutional layer is not only the output from the last layer, but a concatenation of all preceding feature maps within a block.
This means the number of feature maps for inputs increases with a growth parameter $F$, which is the number of output feature maps per layer.
Furthermore, each layer consists of a batch normalization, a ReLU activation and a convolutional layer with kernel size $K$.
Every densely connected block proceeds a transition layer.
The output of the transition layer is the only input to the next densely connected block.
It interrupts the growth process of the number of feature maps.
The transition layer is a $1\times1$ convolutional layer that outputs $F_0$ feature maps.

§.§ Serial Turbo Autoencoder

The serial TurboAE <cit.> is based on the structure of serially concatenated codes with an iterative (turbo) decoding architecture.
However, the encoders and decoders are implemented as CNN.
The system model is shown in Fig. <ref>.
The encoder consists of an outer encoder, an interleaver and an inner encoder.
The corresponding decoder is based on a CNN for decoding the inner code and a CNN for decoding the outer code.
Again, both are connected by an interleaver and a deinterleaver.
For training, the iterations of the decoder are unfolded, yielding a deep neural decoder with intermediate (de-)interleaver layers.
The difference to the straight-forward training with SGD is the alternating training schedule for the encoder ($T_\mathrm{TX}$ updates) and the decoder ($T_\mathrm{RX}$ updates).
For further details, the reader is referred to <cit.>.

box =[rectangle,  draw=black, thick, align=flush center,minimum width=1.2cm, minimum height=1.2cm, text centered];
intbox =[rectangle,  draw=black, thick, align=flush center,minimum width=0.9cm, minimum height=0.9cm, text centered];
arrow = [thick,->]

[] (start) at (0,0) ;
[box] (enco) at (1.2,0) Enc O
[intbox] (inte) at (2.6,0) $\pi$;
[box] (enci) at (4,0) Enc I
[box, rot,  text=black] (channel) at (5.8,0) Channel;
[box] (deci) at (7.6,0) Dec I
[intbox] (int1) at (8.8,0.9) $\pi^{-1}$;
[intbox] (int2) at (8.8,-0.9) $\pi$;
[box] (deco) at (10,0) Dec O
[] (end) at (11.2,0) ;

[arrow] (start) – (enco);
[arrow] (enco) – (inte);
[arrow] (inte) – (enci);
[arrow] (enci) – (channel);
[arrow] (channel) – (deci);
[arrow] (deci) |- (int1);
[arrow] (int1) -| (deco);
[arrow] (deco) |- (int2);
[arrow] (int2) -| (deci);
[arrow] (deco) – (end);

[draw=apfelgruen, dashed] (0.3,1.45) rectangle (4.8,-1.45);
[draw=mittelblau, dashed] (6.7,1.45) rectangle (10.8,-1.45);

[] at (2.6,1.7) Encoder;
[] at (8.8,1.7) Decoder;

Structure of the CNN-based serial TurboAE communication system consisting of a real-valued encoder, channel, and an iterative decoder.

§.§ Training with Gaussian Priors

TGP <cit.> is a method to reduce complexity during training for systems with concatenated components in contrast to unfolding the decoder.
The idea is to train each component individually and isolated with a single decoding iteration.
Only during inference the components are concatenated as in the desired system.
The alignment of the components can be done with EXIT charts.
For training, the a priori information from the other component has to be generated artificially.
The generation process is well known under two conditions. First, the information, in form of LLR, is Gaussian distributed. Second, the variable for which we generate the a priori LLR consists of bits.
The a priori LLR distribution $L_u^\mathrm{A}$ for a bit $u$ and a set a priori information $I_\mathrm{A}$ is then given by

\begin{align}
\label{eq:generate_prior}
L_u^\mathrm{A} &\sim \mathcal{N}\left((2u-1)\cdot \mu(I_\mathrm{A}), 2\mu(I_\mathrm{A})\right)
\end{align}

where $\mu(I_\mathrm{A})\approx\frac{1}{2}\left(-\frac{1}{H_1}  \log_2 \left( 1-I_\mathrm{A}^{\frac{1}{H_3}}  \right)  \right)^\frac{1}{H_2}$ with constants $H_1=0.3073$, $H_2=0.8935$ and $H_3=1.1064$ <cit.>.

§.§ EXIT Charts

EXIT charts <cit.> are a design tool for iterative, component based systems.
The key idea is to calculate the input/output behavior of each component and predict the decoding trajectory of the concatenated system.
The behavior is displayed by the produced extrinsic information from a certain a priori information.
The a priori LLR can be generated according to (<ref>) and the information between LLR $\Lm$ and bits $\uv$ can be approximated <cit.> by

\begin{equation}
I(\Lm, \uv) \approx 1- \sum_i \log_2 \left( 1+\exp(-(2u_i-1) \cdot L_i) \right).
\end{equation}

The chart shows these two curves where one is flipped along the first bisector.
This means that the x-axis shows the a priori information of one component and the extrinsic information from the other component and vice versa for the y-axis.
The decoding trajectory can be estimated by iterating “ping-pong”-wise between the two curves.
The intersection of the curves indicates the maximum reachable mutual  information by iterating between the components.
However, the estimated trajectory is only accurate if the exchanged information is uncorrelated between the components.
This does not hold for short block lengths, as cycles in the decoding graph lead to correlated information exchange.
A more appropriate design tool for short block lengths are scattered EXIT charts <cit.>.
Here, the chart consists of many trajectories displayed as a scatter plot.

§ COMPONENT TRAINING FOR SERIAL ARCHITECTURES

§.§ Identifying Component Interfaces

box =[rectangle,  draw=black, thick, align=flush center,minimum width=1.2cm, minimum height=1.2cm, text centered];
arrow = [thick,->]

[] (start) at (0.5,0) ;
[box] (enc) at (2,0) Enc I
[dspadder] (channel) at (3.5,0) ;
[] (n) at (3.5,-1.5) $\mathcal{N}(0,\sigma^2)$;
[] (la) at (6,-1.5) $\mathcal{N}\left(\cv  \mu(I_\mathrm{A}), 2\mu(I_\mathrm{A})\right)$;
[box] (dec) at (6,0) Dec I
[] (end) at (7.5,0) ;

[arrow] (start)– node[midway, above]$\cv$node[below]$\in\!\!\mathbb{F}_2^{n}$ (enc);
[arrow] (enc)– node[midway, above]$\xv$node[below]$\in\!\!\RR^{n}$ (channel);
[arrow] (channel)– node[midway, above]$\yv$ (dec);
[arrow] (dec)– node[midway, above]$\Lm^\mathrm{T}_\cv$ (end);
[arrow] (la)– node[midway, left]$\Lm^\mathrm{A}_\cv$ (dec);
[arrow] (n) – node[midway, left]$\nv$ (channel);

Inner autoencoder with Gaussian prior channel

box =[rectangle,  draw=black, thick, align=flush center,minimum width=1.2cm, minimum height=1.2cm, text centered];
arrow = [thick,->]

[] (start) at (0.1,0) ;
[box] (enc) at (1.6,0) Enc O
[box] (bin) at (3.7,0) $\operatorname{sign}(\cdot)$;
[dspadder] (channel) at (5.8,0) ;
[] (n) at (5.8,-1.5) $\mathcal{N}(0,\sigma^2)$;
[box] (demap) at (7,0) BPSK
[box] (dec) at (9,0) Dec O
[] (end1) at (10.5,0.2) ;
[] (end2) at (10.5,-0.2) ;

[arrow] (start) – node[midway, above]$\uv$node[below]$\in\!\!\mathbb{F}_2^{k}$  (enc);
[arrow] (enc) – node[midway, above]$\cv_\mathbb{R}$node[below]$\in\!\!\RR^{n}$ (bin);
[arrow] (bin) – node[midway, above]$\cv$node[below]$\in\!\!\{\pm1\}^{n}$ (channel);
[arrow] (channel) – (demap);
[arrow] (demap) – node[midway, above]$\Lm^\mathrm{A}_\cv$ (dec);
[arrow] (dec.east |- end1) – node[midway, above]$\Lm^\mathrm{T}_\uv$ (end1);
[arrow] (dec.east |- end2) – node[midway, below]$\Lm^\mathrm{E}_\cv$ (end2);
[arrow] (n) – node[midway, left]$\nv$ (channel);

Outer autoencoder

Block diagrams of the (non-iterative) autoencoders.

The serial TurboAE can be interpreted as an inner autoencoder, and an outer autoencoder with BPSK modulation and an virtual channel in between.
This virtual channel consists of the inner autoencoder and the actual channel, but can be abstracted to an AWGN channel with an certain SNR.
The concept is shown in Fig. <ref>.
The goal of the component training is to optimize each autoencoder isolated.
Thus, we first identify the components and their interfaces.
For a serial TurboAE with rate $R=\frac{k}{n}= R_\mathrm{O}\cdot R_\mathrm{I}=\nicefrac{1}{2}$ we choose the outer and inner autoencoder to have rate $R_\mathrm{O}=\nicefrac{1}{2}$ and $R_\mathrm{I}=1$, respectively.
<cit.> showed that this allocation of the rate is optimal for asymptotic block lengths and <cit.> observed it empirically.
The inner autoencoder is shown in Fig. <ref>.
The input to the inner decoder are the real-valued channel observations $\yv$ and the available a priori information in form of LLR $\Lm^\mathrm{A}_\cv$ from the (artificial) outer decoder.
The output are the extrinsic LLR $\Lm^\mathrm{E}_\cv = \Lm^\mathrm{T}_\cv - \Lm^\mathrm{A}_\cv$, where the total LLR $\Lm^\mathrm{T}_\cv$ are the direct output of the NN.
As loss, we choose a BCE loss between $\cv$ and $\Lm^\mathrm{T}_\cv$.
To generate artificial a priori LLR for the inner autoencoder, the outer autoencoder must use BPSK symbols.
The outer autoencoder is shown in Fig. <ref>.
The input to the outer decoder are the a priori LLR of the coded bits $\Lm^\mathrm{A}_\cv$.
The outputs are not only the uncoded bit estimates $\Lm^\mathrm{T}_\uv$, but also the refined extrinsic LLR $\Lm^\mathrm{E}_\cv = \Lm^\mathrm{T}_\cv - \Lm^\mathrm{A}_\cv$, where $\Lm^\mathrm{T}_\cv$ is the direct output of the NN.
The outputs can be trained with a BCE loss between $\uv$ and $\Lm^\mathrm{T}_\uv$, and $\cv$ and $\Lm^\mathrm{T}_\cv$, respectively.
The losses are added without a weighting factor, as they are in the same order of magnitude.

§.§ Autoencoder with Binary Modulation

Influence of clipping on the learned modulation.

Forcing a BPSK modulation can be seen as a 1 bit quantization of the encoder output layer. The authors of
<cit.> propose to first train the output layer with real valued outputs and apply binarization with a  STE for gradient computation afterwards.
This procedure outperforms training a model from scratch with  binarization and a  STE for gradient computation.
Here, we propose a new quantization strategy based on the underlying communication problem.
Similar to <cit.>, we first train with a real valued output layer and only start the quantization process once the system is trained.
For the latter, we propose to clip $\xv$ with a threshold $x_\mathrm{clip}$, which is gradually decreased from $1.5$ to $1.0$ in steps of $0.1$ every $10$ epochs.
The key idea is that the clipping is applied after normalization of $\xv$, i.e., reducing the energy of the transmission.
Consequently, the encoder tries to compensate for this energy loss by increasing the probability of symbols with larger magnitude and decreasing the probability of symbols with smaller magnitudes.
Once $x_\mathrm{clip}\approx 1$, the encoder must converge to a BPSK modulation to transmit with the maximum possible energy.
For inference we use a binarizer.
The advantage of this approach is that no STE is used, thus, no gradient mismatch between forward and backward path during training.
The concept and intuition is visualized in Fig. <ref>.

§.§ Fitting Components via EXIT Charts

The outer autoencoder is trained as in <cit.>, where it was found that training in the waterfall region leads to the best overall performance.
This means that we train the outer autoencoder to be as good as possible and then fit the inner autoencoder to the resulting outer autoencoder.
Moreover, the proposed design via EXIT charts allows to fit autoencoders for large block lengths.
We can train the CNN-based autoencoders on short block lengths with a certain EXIT behavior and evaluate the concatenated autoencoders on large block lengths to achieve scaling to large block lengths.
Similar to <cit.> we propose a two step training process.
First, we train the inner autoencoder with high a priori information $I_\mathrm{A}>0.8$.
The training process is stopped, once the slope of the EXIT characteristic starts increasing (it starts out flat).
Second, set a certain fraction $\alpha$ of each batch to $I_\mathrm{A}=0$ and continue training.
As a result, the autoencoder is optimized for high a priori information, which is to be expected from the outer lower rate code, but a reasonably high decoding performance without a priori information to start the iterative decoding.
We observed that the fraction $\alpha$ directly relates to the slope of the EXIT characteristic.
A lower $\alpha$ leads a to steeper slope and vice versa.
This is visualized in Fig. <ref> for $\alpha=0.075$ and $\alpha=0.085$ at $E_\mathrm{b}/N_0=4\,$dB.
Further, another inner autoencoder is shown for $\alpha=0.08$ at $E_\mathrm{b}/N_0=2\,$dB.
Lastly, two trajectories are shown for an interleaver length of $n=128$ and $n=2048$.
We can see that the trajectory for $n=128$ does not converge and Fig. <ref> reveals that even for $\alpha=0.075$ at $E_\mathrm{b}/N_0=4\,$dB, the trajectory corridor is quite small.
This means the for $n=128$ the inner and outer autoencoder are fitted nicely.
The other trajectory in Fig. <ref> for $n=2048$ converges to the intersection point which is just shy of $1.0$, determining the best possible performance at $E_\mathrm{b}/N_0=2\,$dB even for $n \to \infty$.

0.87![spy using outlines=rectangle, magnification=6, connect spies]
xmin = 0,
mark size=0.0005pt,
legend style=at=(axis cs:0,1),anchor=north west,
minor grid style=gray!25,
major grid style=gray!25,
cycle list name=corporate colours markers,
line width=1pt,
each nth point=1,
filter discard warning=false,
unbounded coords=discard,
legend cell align=left,

+ [lila, dotted]
table[x=x ,y=y,col sep=comma]./tikz/results/ae_128_precursor_small_snr2_065_precursor_better_decoder_finetune/traj_n128.txt;

+ [lila, dashed]
table[x=x ,y=y,col sep=comma]./tikz/results/ae_128_precursor_small_snr2_065_precursor_better_decoder_finetune/traj_n2048.txt;

+ [mittelblau, dashed]
table[x=mi ,y=e1,col sep=comma]./tikz/results/ae_128_precursor_17epoch_zpa_075/exit.txt;

+ [mittelblau, dotted]
table[x=mi ,y=e1,col sep=comma]./tikz/results/ae_128_precursor_17epoch_zpa_085/exit.txt;

+ [mittelblau]
table[x=mi ,y=e1,col sep=comma]./tikz/results/ae_128_precursor_small_snr2_065_precursor_better_decoder_finetune/exit.txt;

+ [rot]
table[x=e2 ,y=mi,col sep=comma]./tikz/results/ae_128_precursor_small_snr2_065_precursor_better_decoder_finetune/exit.txt;

[-latex, line width=1.0pt, color=rot] (axis cs:0.325,  0.1) – node[below,pos=0.01,black]Outer autoencoder $R_\mathrm{O}=\frac{1}{2}$ (axis cs:0.15, 0.36);

[-latex, line width=1.0pt, color=lila] (axis cs:0.77,  0.55) – node[below,pos=0.01,black]$n=2048$ (axis cs:0.77, 0.63);

[-latex, line width=1.0pt, color=lila] (axis cs:0.6,  0.5) – node[below,pos=0.01,black]$n=128$ (axis cs:0.6, 0.61);

[-latex, line width=1.0pt, color=mittelblau] (axis cs:0.1,  0.61) – node[below,pos=0.01,black]$\alpha=0.075$ (axis cs:0.1, 0.7);

[-latex, line width=1.0pt, color=mittelblau] (axis cs:0.1,  0.83) – node[above,pos=0.01,black]$\alpha=0.085$ (axis cs:0.1, 0.74);

[rotate=10] at  (axis cs:0.6, 0.84) $E_\mathrm{b}/N_0=4\,\text{dB}$;

[rotate=22] at  (axis cs:0.5, 0.65) $E_\mathrm{b}/N_0=2\,\text{dB}$, $\alpha=0.08$;

[mittelblau, line width=.7pt] (axis cs:0.9, 0.82) ellipse (0.35cm and 0.65cm);
at  (axis cs:0.78, 0.95) Inner autoencoder $R_\mathrm{I}=1$;

(spypoint) at (axis cs:0.99
(spyviewer) at (axis cs:0.75,0.25);

[width=3cm,height=2cm, fill,fill opacity=0.9] on   (spypoint) in node [fill=white] at (spyviewer);

EXIT chart of the learned components.

Scattered EXIT chart for ($k=64$, $n=128$). Each point represents an encountered input/output mutual information in a trajectory.

§.§ Discussion Component vs Unfolded Training

A problem with unfolded training is the computational complexity, as
every unfolded iteration runs thorugh a forward and a backward pass.
In contrast, TGP is based on a single iteration and, thus, saves computational complexity and time by a factor of the number of iterations $N_\mathrm{it}$ in the decoder.
While the unfolding increases complexity, it provides the opportunity for the decoder to mitigate short block length effects, i.e., short cycles in the graph and correlated a priori information.
As the TGP trained decoder never experiences correlated a priori information during training, it cannot mitigate these effects.
An advantage of TGP, in terms of convergence over training epochs, is that the input to the decoders during training is always of good quality, since the a priori LLR are not output of a previous (potentially untrained) decoder.
Therefore, the TGP approach benefits from a fast convergence, especially at the beginning of training.
Lastly, TGP does not directly optimize the BER as unfolded training does <cit.>.
While the decoders optimize the BER in every iteration, the EXIT behavior of the components determine the performance in terms of BER and BLER.
We observed that inner components with a steeper slope tend to perform better in terms of BLER and worse in BER than an inner components with a more gentle slope.
The steeper slope of the inner component leads to a later intersection with the outer component but a tighter corridor for the trajectory, as shown in Fig. <ref>.
Thus, the outer component can output better bit estimates, if the trajectory converges.
However, the trajectory converges less often due to the tighter corridor.

§ RESULTS

For the serial TurboAE the encoders are plain CNN and the decoder are DCCNN with hyperparameters for architecture and training as in Tab. <ref>.
For all evaluations we use interleavers according to the LTE standard <cit.>.

Hyperparameters for architecture and training the serial TurboAE

Parameter    Value

CNN Layers     5

CNN $F,K$     $100,5$

DCCNN Blocks     $3$

Layers per Bl.     $3$

1. DCCNN $F_0,F,K$     $16,12,5$

2. DCCNN $F_0,F,K$     $16,16,5$

3. DCCNN $F_0,F,K$     $16,12,5$

$(k,n)$     $(64,128)$

Padding     Circ. <cit.>

Parameter    Value

Loss     BCE

$T_{\mathrm{TX}}$, $T_{\mathrm{RX}}$      $100$, $500$

Batch size     $500-2000$

Learning rate     $10^{-4}-10^{-6}$

Encoder SNR     $4.0\,\mathrm{dB}$

Enc. $I_\mathrm{A}$     $0.8-1$

Decoder SNR     $0.5-4.0\,\mathrm{dB}$

Dec. $I_\mathrm{A}$     $0.6-0.9$

$\alpha$     $0.075$

§.§ Decoding Performance for Large Block Lengths

BER over an AWGN channel for long codes ($R=0.5$), based on short ($n=128$) component codes.

Fig. <ref> shows the BER of the serial TurboAE with TGP compared to the LTE turbo code for large block lengths ($k\in\{800,\,1024\}$, $n\in\{1600,\,2048\}$, $R=\nicefrac{1}{2}$).
To the best of our knowledge, this is the first time that a competitive performance of an autoencoder is demonstrated  for a message length of $k \approx 1000$.
The performance for the serial TurboAE is evaluated with the outer autoencoder and inner autoencoder ($E_\mathrm{b}/N_0=2\,\text{dB}$, $\alpha=0.08$) shown in Fig. <ref>.
For the evaluation we increased the number of inputs to the CNN and increased the size of the interleaver.
Also the turbo-product framework <cit.> can be used in case the NN are not based on CNN.
Note, the components are trained with $k=64,~n=128$ and are not trained or finetuned for longer lengths.
While the performance is roughly $0.1\,$dB worse than the LTE turbo code in the low SNR regime, the serial TurboAE shows an error floor for higher SNR.
This error floor behavior is due to the intersection in Fig. <ref> being just shy of a mutual information of $1.0$.

§.§ Comparison of TGP and Unfolded Training

BLER over an AWGN channel ($k=64$, $n=128$, $N_\mathrm{it}=6$).

The decoding performance after training is shown in Fig. <ref> in terms of BLER for short block lengths ($k=64$, $n=128$, $R=\nicefrac{1}{2}$).
The serial TurboAE with TGP and $N_\mathrm{it}=6$ iterations performs slightly better than the serial TurboAE with unfolded training.
Both perform marginally better than the LTE turbo code.
Further, the outer code with the proposed gradual clipping outperforms the STE-based proposition from <cit.>.
A more prominent advantage shows the training behavior which is displayed in Fig. <ref>.
The figure compares the valdiation BER of the components with TGP and the unfolded training. Note, the BER of the outer code is without iterative decoding and the inner code is with iterative decoding.
Both need significantly fewer epochs for training than the unfolded training.
Finally, we observed that component-wise TGP is more robust in a sense that the component autoencoders consistently converged to the same shown performance, while for the unfolded training, the shown results are from the best run out of many.

§ DISCUSSION ON MODEL COMPLEXITY

A valid criticism of autoencoders for channel coding is the increased computational and memory complexity.
The proposed systems <cit.> consists of a huge amount of weights, see Tab. <ref>.
However, the complexity is simply outside the scope of these works.
Therefore, these works can be seen as design time systems, where a great exploration space is needed to find good solutions.
This exploration space of a NN is usually coupled to the number of trainable weights.
However, once a good solution is found, the knowledge can be distilled into smaller NN <cit.>.
The intuition is that the solution is inside the solution space of the larger and the smaller NN.
However, the smaller NN can not converge to the solution, as the needed convergence trajectory is not inside the solution space.
A common knowledge distillation approach is the student-teacher approach.
Here, the student is influenced by a regression loss with the outputs of the teacher.
We applied this approach to the encoder with an MSE loss.
A smaller CNN-based encoder with a total of $148$ weights (reduction by $99.96\,\%$) for the inner and outer encoder in total was able to represent the solution from the large encoder without any loss in performance.

Approximate number of weights

Name    Encoder    Decoder     Layer type

TurboAE <cit.>    $3.0\cdot10^4$     $2.4\cdot10^6$     Conv.

Ser. TurboAE <cit.>    $4.1\cdot10^5$     $2.4\cdot10^6$    Conv.

ProductAE <cit.>     $2.8\cdot10^5$     $1.9\cdot10^6$    Dense

This work     $148$     $1.2\cdot10^5$    Conv.

Convergence speed comparison in terms of BER vs. training epochs ($k=64$, $n=128$).

§ CONCLUSION

We introduced the component-wise TGP for the serial TurboAE and demonstrated a faster and more consistent training.
By training the components to match a desired EXIT curve, the resulting serial TurboAE shows a competitive BER for message lengths of $k\approx 1000$.
To the best of our knowledge, this is the first autoencoder that performs close to classical codes in this regime.
Furthermore, we proposed a new quantization strategy to force a trainable encoder to a BPSK modulation.
Lastly, we demonstrated a reduction of the number of weights by $99.96\,\%$ in the encoder after training.
Future works could include the exploration of different autoencoders as components and complexity reduction in the decoder.


# NOMA Assisted Multi-MEC Offloading for IoVT Networks

Fengqian Guo, Hancheng Lu, Bo Li, Dingxuan Li, and Chang Wen Chen

## Abstract

Nowadays, Internet of Video Things (IoVT) grows rapidly in terms of quantity
and computation demands. In spite of the higher local computation capability
on visual processing compared with conventional Internet of Things devices,
IoVT devices need to offload partial visual processing tasks to the mobile
edge computing (MEC) server wirelessly due to its larger computation demands.
However, visual processing task offloading is limited by uplink throughput and
computation capability of the MEC server. To break through these limitations,
a novel non-orthogonal multiple access (NOMA) assisted IoVT framework with
multiple MEC servers is proposed, where NOMA is exploited to improve uplink
throughput and MEC servers are co-located with base stations to provide enough
computation capability for offloading. In the proposed framework, the
association strategy, uplink visual data transmission assisted by NOMA and
division of the visual processing tasks as well as computation resource
allocation at the MEC servers are jointly optimized to minimize the total
delay of all visual processing tasks, while meeting the delay requirements of
all IoVT devices. Simulation results demonstrate that significant performance
gains can be achieved by proposed joint optimization with NOMA transmission
and multi-MEC offloading in the heterogeneous IoVT network.

Fengqian Guo, Hancheng Lu, Bo Li and Dingxuan Li are with University of
Science and Technology of China; Chang Wen Chen is with the Department of
Computing, Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong SAR,
China.

## Introduction

The Internet of Things (IoT) technology is developing rapidly. Billions of
connected sensors have been deployed for various applications. Among them, IoT
with visual sensors (e.g, cameras in smartphones, vehicles and buildings) have
emerged into a new sub-field of IoT called Internet of Video Things (IoVT)[1],
or visual IoT[2]. With unique characteristics in terms of sensing,
transmission, storage, and analysis, IoVT has gained much attention from
academia and industry. It has been applied in specific scenarios like public
security, traffic analysis, industrial automation and smart city [7, 4, 3, 5],
which imposes high requirements on computation and transmission resources.

The most distinctive feature of IoVT lies in processing of sensored visual
data. Most IoVT applications or visual IoT applications rely on visual
processing to achieve specific objectives, for example, recognition and
tracking. In [8], to analyze the human action in sport training videos, a deep
learning based action recognition method is proposed . The proposed method
utilizes mathematical models like neural networks and image processing
techniques to sense the environment. A compact deep neural network-based face
recognition method for face recognition in public safety surveillance system
with visual IoT is presented in [6]. A novel visual IoT architecture
considering the interaction among visual IoT devices is proposed in [2] to
improve end-to-end performance for next generation smart cities. In [9], low-
power and real-time multiple object visual tracking is implemented in an IoVT
network with a camera and wireless connection capability.

Different from conventional IoT devices, in general, IoVT devices have higher
local computation capability for visual processing [1]. In spite of this,
visual processing task offloading is still necessary in IoVT networks, due to
the huge computation requirements on visual processing that cannot be met by
IoVT devices. It has been proposed to offload visual processing tasks to the
mobile edge computing (MEC) server with tolerant delay [4, 7]. To implement
MEC offloading, MEC servers are deployed at the edge of radio access networks
(e.g., base station (BS)). Limited by the cost and hardware factors, IoVT
devices may not be able to finish all of their visual processing tasks by
themselves on time. In this case, IoVT devices can offload part of their
visual processing tasks to MEC servers [1]. Usually, IoVT devices tend to
handle the visual processing tasks with higher delay sensitive and lower
computation demands locally, and offload the visual processing tasks with
lower delay sensitive and higher computation demands to MEC servers.

Many research attempts have been done on MEC offloading in IoVT networks,
where delay and energy consumption are mostly concerned. In [10], tremendous
visual data is reduced by deep learning models in computational IoVT devices
and the transmission function is optimized for high efficiency, with the goal
to obtain the least transmission delay for MEC offloading under dynamic uplink
bandwidth. To reduce system energy consumption considering the resource
constraints of the IoVT devices and MEC server, authors in [7] optimize the
content generation rates of IoVT devices to make full use of the computation
resources at the MEC server. For urban traffic surveillance characterized by
low delay and real-time processing, the dynamic video stream processing scheme
is investigated in [3], aiming at reducing the processing time at the MEC
server. In [11], the MEC offloading policy is optimized by differential
treating low-to-high workloads of visual data processing so that the IoVT
requirements on energy consumption and processing delay can be satisfied.
Particularly, the MEC server gives priority to the IoVT device desiring energy
conservation over low latency with lower workload. To meet the delay and
energy constraints in the IoVT network, authors in [12] present a generic
architecture of an IoVT based video surveillance system, where the image-video
compression and coding tasks are offloaded to the MEC server. In order to
satisfy the low latency and low energy consumption requirements of visual IoT
devices, the author of [4] propose a visual IoT architecture for MEC
offloading, where the conventional multimedia streaming is changed to feature-
type transmission.

However, aforementioned work only considers single MEC server offloading in
IoVT networks and might not provide enough computation resources to meet the
visual processing task offloading demands. Compared with traditional IoT, IoVT
produces a much larger amount of data for processing. Moreover, processing of
visual data is much more complex than that of common IoT data. In the context
of MEC offloading, it means that IoVT imposes much more requirements on
computation resources. This observation motivates us to consider the IoVT
scenario where multiple MEC servers are deployed with BSs and investigate
multi-MEC offloading. Another issue is the limitation of wireless uplink
throughput for visual data transmission. Most existing work uses compression
technologies to reduce the amount of transmitted visual data, leading to
additional pressure on computation resources and energy consumption at the
IoVT devices. Differently, we introduce non-orthogonal multiple access (NOMA)
[13, 14] in the IoVT network to achieve more wireless uplink throughput.
Particularly, power allocation is optimized with consideration of the power
constraints at the IoVT devices.

In this article, we propose a novel NOMA assisted IoVT framework with multi-
MEC. In the proposed framework, considering the heterogeneity of wireless
channel conditions as well as computation capability at IoVT devices and MEC
servers, we perform jointly optimization on the association strategy, the
power allocation strategy of uplink NOMA, division of visual processing tasks
and the way that MEC servers allocate their computation resources, to minimize
the total delay of all visual processing tasks while meeting the delay
requirements of all IoVT devices.

The rest of this article is organized as follows. An IoVT network with multi-
MEC as well as the challenges for multi-MEC offloading are firstly described.
Then, to address these challenges, a novel NOMA assisted IoVT framework with
multi-MEC is proposed and optimization is performed in the proposed framework,
followed by performance evaluation and analysis. Finally, we conclude this
article and discuss future research issues.

## Challenges of IoVT Networks with Multi-MEC

Figure 1: A heterogeneous IoVT system with multi-MEC.

An IoVT scenario with multiple BSs is illustrated in Fig.1, where each BS is
deployed with an MEC server. To show the heterogeneity of the IoVT network,
different IoVT devices have different computation capability and request
different visual processing tasks. The computation demands and delay sensitive
of the visual processing tasks are different, and are indicated by different
lengthes and different colors of the rectangles beside IoVT devices,
respectively. Computation capability of different IoVT devices are indicated
by different heights of the cylinders beside IoVT devices. To meet the
computation and delay demands of the visual processing tasks, IoVT devices
offload partial visual processing tasks to MEC servers located at BSs.

In order to reduce the total delay as well as to make full use of computation
resources at multiple MEC servers and IoVT devices in the heterogeneous IoVT
network, there are several challenges to be addressed. These challenges can be
summarized as issues on MEC association, visual data uplinking and task
division, corresponding to three problems (i.e., where to offload? how to
offload? and which to be offloaded?).

MEC Association: To meet the huge computation requirements from visual
processing, multiple MEC servers are deployed with BSs. In such multi-MEC IoVT
network, the first problem for each IoVT device is where the visual processing
tasks should be offloaded.

As each MEC server is co-located with an BS, the MEC association problem is
equivalent to the user association problem in wireless networks. Traditional
association strategies are based on channel conditions, where a user
associates to an BS with the nearest distance or the strongest signal
strength. However, in the context of multi-MEC offloading, these association
strategies might result in unbalanced computation load among MEC servers and
thus increase the possibility of offloading failures. For example, for an IoVT
device, the MEC server co-located with its nearest BS might not has sufficient
computation resources to satisfy its visual processing task offloading demand,
while some other MEC servers within its association range has enough
computation resources. Therefore, to achieve better utilization efficiency of
computation resources for MEC servers and reduce offloading failures, besides
channel conditions, visual processing task offloading demands of different
IoVT devices and computation resources at different MEC servers should also be
jointly considered in designing the association strategy for multi-MEC
offloading.

Visual Data Uplinking: In the IoVT network, there is a contradiction between
limited wireless uplink throughput and huge visual data transmission
requirements. Hence, the problem of how to offload required visual data to MEC
servers over wireless links is critical for visual processing task offloading,
which is also referred to as visual data uplinking problem.

Most existing work focuses on compression based uplinking technologies where
visual data is compressed before offloaded to MEC servers. However, visual
data compression will incur additional computation resources and power
consumption at the IoVT devices. Another way to address the visual data
uplinking problem is to increase uplink throughput by improving the resource
utilization efficiency of wireless links. In this way, one of the most
promising technologies is NOMA. With NOMA transmission, multiple signals are
superposed and transmitted simultaneously using the same time/frequency
resource. There have been some works that combine NOMA with MEC in the
traditional wireless network to reducing total delay or total energy
consumption. To minimize the overall delay of all users, in [13], the authors
jointly optimize the workloads division and time domain resource allocation of
users in a NOMA-assisted multi-access MEC offloading system. In this system,
the computation task of each user is divided into several parts and these
parts are offloaded to different MEC servers. Nevertheless, the coupling
relationship with MEC association and delay constraints on offloaded visual
processing tasks make the application of NOMA in the IoVT network is much more
challenging than in the traditional wireless network. Particularly, more
sophisticated power allocation is necessary for visual data uplinking
according to the energy constraints at the IoVT devices.

Task Division: In the IoVT network, both IoVT devices and MEC servers have
computation capability for visual processing. In this case, the problem that
which visual processing tasks should be chosen to be offloaded to the MEC
servers arises. Each IoVT device will divide the visual processing tasks into
two parts, one for local processing and the other for MEC offloading. The
results of task division will directly determine the MEC offloading
performance in terms of delay.

In essence, task division is to match the visual processing tasks with
computation capability at the MEC servers and IoVT devices under the delay
constraint on each visual processing task, aiming at minimizing the total
delay of all visual processing tasks. The heterogeneity of the visual
processing task offloading demands from the IoVT devices as well as the
heterogeneity of the computation capability of the MEC servers and IoVT
devices make it intractable to achieve optimal task division. For each IoVT
device, computation capability of the MEC server that it associates to is an
important factor that should be considered in task division. In the multi-MEC
IoVT network, there might be multiple MEC servers in the association range of
the IoVT device. Hence, The task division problem is much more complex for
multi-MEC offloading.

Figure 2: The proposed NOMA assisted IoVT framework with multi-MEC.

## A Novel NOMA Assisted IoVT Framework with Multi-MEC

To address the challenges in the previous section, we propose a novel NOMA
assisted IoVT framework with multi-MEC as shown in Fig.2. In the proposed
framework, we jointly optimize the association strategy, uplink wireless
transmission assisted by NOMA and division of visual processing tasks as well
as computation resources to meet the delay requirements of all IoVT devices,
with the goal to minimize the total delay of all visual processing tasks. The
work flow of the proposed joint optimization are described in Fig.3.

### Game Based MEC Association

In this article, we consider an IoVT network where an MEC server is deployed
with each BS. For convenience, an MEC server and the BS with which the MEC
server located is denoted as an MEC-BS. We assume that there are $M$ MEC-BSs
and $N$ IoVT devices in the IoVT network. With consideration of visual
processing task offloading demands of IoVT devices and computation capability
of MEC-BSs, we model the MEC association problem as a many-to-one matching
game where IoVT devices and MEC-BSs are two player sets, denoted by
$\mathcal{V}=\\{v_{1},v_{2},\ldots,v_{N}\\}$ and set
$\mathcal{B}=\\{b_{1},b_{2},\ldots,b_{M}\\}$, to be matched together.

We define a matching strategy $\Phi$ from set $\mathcal{V}$ to set
$\mathcal{B}$. $\Phi(v_{i})=b_{j}$ means that IoVT device $v_{i}$ associates
to MEC-BS $b_{j}$. In particular, $\Phi(v_{i})=\varnothing$ means that IoVT
device $v_{i}$ does not associate to any MEC-BS. In the matching game, each
player have its own preference list. That is to say, an IoVT device prefers to
associate to the MEC-BS with better channel conditions and sufficient
computation resources, while an MEC-BS prefers to serve more IoVT devices with
better channel conditions under delay constraints of visual processing tasks.
The preference lists of $v_{i}$ and $b_{j}$ are denoted by
$\mathcal{P}(v_{i})$ and $\mathcal{P}(b_{j})$, respectively. The utility
functions of $v_{i}$ and $b_{j}$ with matching strategy $\Phi$ are denoted by
$l(\Phi,v_{i})=\mathbf{I}(\Phi(v_{i})\neq\varnothing)$ and
$l(\Phi,b_{j})=\sum\limits_{v_{i}\in\mathcal{V}}\mathbf{I}(\Phi(v_{i})=b_{j})$,
respectively, where $\mathbf{I}(\cdot)$ is the indicator function. Let
$\Gamma(\mathcal{V},\mathcal{B},\Phi)=\sum\limits_{v_{i}\in\mathcal{V}}\mathbf{I}(\Phi(v_{i})\neq\varnothing)$
denote the number of IoVT devices associating to MEC-BSs when matching
strategy $\Phi$ is applied. We define the many-to-one matching game as the
following tuple.

$\mathscr{G}=\Big{(}\mathcal{V},\mathcal{B},\Phi,\mathcal{P},\Gamma(\mathcal{V},\mathcal{B},\Phi)\Big{)},$
(1)

The Nash Equilibrium (NE) is a strategy profile where no player has incentive
to deviate unilaterally. The definition of NE of the proposed game model is as
follows:

######  Definition 1.

: An NE of game $\mathscr{G}$ is reached if no IoVT device can associated to a
preferred MEC-BS and no MEC-BS can accept more IoVT devices.

To find an NE, Gale-Shapley algorithm is adopted, which is described as
follows.

1) Each IoVT device (e.g., $v_{i},~{}(1\leq i\leq N)$), sends an association
request to the first MEC-BS (e.g., $b_{j},~{}1\leq j\leq M$), in its
preference list $\mathcal{P}(v_{i})$ (i.e., $\Phi(v_{i})=b_{j}$);

2) Each MEC-BS (e.g., $b_{j},~{}(1\leq j\leq M)$), attempts to serve IoVT
devices associating to it. If the delay and computation resource requirements
of all IoVT devices can be satisfied, $b_{j}$ accepts all of them. Otherwise,
$b_{j}$ rejects the last IoVT device (e.g., $v_{i}$) that associates to it
according to its preference list $\mathcal{P}(b_{j})$, and set
$\Phi(v_{i})=\varnothing$.

3) If IoVT device $v_{i}$ is rejected by MEC-BS $b_{j}$, $v_{i}$ will delete
$b_{j}$ from its preference list (i.e.,
$\mathcal{P}(v_{i})=\mathcal{P}(v_{i})\setminus b_{j}$), and sends an
association request to the first MEC-BS (e.g., $b_{k}$) in its new preference
list $\mathcal{P}(v_{i})$ (i.e., $\Phi(v_{i})=b_{j}$);

4) Steps 2-3 will repeat until all IoVT devices finish their association or
all MEC-BS can not accept any more association request (i.e.,
$\mathcal{P}(v_{i})=\varnothing$ if $\Phi^{*}(v_{i})\neq\varnothing$).

Based on this algorithm, we can prove that Game $\mathscr{G}$ is bound to stop
at a NE after a limited number of iterations: In this algorithm, each IoVT
sends association requests to MEC-BSs in the descending order of its
preference list. Therefore, when game $\mathscr{G}$ stop, no IoVT player can
associated to a preferred MEC-BS, and no MEC-BS can accept more IoVT devices.
In addition, the size of the preference list of unassociated V-IoT player
$v_{i}$, $\mathcal{P}(v_{i})$, is reduced after each iteration. In other
words, a matching strategy $\Phi$ will not be selected repeatedly. So game
$\mathscr{G}$ is bound to stop at a NE after a limited number of iterations.

### NOMA Based Visual Data Uplinking

After MEC association, uplink NOMA transmission is performed by the IoVT
devices associating to the same MEC-BS. Consider an MEC-BS $b_{j}$. Let
$V(b_{j})$ denote the set of IoVT devices that associate to $b_{j}$. All IoVT
devices simultaneously transmit the signals that carry their visual data
information. In this case, $b_{j}$ receives a superposition form of the
signals from these devices. With SIC, $b_{j}$ first decodes the signal with
the strongest strength for the IoVT device that transmits it by regarding the
signals from other IoVT devices as noise, and then subtracts the decoded
signal from the original received signal. By repeating this process, all
signals from different IoVT devices can be decoded.

The SIC decoding order determines the offloading delay of the IoVT devices.
The IoVT device whose visual data are decoded first will suffer low offloading
delay. In the traditional uplink NOMA system, the SIC decoding order is the
descending order of channel conditions. However, in the context of MEC
offloading, different IoVT devices have different delay requirements for
visual processing task offloading. Hence, we propose a deadline aware SIC
decoding order, which is described in the right of Fig.3. Three IoVT devices
transmit their signals simultaneously in stage $\small{\small{1}}⃝$ until
uplink transmission of $v_{1}$ is finished. The MEC-BS decodes the signals of
these IoVT devices by SIC in ascending order of their delay requirements
(i.e., deadlines). The signal of $v_{1}$ is first decoded. It also means that
$v_{1}$ is the first one to finish its uplink transmission. After stage
$\small{\small{1}}⃝$, $v_{2}$ and $v_{3}$ transmit their signals
simultaneously in stage $\small{\small{2}}⃝$ until uplink transmission of
$v_{2}$ is finished. Then $v_{3}$ transmits its signal in stage
$\small{\small{3}}⃝$.

To achieve the desired SIC decoding order, power allocation should be
carefully considered at the IoVT devices based on their channel conditions.
Assume that transmit power of two IoVT devices $v_{i}$ and $v_{k}$ are $p_{i}$
and $p_{k}$, respectively, and the channel gains are $h_{i}$ and $h_{k}$,
respectively [15]. If visual data of $v_{k}$ is decoded before that of
$v_{i}$, $|h_{i}|^{2}p_{i}<\beta|h_{k}|^{2}p_{k}$ should be satisfied, where
$\beta\in(0,~{}1]$ can be set according to decoding capability of MEC-BS
$b_{j}$. Based the above analysis, we can design the power allocation strategy
to satisfy the delay requirements of the IoVT devices. An example is shown in
the right of Fig.3, where transmit power of $v_{i}$ is allocated as
$p_{i}=min(\tfrac{\beta|h_{i-1}|^{2}p_{i-1}}{|h_{i}|^{2}},P_{i}),~{}i=1,2,3$.
$P_{i}$ is the transmit power constraint of $v_{i}$.

Figure 3: The work flow of proposed joint optimization and NOMA assisted
transmission.

### Delay Driven Visual Processing Task Division

For each IoVT device, the visual processing task of each IoVT device is
divided into two parts, one part is processed locally and the other one part
is processed at MEC server. We assume that computation capability can be
indicated by the visual processing rate. For IoVT device $v_{i}$, let $L_{i}$,
$R_{i}$ and $U_{i}$ denote its local visual processing rate, achievable uplink
throughput and the visual processing rate that the MEC server allocated to it,
respectively. Let $C_{i}$ and $\alpha_{i}$ denote the total computation-
workload requirement of IoVT device $v_{i}$ and the proportion of its visual
processing task offloaded to the MEC server, respectively. Then the local
visual processing delay, the delay produced by visual data offloading and
visual processing at the MEC server are $\tfrac{(1-\alpha)C_{i}}{L_{i}}$,
$\tfrac{\alpha C_{i}}{R_{i}}$ and $\tfrac{\alpha C_{i}}{U_{i}}$, respectively.

The final delay of each visual processing task is determined by the larger of
the local visual processing delay and the delay produced by visual data
offloading and visual processing at the MEC server, i.e.
$max(\tfrac{(1-\alpha)C_{i}}{L_{i}},\tfrac{\alpha C_{i}}{R_{i}}+\tfrac{\alpha
C_{i}}{U_{i}})$. To make full use of both the local computation resources and
the computation resources at the MEC server, task division should make these
two kinds of delay be the same, i.e.
$\tfrac{(1-\alpha)C_{i}}{L_{i}}=\tfrac{\alpha C_{i}}{R_{i}}+\tfrac{\alpha
C_{i}}{U_{i}}$. Then, we can obtain optimal task division as
$\alpha=\tfrac{1}{L_{i}}/(\tfrac{1}{R_{i}}+\tfrac{1}{U_{i}}+\tfrac{1}{L_{i}})$.

Each MEC server performs computation resource allocation to handle the visual
processing tasks offloaded by the IoVT devices associating to it. Two
principles are considered by the MEC server: 1) satisfy as many computation
resource requirements of the offloaded visual processing tasks as possible. 2)
minimize the total delay of all visual processing tasks. Based on these two
principles, the MEC server first allocates the minimum required computation
resources to each offloaded visual processing task. Then, the remaining
computation resources are allocated to the offloaded visual processing task
whose delay can be reduced most. Actually, such computation resource
allocation problem can be treated as a water-filling problem, which has been
well studied in wireless communication.

In the proposed framework, the issues on MEC association, visual data
uplinking and task division are related to each other because of delay. If an
IoVT device associate to the nearest BS-MEC where the computation resources
are insufficient, although the delay for visual data uplinking may be very
small, the delay produced by visual processing at the MEC server will be too
large to meet the deadline. Hence as shown in Fig.3, in the work flow of the
proposed framework, each IoVT device first try to associate to the neatest BS-
MEC. Then the achievable uplink throughput based on NOMA (i.e., $R_{i}$) is
calculated. If this MEC server can provide sufficient computation resources to
this IoVT device, so that the visual processing tasks can be finished before
deadline, this BS-MEC will accept the association request of this IoVT device,
otherwise this BS-MEC will reject this association request and this IoVT
device will try to associate to another BS-MEC. After the IoVT association is
finished, each MEC server will divide its computation resources to the
associated IoVT devices by water-filling algorithm.

## Performance evaluation

In this section, the performance of the proposed framework is evaluated by
simulations. In the simulations, nine BSs are respectively placed in (-200m,
200m), (0m, 200m), (200m, 200m), (-200m, 0m), (0m, 0m), (200m, 0m), (-200m,
-200m), (0m, -200m), (200m, -200m) and IoVT devices are randomly distributed
in a 600m$\times$600m rectangular area around BSs, as shown in Fig. 4(a). Each
BS is deployed with an MEC server. The large-scale channel gain is
$128.1\\!+\\!37.6$log${}_{10}(d_{n}[$km$])$ dB. The Rayleigh fading
coefficient follows an i.i.d. Gaussian distribution as
$\beta\\!\\!\thicksim\\!\mathcal{CN}(0,1)$. We set the noise power to
$\sigma^{2}=BN_{0}$, where $B\\!\\!=\\!\\!2$ MHz and $N_{0}\\!=$-$174$ dBm/Hz.
The computation-workload requirement of each IoVT device is randomly chosen
from $5$ to $10$ Mbits. The deadline of each IoVT device is randomly chosen
from $0.1$ to $2$ s. Computation capability of each IoVT device is randomly
chosen from $1$ to $10$ Mbits/s. Computation capability of each MEC server is
randomly chosen from $0.4$ to $2$ Gbits/s.

Figure 4: Simulation results. (a) Distribution of BSs and IoVT devices (b)
Total delay vs. Number of IoVT devices.

In this paper, if a BS-MEC can-not provide enough computation Resources to an
IoVT device, the BS-MEC will reject the association request of this IoVT
device. If an IoVT device is rejected by all BS-MEC, we call it “Unassociated
IoVT devices” as shown in Fig. 4(a). Fig. 4(a) shows the distribution of $9$
BSs and $60$ IoVT devices in the proposed framework. The IoVT devices in the
same color associate to the same BS. The distance between the IoVT device and
BS indicates channel condition. Fig. 4(b) shows computation capability and the
number of IoVT devices associating to each BS.

In this paper, each MEC server is co-located with an BS, so the IoVT devices
not only prefer to associate with the BS with the best channel condition but
also the BS with the MEC server with sufficient computation resources.
Furthermore, considering the uneven distribution of the IoVT devices, if all
IoVT devices associate to the nearest BS, the computation load among MEC
servers may be unbalanced and thus the possibility of offloading failures
increases. Therefore, the traditional association strategies according to
distance are no longer effective. In the proposed framework, the channel
condition and the computation resources are jointly considered in MEC
association as stated in the game process of matching game $\mathscr{G}$. So
in Fig. 4(a), when the computation resources of the nearest MEC server is
insufficient, the IoVT devices may associate to the other MEC server. By
considering both computation capability and channel conditions when performing
IoVT device association, the MEC server with more computation resources will
serve more IoVT devices as shown in Fig. 4(b). This result confirms that the
proposed association strategy can allocate computation resources of MEC
servers more reasonably.

To evaluate the performance of proposed joint optimization, in Fig. 5(a) and
Fig. 5(b), we vary the number of IoVT devices from 10 to 55 to observe its
effect on the probability of unsuccessful association and the total delay of
all IoVT devices, respectively. The delay of the IoVT device without
association is set to $10$s for penalty. For comparison, the red lines are the
results of the proposed framework with the traditional distance based
association, and the blue lines are the conventional MEC offloading framework
where the traditional distance based association and the channel condition
based SIC decoding order are adopted. We can see that both the probability of
unsuccessful association and the total delay of all IoVT devices increase with
the number of IoVT devices. The reason is that, as the number of IoVT devices
increases, the interference among IoVT devices associating to the same BS will
increase, which will lead to lower uplink throughput. Therefore, the
transmission delay will increase. Furthermore, considering that the
computation resources of each MEC server is limited, the computation resources
allocated to each IoVT device will be reduced as the number of IoVT devices
increases. In addition, proposed joint optimization outperforms the reference
schemes. The reason can be described as follows. First, the visual processing
task with nearest deadline will be transmitted with the highest priority by
NOMA at the IoVT device, therefore, the probability of unsuccessful
association is reduced. Second, with the proposed association strategy, the
MEC server with more computation resources will serve more IoVT devices as
shown in Fig. 4(b). In other words, proposed computation resource allocation
is more reasonable.

Figure 5: Simulation results. (a) Probability of successful association vs.
Number of devices (b) Total delay vs. Number of devices.

## Conclusion and Future Work

To break through the bottlenecks of uplink throughput and computation
capability of single MEC server in IoVT networks, in this article, we propose
a novel NOMA assisted IoVT framework with multi-MEC. In the proposed
framework, to minimize the total delay of all visual processing tasks, we
perform joint optimization on the association strategy, uplink wireless
transmission assisted by NOMA, division of the visual processing tasks at the
IoVT devices and computation resource allocation at the MEC servers under the
delay constraint of each IoVT device. Simulation results show that the total
delay and the probability of unsuccessful association of the IoVT device can
be reduced significantly compared with the reference schemes.

As future work, some open issues on MEC offloading in IoVT networks are
discussed as follows.

Cooperative Offloading with Cloud: Due to powerful computation capability of
cloud, many studies propose to offload the visual processing tasks to cloud.
In this case, task division of visual processing tasks will become more
challenging with additional consideration of computation capacity of cloud.
Visual processing task offloading should be cooperatively performed by the
IoVT devices, MEC servers and cloud servers. Furthermore, as cloud servers are
usually serval hops away from the IoVT devices, the visual data uplinking
problem will much more complex. New research issues, such as backhaul
scheduling, should be concerned.

Evaluation Metrics for Offloading: More sophisticated evaluation metrics
besides delay are expected according to IoVT applications. For example, the
evaluation metrics for human oriented visual communication is much different
from that for machine-to-machine visual communication. For the former,
perceptual visual quality is important, while the latter prefers to transform
complex visual scenes into simple words. Evaluation metrics have significant
impact on transmission, processing and offloading of visual data. The MEC
offloading mechanism should be carefully designed according to the evaluation
metrics.

Joint Visual Data Coding and Uplinking: As IoVT devices become more and more
intensive in smart cities, especially in monitoring for public security, it is
likely that there exists quite a lot of redundancy among the visual data
generated by neighboring IoVT devices. The redundancy will result in a huge
waste of uplink transmission and computation resources. To deal with this
issue, collaborative coding and wireless transmission can be jointly studied.
For example, distributed video coding and NOMA uplink transmission can be
integrated to achieve optimal offloading performance.

## Acknowledgment

This work was supported by National Key R&D Program of China under Grant
2020YFA0711400 and National Science Foundation of China under Grant 61771445,
61631017, 91538203.

## References

  * [1] C. W. Chen, “Internet of Video Things: Next-Generation IoT With Visual Sensors,” IEEE Internet of Things Journal, vol. 7, no. 8, Aug. 2020, pp. 6676-6685.
  * [2] W. Ji, J. Xu, H. Qiao, M. Zhou and B. Liang, “Visual IoT: Enabling Internet of Things Visualization in Smart Cities,” IEEE Network, vol. 33, no. 2, Mar./Apr. 2019, pp. 102-110.
  * [3] N. Chen, Y. Chen, Y. You, H. Ling, P. Liang and R. Zimmermann, “Dynamic Urban Surveillance Video Stream Processing Using Fog Computing,” IEEE Second International Conference on Multimedia Big Data (BigMM), Taipei, Apr. 2016, pp. 105-112.
  * [4] W. Ji, B. Liang, Y. Wang, R. Qiu and Z. Yang, “Crowd V-IoE: Visual Internet of Everything Architecture in AI-Driven Fog Computing,” IEEE Wireless Communications, vol. 27, no. 2, Apr. 2020, pp. 51-57.
  * [5] Y. Gui, H. Lu, F. Wu and C. W. Chen, “Robust Video Broadcast for Users with Heterogeneous Resolution in Mobile Networks,” _IEEE Trans. on Mobile Computing_ , pp. 1-1, June 2020.
  * [6] Oh S. H., Kim G. W., Lim K. S., “Compact deep learned feature-based face recognition for Visual Internet of Things,” Journal of Supercomputing, vol. 74, no. 12, Dec. 2018, pp. 6729-6741.
  * [7] S. S. N. Perala, I. Galanis and I. Anagnostopoulos, “Fog Computing and Efficient Resource Management in the era of Internet-of-Video Things (IoVT),” in Proc. IEEE Int. Symp. Circuits Syst (ISCAS), Florence, Italy, May 2018, pp. 1-5.
  * [8] J. Xiong, L. Lu, H. Wang, J. Yang and G. Gui, “Object-Level Trajectories Based Fine-Grained Action Recognition in Visual IoT Applications,” IEEE Access, vol. 7, July 2019, pp. 103629-103638.
  * [9] B. Blanco-Filgueira, D. Garcia-Lesta, M. Fernandez-Sanjurjo, V. M. Brea and P. Lopez, “Deep Learning-Based Multiple Object Visual Tracking on Embedded System for IoT and Mobile Edge Computing Applications,” IEEE Internet of Things Journal, vol. 6, no. 3, June 2019, pp. 5423-5431.
  * [10] Y. Wang, J. Xu and W. Ji, “A Feature-based Video Transmission Framework for Visual IoT in Fog Computing Systems,” ACM/IEEE Symposium on Architectures for Networking and Communications Systems (ANCS), Cambridge, United Kingdom, Sept. 2019, pp. 1-8.
  * [11] H. Trinh et al., “Energy-Aware Mobile Edge Computing and Routing for Low-Latency Visual Data Processing,” IEEE Trans. Multimedia, vol. 20, no. 10, Oct. 2018, pp. 2562-2577.
  * [12] T. Sultana and K. A. Wahid, “Choice of Application Layer Protocols for Next Generation Video Surveillance Using Internet of Video Things,” IEEE Access, vol. 7, Mar. 2019, pp. 41607-41624.
  * [13] Y. Wu, K. Ni, C. Zhang, L. P. Qian and D. H. K. Tsang, “NOMA-assisted Multi-access Mobile Edge Computing: A Joint Optimization of Computation Offloading and Time Allocation,” IEEE Trans. Veh. Technol., vol. 67, no. 12, Dec. 2018, 12244-12258.
  * [14] M. Zhang, H. Lu, F. Wu and C. W. Chen, “NOMA-Based Scalable Video Multicast in Mobile Networks With Statistical Channels,” _IEEE Trans. on Mobile Computing_ , vol. 20, no. 6, pp. 2238-2253, June 2021.
  * [15] Y. Wang, H. Lu and H. Sun, “Channel Estimation in IRS-Enhanced mmWave System with Super-Resolution Network,” _IEEE Commun. Lett._ , pp. 1-1, May 2021.

## Biographies

Fengqian Guo received the B.S. degree in 2017 from the Jiangnan University,
Wuxi, China, where he is currently pursuing the Ph.D. degree in communication
and information systems with the Department of Electronic Engineering and
Information Science, University of Science and Technology of China (USTC),
Hefei, China. His current research interests include wireless transmission and
multiple access networks.

Hancheng Lu (M’07) received the Ph.D. degree in communication and information
systems from the University of Science and Technology of China (USTC), Hefei,
China, in 2005. He is currently an associate professor with the Department of
Electronic Engineering and Information Science, USTC. His research interests
include multimedia communication and networking, resource optimization in
wireless heterogeneous networks.

Bo Li received the B.S. degree in 2020 from the Dalian University of
Technology (DLUT), Dalian, China. He is currently pursuing the M.S. degree in
electronics and communication engineering with the Department of Electronic
Engineering and Information Science, University of Science and Technology of
China(USTC), Hefei, China. His current research interests include wireless
communication.

Dingxuan Li received the B.E. degree in 2020 from the University of Science
and Technology of China (USTC), Hefei, China, where he is currently working
toward the MA.Eng. degree in communication and information systems with the
Department of Electronic Engineering and Information Science. His research
interests include wireless network power allocation and federated learning.

Chang Wen Chen (F’04) is currently Chair Professor of Visual Computing at The
Hong Kong Polytechnic University. He has previously served as an Empire
Innovation Professor of Computer Science and Engineering at the University at
Buffalo, State University of New York from 2008 to 2021. From 2017 to 2020, He
served as Dean of the School of Science and Engineering at The Chinese
University of Hong Kong, Shenzhen. He was Allen Henry Endow Chair Professor at
the Florida Institute of Technology from 2003 to 2007. He was on the faculty
of Electrical and Computer Engineering at the University of Rochester from
1992 to 1996 and on the faculty of Electrical and Computer Engineering at the
University of Missouri-Columbia from 1996 to 2003.

He has been the Editor-in-Chief for IEEE Trans. Multimedia from 2014 to 2016.
He has also served as the Editor-in-Chief for IEEE Trans. Circuits and Systems
for Video Technology from 2006 to 2009. He has been an Editor for several
other major IEEE Transactions and Journals, including the Proceedings of IEEE
and IEEE Journal of Selected Areas in Communications. He has served as
Conference Chair for several major IEEE, ACM and SPIE conferences related to
multimedia, video communications and signal processing, including ACM MM2020
and ACM MM2018 in recent years. His research expands a broad range of topics
in multimedia communication, Internet of Video Things, multimedia systems,
image/video processing, machine learning, and multimedia signal processing.

He received his BS from University of Science and Technology of China in 1983,
MSEE from University of Southern California in 1986, and Ph.D. from University
of Illinois at Urbana-Champaign in 1992. He and his students have received
nine (9) Best Paper Awards or Best Student Paper Awards. He has also received
several research and professional achievement awards, including Sigma Xi
Excellence in Graduate Research Mentoring Award in 2003, Alexander von
Humboldt Research Award in 2009, the University at Buffalo Exceptional Scholar
Sustained Achievement Award in 2012, the State University of New York System
Chancellor’s Award for Excellence in Scholarship and Creative Activities in
2016, and the Distinguished ECE Alumni Award from University of Illinois at
Urbana-Champaign in 2019. He is an IEEE Fellow since 2005 and an SPIE Fellow
since 2007.