# Reinforcement Learning - based Adaptation and Scheduling Methods for Multi-
source DASH

Nghia T. Nguyen, Long Luu, Phuong L. Vo,
Thi Thanh Sang Nguyen, Cuong T. Do, Ngoc-thanh Nguyen Nghia T. Nguyen, Phuong
L. Vo, Thi Thanh Sang Nguyen, and Long Luu are with the International
University - Vietnam National University, Ho Chi Minh City, Vietnam; Cuong T.
Do is with Kyung Hee University, 446-701, Korea; Ngoc-thanh Nguyen is with
Wroclaw University of Science and Technology, Poland.
Email: {ntnghia, vtlphuong<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>ngoc-
<EMAIL_ADDRESS>
This is an extended version of the conference paper [20]. This research is
funded by Vietnam National University HoChiMinh City (VNU-HCM) under grant
number DS2020-28-01. Dr. Phuong L. Vo is the corresponding author.

###### Abstract

Dynamic adaptive streaming over HTTP (DASH) has been widely used in video
streaming recently. In DASH, the client downloads video chunks in order from a
server. The rate adaptation function at the video client enhances the user’s
quality-of-experience (QoE) by choosing a suitable quality level for each
video chunk to download based on the network condition.

Today networks such as content delivery networks, edge caching networks,
content-centric networks, etc. usually replicate video contents on multiple
cache nodes. We study video streaming from multiple sources in this work. In
multi-source streaming, video chunks may arrive out of order due to different
conditions of the network paths. Hence, to guarantee a high QoE, the video
client needs not only rate adaptation, but also chunk scheduling.

Reinforcement learning (RL) has emerged as the state-of-the-art control method
in various fields in recent years. This paper proposes two algorithms for
streaming from multiple sources: RL-based adaptation with greedy scheduling
(RLAGS) and RL-based adaptation and scheduling (RLAS). We also build a
simulation environment for training and evaluating. The efficiency of the
proposed algorithms is proved via extensive simulations with real-trace data.

Keywords: multi-source streaming, reinforcement learning, proximal policy
optimization, dynamic adaptation streaming over HTTP.

##  1 Introduction

A significant part of Internet traffic today is video streaming [1]. Dynamic
adaptive streaming over HTTP (DASH) is a primary technique used to stream a
video from a server to a video player. In DASH, videos are encoded in multiple
quality levels. Furthermore, videos are partitioned into video chunks. Each
chunk contains media data in a short interval of playback time. Video players
(clients) request the chunks with suitable quality levels based on the current
network condition [2, 3, 4, 5]. The downloaded chunks are buffered in the
client’s memory before being played. Buffer size is the total playing time of
the wait-to-be-played chunks. When a new video chunk is successfully
downloaded, the buffer size increases by a chunk length. When a chunk is
played, the buffer size is decreased by the chunk length. The buffer size has
an upper threshold level. When the buffer exceeds the threshold, the client
will pause downloading a new chunk, wait for the buffer to decrease below the
threshold, and then resume downloading. The client rebuffers when the chunk
will be played is not in the buffer. Rebuffering causes video freezes.

The rate adaptation function in video clients is essential in providing a high
quality-of-experience (QoE) for the user. Various adaptation methods are
proposed for DASH. The throughput-based adaptation method chooses the quality
level for the next chunk such that the bitrate does not exceed the estimated
throughput [7, 6]. The throughput is usually estimated by the mean or harmonic
mean of several last requested chunks. The buffer-based methods observe the
buffer level to decide the encoding quality level [8, 9]. Both the throughput-
based method and BOLA, a buffer-based method [8], are employed in Dash.js
reference client [6]. Some methods combine these two approaches [10].

On the other hand, several networks today such as content delivery networks,
edge caching networks, content-centric networks, etc. replicate popular videos
at the routers to reduce network congestion and delay. Utilizing multiple
sources to stream a video to a user is studied in this paper. When streaming
from multiple sources, quality control is much more complicated than streaming
from a single source. In multi-source streaming, the quality control includes
not only rate adaptation, i.e., choosing the quality levels for the chunks,
but also chunk scheduling, i.e., which chunk indices are requested on each
path (see Fig. 1). Due to the difference in the network conditions of the
connections, the chunks may arrive at the video client out of order. For
example, assume that the maximum buffer size of the client is 3 chunks and
there are two paths. With bad scheduling, path 2 is downloading chunk 2 while
path 1, with very high throughput, already downloaded chunks 1, 3, 4.
Therefore, the buffer is full, however, the video playing is frozen since the
client waits for chunk 2.

Figure 1: Multi-source video streaming.

Some previous works have studied multi-source streaming [11, 12, 13]. In [11],
MSPlayer can download video content from multiple servers. The authors in [11]
consider the chunks with only one quality level, however, the chunk size
varies. They focus on the chunk scheduling problem. The client estimates the
path quality to request chunk indices and chunk sizes for the paths. In work
[12], MP-H2 protocol is designed on top of HTTP/2. MP-H2 splits the video into
many chunks, and the client requests chunks over multiple network connections
such as wi-fi and cellular. Chunk sizes are calculated based on bandwidth and
round-trip-time of the connections. A chunk scheduling algorithm is applied to
download the chunks over multiple paths without any adaptation method [12].
The work [13] has proposed a bitrate adaptation algorithm for DASH, called DQ-
DASH, that allows to download multiple video chunks from various servers in
parallel to enhance QoE. Distributed queueing theory is applied to address the
situation when multiple clients send requests to many servers simultaneously.
Different from [11, 12, 13], our proposed framework jointly apply both rate
adaptation and chunk scheduling.

Reinforcement learning (RL) has been widely used in many fields recently. The
works [14, 15, 16] have applied RL algorithms for single-source adaptive
streaming. The actions of RL agents in these works are the quality levels of
video chunks. The work [14] applied a Q-learning method for DASH. Buffer and
network bandwidth are discretized for the discrete state space. The study [15]
used Asynchronous Advantage Actor-Critic (A3C) algorithm for rate adaptation.
The work [16] has proposed D-DASH that applied a Deep Q-learning to choose the
quality level for the chunks.

In this work, we also use RL algorithm for rate adaptation and chunk
scheduling in streaming from multiple sources. However, there are several
challenges we cannot simply extend the RL framework for single-source
streaming in [15, 16] to multi-source streaming straightforwardly:

  * •

The action space of multi-source streaming must be redesigned to integrate
scheduling. An action must include a chunk index and a quality level.

  * •

The RL algorithms need a simulation environment to train the model. In the
environment for single-source streaming, when the agent takes action, i.e.,
downloads a new chunk, the environment immediately returns a reward value
associated with that chunk, which is calculated from the utility of the
chunk’s quality, the quality switch penalty between two consecutive chunks and
the rebuffering penalty. The rebuffering penalty is calculated when action is
taken. However, in multi-source streaming, where the chunks may arrive to the
client out of order, the simulation environment cannot estimate the
rebuffering time right when an action is taken.

  * •

The simulation environment for the single-source streaming is open [15].
However, the simulation environment for the multi-source streaming is not
available in the literature, as far as we know.

Multipath transmission control protocol (MPTCP) [17, 18, 19] also utilizes
multiple paths to transmit data from a source to a destination. It is shown
that MPTCP achieves high throughput, provides a smooth hand-off, and improves
the high availability of TCP connection. However, the MPTCP adoption has been
prolonged because of the middlebox problem. Moreover, it requires modifying
the kernels of both client and server. Our proposed framework can be applied
to stream a video from a single source to a video player over multiple paths
as MPTCP does. However, the MPTCP is a source-control protocol at the
transport layer, whereas our proposed multi-source streaming is a client-based
control protocol at the application layer. Therefore, the proposed multi-
source streaming does not need to modify the kernel as well as overcomes the
middlebox problem.

The contributions of our work include:

  1. 1.

We propose two RL-based frameworks for rate adaptation and chunk scheduling in
multi-source streaming called RL-based adaptation with greedy scheduling
(RLAGS) and RL-based adaptation and scheduling (RLAS).

  2. 2.

We build an environment, which is an event-driven simulation that simulates a
client downloading chunks from multiple sources and playing the chunks for
training and testing.

  3. 3.

We conduct extensive simulations with real-trace bandwidth to evaluate the
performance of the proposed methods. Both RLAGS and RLAS outperform the other
baseline methods used with greedy scheduling for multi-source streaming, i.e.,
throughput-based and BOLA. The source code is available at
https://github.com/ntnghia1908/Master_Thesis.

This is the extended work of our conference paper [20]. In this paper, RLAS
improves the proposed algorithm in [20] by using invalid action masking to
avoid duplicate downloads. In addition, we propose RLAGS algorithm with a
greedy scheduling. We also add more evaluations in various network scenarios.
The outline of the paper is as follows. Section I has presented the motivation
and related works. Section II describes the RL model applied in rate
adaptation and chunk scheduling for video streaming from multiple sources. The
simulation environment and results are presented in Section III. Section IV
concludes the work.

##  2 Reinforcement learning frameworks for DASH

This section describes the RL framework, including reward function, action
space, and state space. Two chunk scheduling policies are considered, i.e.,
greedy and RL-based scheduling, which leads to two proposed algorithms, RLAGS
and RLAS, respectively.

###  2.1 Reward

Table 1: Main notations Notations |  Descriptions
---|---
$B^{\max}$ |  maximum buffer size (in seconds)
$N$ |  number of video chunks
$L$ |  number of quality levels in action space
$W$ |  number of chunks in action space
$\mathcal{A}$ |  action space
$r_{t}$ |  reward estimated at step $t$
$s_{t}$ |  environment state at step $t$
$q_{i}$ |  utility of quality level $i$
$\beta$ |  quality-switch coefficient
$\gamma$ |  rebuffering coefficient

We apply a similar reward function used in [15, 16], which captures utility,
switch penalty, and rebuffering penalty. Assume that a time step begins when
the client requests a video chunk. The episode ends when the client finishes
playing the video.

####  2.1.1 Reward for single-source streaming:

Assume that step $t$ starts when the client requests for chunk $t$, the reward
at step $t$ in single-source adaptive streaming is given by [15, 16] (see
Table 1 for the notation descriptions):

$\displaystyle r_{t}=q_{t}-\beta\mid
q_{t}-q_{t-1}\mid-\gamma\phi_{t}-\delta[\max(0,B^{min}-B_{t})]^{2},\quad
t=2,\ldots,N,$ (1)

where

  * •

$q_{t}$ is the utility corresponding to the quality level of chunk $t$;

  * •

$\mid q_{t}-q_{t-1}\mid$ penalties the difference in quality levels between
two consecutive chunks;

  * •

$\phi_{t}$ is rebuffering time is seconds;

  * •

$[\max(0,B^{min}-B_{t})]^{2}$ is an optional penalty that is applied whenever
the buffer level is below a threshold $B^{min}$. This term helps to reduce the
risk of rebuffering.

If $d_{t}$, i.e., the download time of chunk $t$, is greater than remaining
time in buffer, which is $B_{t}$, then rebuffering time $\phi_{t}$ is
$d_{t}-B_{t}$, otherwise, there is no rebuffering. Hence, the rebuffering time
associated with chunk $t$ in single-source streaming is given by the following
formula

$\phi_{t}=\max(0,d_{t}-B_{t}).$ (2)

####  2.1.2 Reward for multi-source streaming:

Formula (2) is not correct in the multi-source streaming environment since the
buffer at the client may not store consecutive chunks. For example, the buffer
may have chunks 3, 5, 6, and 7, while chunk $4$ has not fully received on the
low-throughput path. Therefore, in the multi-source environment, the reward is
estimated when playing chunks in a step. We assume that a step starts when the
client requests a chunk and ends when the client requests a new chunk or
reaches the end of the episode. The reward returned at step $t$ in the multi-
source streaming environment is given by

$\displaystyle r_{t}=\sum_{i}q_{i}-\beta\sum_{i}\mid
q_{i}-q_{i-1}\mid-\gamma\phi_{t},$ (3)

where $i$ is any chunk index played, and $\phi_{t}$ is the cumulative
rebuffering time in step $t$. The terms $\sum_{i}q_{i}$, $\beta\sum_{i}\mid
q_{i}-q_{i-1}\mid$, and $\gamma\phi_{t}$ are called utility, switching
penalty, and rebuffering penalty, respectively.

###  2.2 State space

The state $s$ of the proposed reinforcement learning frameworks includes the
following components

  * •

vector of network throughput measurements of last 06 video chunks on each
path;

  * •

vector of chunk sizes of $L$ quality levels of next $W$ chunks count from
playing chunk (length $W\times L$);

  * •

the vector of quality levels of next $W$ chunks counted from the playing
chunk, if the chunks have not yet downloaded, their quality level is set to 0;

  * •

current buffer size in seconds;

  * •

number of remaining chunks that have not yet played;

  * •

quality level of the playing chunk; and

  * •

download times of last 06 video chunks on each path.

###  2.3 Scheduling policies and action spaces

We assume that the request for a new chunk is sent on a path right after the
downloading chunk on that path is fully received if the buffer size is under
$B^{\max}$. Otherwise, the client will pause sending a new request. We
consider two scheduling policies, i.e., greedy scheduling and RL-based
scheduling, corresponding to two proposed methods, RLAGS and RLAS,
respectively.

####  2.3.1 Greedy scheduling

In greedy scheduling, the chunk is requested in order. When the client
downloads a new chunk from a source, it requests the chunk index, the smallest
index that has not been or is being downloaded. Therefore, RLAGS agent only
decides the quality level of the chunk to request. The action space of RLAGS
includes the quality levels of video chunks:

$\mathcal{A}^{\rm{RLAGS}}=\\{l_{i}|i=1,\ldots,L\\},$ (4)

where $L$ is the number of quality levels of video.

For example, in Fig. 2 (upper figure), chunks 1-3 have been played, chunk 4 is
playing, and chunk 5 has already been requested. The next request is for chunk
6, with the quality level decided by RLAGS.

####  2.3.2 RL-based scheduling

RLAS method uses RL-based scheduling. When the client requests a new chunk,
the RL agent decides both the index and quality level. Assuming that the
maximum number of chunks stored in the video buffer is $W$, the number of
quality levels is $L$. Action space of RLAS method is defined as

$\mathcal{A}^{\rm{RLAS}}=\\{(c_{i},l_{j})|c_{i}\in[1,W],j\in[1,L]\\}.$ (5)

If the playing chunk is $c_{t}$ and the RL agent takes action
$a_{t}=(c_{i},l_{j})$ to download chunk on a path at time step $t$, the client
will download chunk index $c_{t}+c_{i}$ at quality $l_{j}$ on this path.

For example, in Fig. 2, if quality levels for each chunk are low, medium, and
high ($L=3$), $W=4$. Assume that at current time $t$, the playing chunk is
$c_{t}=4$ and the RL agent takes action $a_{t}=(3,2)$. It means that the agent
will download chunk index $4+3=7$ with quality level $2$, which is medium
quality (see Fig. 2).

Figure 2: The action spaces of RLAGS and RLAS. (The shade regions represent
the chunks that have been requested.)

####  2.3.3 Invalid action masking

With RLAS, there are invalid actions in some steps. Firstly, the two-dimension
action space of RLAS allows the possibility of re-download the same chunk
index again if that chunk has not been played. The chunk index already
downloaded is considered an invalid action to avoid duplicate download.
Secondly, since RLAS’s action space is a sliding window that shifts forward by
one chunk when a new chunk is played, some actions are invalid when the number
of remaining chunks is less than the window side $W$. Hence, the valid actions
of RLAS are given by

$\\{(c_{i},l_{j})\mid c_{i}\in[1,\min{[W,N-c_{t}]},c_{i}\textrm{ has not been
requested},j\in[1,L]\\},$ (6)

where $N$ is the number of chunks of the video, and chunk $c_{t}$ is the chunk
being played.

There are several approaches to dealing with invalid actions. Two common ones
are invalid action penalty and invalid action masking. With the invalid action
penalty approach, the rewards resulting from the invalid actions are set to
negative values. With invalid action masking, the action is sampled among the
valid actions in each step. These approaches are well investigated and
implemented in [21]. The authors show that the invalid action masking
outperforms the other approaches theoretically and empirically with the state-
of-the-art proximal policy optimization (PPO) algorithm [21]. Therefore we
also apply PPO in our evaluations.

###  2.4 PPO for multisource DASH

PPO is a policy gradient algorithm that uses two networks, actor and critic,
like A2C or A3C. The actor network estimates the policy directly from the
state. A baseline is subtracted from the return to reduce the variance of a
policy gradient algorithm. A common-used baseline is the value function, which
is estimated by a critic network. To accelerate the training, PPO algorithm
could also use multiple copied environments in parallel, similar to A2C and
A3C.

However, PPO is an improvement from A2C/A3C. To prevent the catastrophic drop
in the performance of the traditional actor-critic algorithms, PPO constraints
the change in policy between two consecutive training steps by introducing a
new clipped surrogate objective. PPO has shown a reliable performance and is
used in many RL applications. We can see the detail of PPO algorithm in [25].

We utilize Stable Baseline3 [26] library to implement PPO in training and
evaluation. Stable Baseline3 includes a set of reliable implementations of
deep RL algorithms and is used in many applications. The invalid action
masking function is also provided with PPO in Stable Baseline3.

##  3 Evaluations

###  3.1 Event-driven environment

We build an environment that simulates the streaming from two sources,
emulating the practical scenarios, e.g., a cell phone uses Wi-Fi and 4G to
connect to video servers, or a laptop connects via Ethernet and Wi-Fi
simultaneously. Scenarios having more than two sources can be easily extended
by modifying reset function. The simulation environment follows Gym interface
to be compatible with Stable Baseline3 [26].111The source code of the
environment is available at
https://github.com/ntnghia1908/Master_Thesis/blob/main/RLAS/menv_baseline.py.

The environment emulates a client downloading chunks on two paths parallelly
and playing the received chunks. An array-type buffer, which stores downloaded
chunk indices, is maintained during an episode. When the client fully receives
a chunk, the chunk index is appended to the buffer, and the buffer size
increases by a chunk length. The client plays the chunks stored in the buffer
sequentially. If a chunk is played, that chunk index is removed from the
buffer, and the buffer size increases by a chunk length.

There are four main events, i.e., DOWN, PAUSE, PLAY, REBUFFER. Every event has
a timestamp, and the program runs through the events iteratively in time order
until the end of the episode. DOWN and PAUSE events are associated with a path
index, whereas PLAY and REBUFFER events are not.

  * •

A DOWN event simulates sending a request for a chunk, say $c_{t}$, on a path.
When the program encounters a DOWN event at time t, at timestamp t + downtime,
where downtime is the time from sending the request for $c_{t}$ to fully
receiving the chunk, a new DOWN event associated with a new chunk is generated
if the buffer size is less than $B^{\max}$. The index and the quality level of
the new chunk are decided by RLAGS or RLAS methods. Otherwise, a PAUSE event
is generated if the buffer size exceeds $B^{\max}$.

  * •

A PAUSE event simulates pausing the download on a path due to the buffer size
exceeding $B^{\max}$. If a PAUSE event is encountered at time $t$, with
timestamp $t+sample$, where sample is a short period (0.05 second in our
program), a new PAUSE event is generated if the buffer size exceeds
$B^{\max}$; otherwise, a new DOWN event associated with a new chunk is
generated.

  * •

A PLAY event occurs when the client starts playing a chunk, say chunk $c_{t}$.
After a PLAY event, at time $t+chunk\\_length$, a new PLAY event associated
with chunk $c_{t}+1$ is generated if this chunk is available in the buffer;
otherwise, a REBUFFER event is generated.

  * •

A REBUFFER event occurs when the chunk going to be played is not in the
buffer. After a REBUFFER event, at time $t+sample$, a new PLAY event
associated with chunk $c_{t}+1$ is generated if this chunk is fully received;
otherwise, a REBUFFER event is generated.

###  3.2 Simulation settings

We evaluate RLAGS and RLAS with Big Bug Bunny video [4]. There are seven
quality levels $300,700,1200,1500,3000,6000,8000$ Kbps ($L=7$). Assume that
the maximum buffer size of the client is $B^{\max}=30$ seconds and the video
chunk length is 4 seconds. The number of chunks in the action space is
$W=\lfloor 30/04\rfloor=7$, which means that if the agent is playing chunk
$i$, the maximum chunk index stored in the buffer is $i+7$. We train with the
first 60 chunks, which results in 240 seconds per episode. Table 2 shows the
parameters of the simulation environment.

Two real-trace datasets are used: a broadband dataset provided by US Federal
Communications Commission (FCC) [22] and a 4G LTE Dataset collected from two
major Irish mobile operators [24].

Figure 3: The distributions of the means of all traces of the datasets.

The FCC dataset contains over one million throughput traces in the “download
speed” category with a granularity of 10 seconds per sample [23]. (It is 5
seconds before 2016.) The 4G dataset has 135 traces, with around 15 minutes
per trace, at 1-second granularity. The traces are collected from Irish mobile
operators with five mobility patterns: static, pedestrian, car, bus, and train
[24].

Since the real bitrates of 8,000 Kbps quality level of the almost chunks are
less than 4000 Kbps [4], we choose the traces with the average throughputs in
$[0.1,2.0]$ Mbps, in which 1800 traces in the FCC dataset and 400 traces in
the LTE one. Fig. 3 shows the distributions of the average throughputs of the
traces from two datasets. We randomly select 80% of traces in each dataset for
training, the remaining ones are for testing.

Table 2: Simulation parameters Environment parameters | Notations |  Values
---|---|---
maximum buffer size | $B^{\max}$ |  30 seconds
number of video chunks | $N$ |  60 chunks
number of quality levels in action space | $L$ |  7
number of chunks in action space | $W$ |  7
quality levels | $l_{i}$ |  $[300,700,1200,1500,3000,6000,8000]$ Kbps
utility | $q_{i}$ |  $\ln(\frac{l_{i}}{l_{1}})$
quality-switch coefficient | $\beta$ |  1
rebuffering coefficient | $\gamma$ |  3.3

We compare RLAGS and RLAS methods with two well-known adaptation methods,
i.e., throughput-based [2] and BOLA [8] (a buffer-based) methods. These
adaptations are originally designed for single-source video streaming. We
apply greedy scheduling to extend them to multi-source streaming. In the
throughput-based method, the quality of the next download chunk on one path is
the highest quality level which is smaller than the harmonic mean of the last
six chunks downloaded on that path.

Table 3 lists some tuned hyper-parameters for RLAGS and RLAS. The not-listed
hyperparameters are used with the default values provided by Stable Baseline3.
We use fully connected neural networks with 64 nodes for each hidden layer. We
tuned the number of hidden layers for the algorithms. Round-trip-times of the
network connections are uniformly random in $[50,100]$ ms.

Each proposed algorithm is trained in five runs, $30,000$ episodes each run.
Each episode chooses a random trace in the training set and starts at a random
point. The throughput trace is circulated if the time from starting point of
an episode to the end of the throughput trace is not enough for the time
playing the episode. The results are the average values in five runs.

Table 3: Tuned hyperparameters used in RLAGS and RLAS. Hyperparameters |  Descriptions |  Tuning ranges |  RLAGS |  RLAS
---|---|---|---|---
learning rate |  learning rate |  uniform: [0.0001, 0.001] |  0.000125 |  7.61e-05
batch size |  minibatch size |  uniform: [59, 590] |  411 |  530
n epochs |  number of epoch when optimizing the surrogate loss |  values: [10, 20, 30] |  10 |  10
gamma |  reward discount factor |  values: [0.99, 1.0] |  0.99 |  1
gae lambda |  factor for trade-off of bias vs. variance for generalized advantage estimator |  values: [0.9, 0.95] |  0.9 |  0.95
clip range |  clipping parameter |  values: [0.2, 0.3] |  0.3 |  0.2
vf coef |  value function coefficient for the loss calculation |  uniform: [0.2, 0.5] |  0.317708 |  0.286954
ent coef |  entropy coefficient for the loss calculation |  values: [0.0, 0.00001, 0.00000001] |  0 |  0
act func |  value function coefficient for the loss calculation |  values: [128, 256, 512] |  256 |  512
features dim |  value function coefficient for the loss calculation |  values: [tanh, relu] |  relu |  tanh
policy net arch layer |  number of policy network layer |  values: [1, 2, 3, 4] |  1 |  3
policy net arch units |  policy network unit. |  values: [64, 128, 256, 512] |  512 |  256
value net arch layers |  number of value network layer |  values: [1, 2, 3, 4] |  3 |  4
value net arch units |  value network unit. |  values: [64, 128, 256, 512] |  512 |  256

###  3.3 Results

Figure 4: Convergence of training phases of RLAGS and RLAS methods. The lines
and shadows are the means and the standard deviations of the running average
rewards of five runs, respectively.

Fig. 4 shows the convergence of both RLAGS and RLAS algorithms in training
with turned parameters given in Table 3. We can see that RLAGS converges
faster than RLAS since RLAGS has fewer actions than RLAS, which are only
quality levels. However, RLAS yields a higher average reward than RLAGS.

Table 4: Rewards of ABR methods when one path is broadband and another path is LTE connection. Methods | reward | utility |  switch penalty |  rebuffering penalty
---|---|---|---|---
THGHPUT | 42.10 | 68.56 | 21.40 | 5.06
BOLA | 77.80 | 129.75 | 27.26 | 24.70
RLAGS | 88.35$\pm$2.04 | 97.86$\pm$2.29 | 8.53$\pm$1.01 | 0.98$\pm$0.52
RLAS | 107.75$\pm$1.91 | 130.68$\pm$5.87 | 17.51$\pm$4.88 | 5.42$\pm$2.62

We test the case when one path is a broadband connection, and the other path
is an LTE connection. The reward, utility, switch penalty, and rebuffering
penalty are given in Table 4. The rewards yielded by RLAGS and RLAS are higher
than the reward by throughput-based and BOLA methods. RLAS achieves the
highest reward, and RLAGS results in a smaller rebuffering penalty.

Figure 5: The test rewards of different adaptation methods when the mean
bandwidth of the first path is in $[1.5,2]$ Mbps and the mean bandwidth of the
second path decreases gradually.

We consider the performance of multisource streaming in the case when the
difference between two paths increases gradually. Particularly, the mean
bandwidth of the first path is from 1.5 Mbps to 2 Mbps and the mean bandwidth
of the second path decreases gradually, in $[1.5,2.0]$ Mbps, in $[1.0,1.5]$
Mbps, in $0.5,1.0$ Mbps, and less than 0.5 Mbps.

We can see from Fig. 5 that the rewards of multisource streaming of all the
methods decrease gradually. The rewards of RLAS are the highest in most of the
cases, which shows the efficiency of the RL-based chunk scheduling. BOLA
yields a higher reward than RLAGS. However, in the extreme case when the mean
bandwidth of two paths is very different, RLAGS and RLAS outperform the
traditional methods.

Table 5: Rewards of ABR methods with the mean bandwidths of the first path is in $[1.5,2.0]$ Mbps and of the second path is less than 0.5 Mbps. Methods | reward | utility |  switch penalty |  rebuffering penalty
---|---|---|---|---
THGHPUT | 17.34 | 68.20 | 32.03 | 18.83
BOLA | -35.78 | 120.11 | 19.10 | 136.78
RLAGS | 66.61$\pm$4.94 | 92.68$\pm$2.66 | 9.74$\pm$1.92 | 16.33$\pm$7.69
RLAS | 57.55$\pm$3.99 | 105.69$\pm$1.77 | 17.54$\pm$1.49 | 30.59$\pm$3.76

Table. 5 shows the performance of the adaptation methods in the extreme case:
the average throughput of the first path is in $[1.5,2.0]$ Mbps, and of the
second path is less than 0.5 Mbps. Overall, RLAGS and RLAS outperform BOLA
method, and their rewards are much higher than those of the throughput-based
method. The reward of RLAGS is a bit higher than RLAS because it has fewer
actions in the action space. Hence, the agent may be easier to learn the
optimum. The throughput-based method has the least rebuffering; however, it
also has the lowest utility. BOLA has the highest utility but also the highest
rebuffering penalty. RLAGS balances the objectives: high utility, low number
of switches, and small rebuffering penalties.

Figure 6: A sample of throughput traces in the extreme case: the mean
bandwidths of the first path is in $[1.5,2.0]$ Mbps and of the second path is
less than 0.5 Mbps.

Figure 7: The quality levels and the buffer occupancy with the sample
throughput traces in Fig. 6.

Fig. 7 shows video quality levels selection and buffer occupancy when the
client experiences a pair of throughput traces, shown in Fig. 6, with
different methods. The video played by the RL-based methods is more stable
than by the other methods. We see that the proposed methods have a smarter
buffer occupancy so that they can download higher quality levels with fewer
switches than other methods.

##  4 Conclusions

We have proposed two novel adaptation and scheduling methods for video
streaming from multiple sources, i.e., RL-based adaptation and greedy
scheduling (RLAGS) and RL-based adaptation and scheduling (RLAS). The state
space, action space, and reward are defined for the methods. We have also
built a GymAI-compatible environment for training and evaluating. Extensive
simulations have shown that the proposed methods outperform the baseline
methods in terms of the user’s QoE. Model-free reinforcement learning
algorithms could not work well in transfer learning [27]. If we run the model
in an untrained environment, the model could yield a low reward. In the
future, we will apply model-based algorithms to bitrate adaptation.

## References

  * [1] Cisco: Cisco Visual Networking Index: Forecast and Methodology, 2016-2021.
  * [2] T. Stockhammer: Dynamic adaptive streaming over HTTP: standards and design principles. In Proceedings of the second annual ACM conference on Multimedia systems, 133-144. (2011)
  * [3] I. Sodagar: The MPEG-DASH Standard for Multimedia Streaming Over the Internet. IEEE MultiMedia, Vol. 18, Issue 4, 62-67. (2011)
  * [4] S. Lederer, C. Müller and C. Timmerer: Dynamic Adaptive Streaming over HTTP Dataset. In Proceedings of the ACM Multimedia Systems Conference, 22-24. (2012) Online: https://dash.itec.aau.at/dash-dataset/.
  * [5] ISO/IEC 23009-1:2014: Dynamic Adaptive Streaming over HTTP (DASH)– part 1: Media Description and Segments format.
  * [6] DASH Reference Client. Accessed: Jun. 28, 2019. [Online]. Available: https://reference.dashif.org/dash.js/
  * [7] J. Jiang, V. Sekar, and H. Zhang: Improving Fairness, Efficiency, and Stability in HTTP-based Adaptive Video Streaming with FESTIVE. In Proceedings of CoNEXT. (2012)
  * [8] K. Spiteri, R. Urgaonkar, and R. K. Sitaraman: BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of 35th Annual IEEE International Conference on Computer Communications (INFOCOM). (2016)
  * [9] T. Y. Huang, R. Johari, N. McKeown, M. Trunnell, and M. Watson: A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In Proceedings of the 2014 ACM conference on SIGCOMM, 187-198. (2014)
  * [10] Z. Li, X. Zhu, J. Gahm, R. Pan, H. Hu, A. C. Begen, and D. Oran: Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE Journal on Selected Areas in Communications, Vol. 32, No. 4, 719-733. (2014)
  * [11] Y.C. Chen, D. Towsley, and R. Khalili: MSPlayer: Multisource and multi-path video streaming. IEEE Journal on Selected Areas in Communications, Vol.34, Issue 8, 2198-2206. (2016)
  * [12] A. Nikravesh, Y. Guo, X. Zhu, F. Qian, and Z. M. Mao: MP-H2: a Client-only Multipath Solution for HTTP/2. In Proceedings of The 25th Annual International Conference on Mobile Computing and Networking, 1-16. (2019)
  * [13] A. Bentaleb, P.K. Yadav, W.T. Ooi, and R. Zimmermann: DQ-DASH: A Queuing Theory Approach to Distributed Adaptive Video Streaming. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 16, No. 1, 1-24. (2020)
  * [14] M. Claeys, S. Latre, J. Famaey, and F. De Turck, “Design and evaluation of a self-learning HTTP adaptive video streaming client,” IEEE communications letters, vol. 18, issue 4, pp. 716–719, 2014.
  * [15] H. Mao, R. Netravali, and M. Alizadeh: Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication, 197-210. (2017)
  * [16] M. Gadaleta, F. Chiariotti, M. Rossi, and A. Zanella: D-DASH: A deep Q-learning framework for DASH video streaming. IEEE Transactions on Cognitive Communications and Networking, Vol. 3, Issue 4, 703-718. (2017)
  * [17] D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley: Design, implementation and evaluation of congestion control for multipath TCP. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, Vol. 11, 8-8. (2011)
  * [18] C. Raiciu, M. Handley, and D. Wischik: Coupled congestion control for multipath transport protocols, RFC6356. (2011)
  * [19] Phuong Luu Vo, Tuan Anh Le, S. Lee, C. S. Hong, B. Kim, H. Song: mReno: a practical multipath congestion control for communication networks. Computing, Vol. 96, No. 3, 189-205. (2014)
  * [20] Nghia T. Nguyen, Phuong L. Vo, Thi Thanh Sang Nguyen, Quan M. Le, Cuong T. Do, and Ngoc-Thanh Nguyen: A Reinforcement Learning Framework for Multi-source Adaptive Streaming. In Proceedings of International Conference on Computational Collective Intelligence, 416-426. (2021)
  * [21] S. Huang and S. Ontanon: A Closer Look at Invalid Action Masking in Policy Gradient Algorithms. In Proceedings of the Thirty-Fifth International Florida Artificial Intelligence Research Society Conference, (FLAIRS 2022), Florida, USA, May 15-18. (2022)
  * [22] US Federal Communications Commission (FCC). [Online]. Available: https://data.fcc.gov/download/measuring-broadband-america/2019/data-raw-2019-sept.tar.gz
  * [23] Tenth Measuring Broadband America Fixed Broadband Report [Online]. Available: Measuring Fixed Broadband - Tenth Report | Federal Communications Commission (fcc.gov)
  * [24] D. Raca, J.J. Quinlan, A.H. Zahran, C.J. Sreenan: Beyond Throughput: a 4G LTE Dataset with Channel and Context Metrics. In Proceedings of ACM Multimedia Systems Conference (MMSys 2018), Amsterdam, The Netherlands. (2018)
  * [25] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov: Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347. (2017)
  * [26] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus and N. Dormann: Stable-Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research, Vol. 22, No. 268, 1-8. (2021)
  * [27] T. M. Moerland, J. Broekens, and C. M. Jonker: Model-based reinforcement learning: A survey. arXiv preprint arXiv:2006.16712. (2020)


# Sensing aided Channel Estimation in Wideband Millimeter-Wave MIMO Systems

Rakesh Mundlamuri1, Rajeev Gangula2, Christo Kurisummoottil Thomas3, Florian
Kaltenberger1 and Walid Saad3 1Communication Systems Department, EURECOM,
Biot, France 2Institute for the Wireless Internet of Things, Northeastern
University, Boston, USA 3 Wireless@VT, Bradley Department of Electrical and
Computer Engineering, Virginia Tech, Arlington, VA, USA

###### Abstract

In this work, the uplink channel estimation problem is considered for a
millimeter wave (mmWave) multi-input multi-output (MIMO) system. It is well
known that pilot overhead and computation complexity in estimating the channel
increases with the number of antennas and the bandwidth. To overcome this, the
proposed approach allows the channel estimation at the base station to be
aided by the sensing information. The sensing information contains an estimate
of scatterers locations in an environment. A simultaneous weighting orthogonal
matching pursuit (SWOMP) - sparse Bayesian learning (SBL) algorithm is
proposed that efficiently incorporates this sensing information in the
communication channel estimation procedure. The proposed framework can cope
with scenarios where a) scatterers present in the sensing information are not
associated with the communication channel and b) imperfections in the
scatterers’ location. Simulation results show that the proposed sensing aided
channel estimation algorithm can obtain good wideband performance only at the
cost of fractional pilot overhead. Finally, the Cramer-Rao Bound (CRB) for the
angle estimation and multipath channel gains in the SBL is derived, providing
valuable insights into the local identifiability of the proposed algorithms.

##  I Introduction

Millimeter wave (mmWave) and terahertz (THz) frequencies are considered to be
a key component of 5G and 6G cellular systems[1]. However, as the operating
frequencies increase, path and absorption losses also increase. Despite these
disadvantages, this approach will allow packing more antennas in a small area
and, then, the network can leverage beamforming techniques to compensate for
the losses operating in such frequencies. However, the gains stemming from
these multiple antenna techniques hinge on the ability to accurately estimate
the channel state information (CSI).

Estimating channel coefficients over a wideband and across multiple antennas
incurs significant resource overhead in terms of resources occupied for
sending pilot symbols. However, it has been observed that the mmWave channel
exhibits a sparse behavior with only a few resolvable multi-paths in angle and
delay domain [2] and [3]. By leveraging such sparsity, several works have come
with compressed sensing (CS) based approaches for channel estimation and
precoder design in mmWave multi-input multi-output (MIMO) systems[4, 5, 6, 7,
8, 9]. However, while used in wideband massive MIMO systems, these approaches
lead to higher complexity due to the requirement of inverting huge matrices
(for every subcarrier) across such antenna arrays.

Figure 1: Uplink multi-path scenario along with the co-located radar.

Since the sparse wireless channel is described by a few geometric multi-path
propagation parameters, one might ask: Can the information on the physical
propagation environment, for example, scatter or reflector locations, be
useful in channel estimation? Indeed, one of the earlier works in [10] has
utilized this key observation. The authors extract physical multi-path
parameters from the CSI measurements in one frequency band and then use them
to construct the CSI in another frequency band. However, no extra pilots are
used in aiding the channel estimation, and they assume that the extracted
multi-path parameters are perfect.

On the other hand, advances in radar and joint communication sensing made it
possible to have real-time dynamic radio environment maps at the communicating
devices. Prior works in [11, 12, 13, 14, 15, 16] use such radar environment
side information for the beam prediction and beam alignment to reduce the
initial synchronization time in vehicular systems. A recent work in [17] tried
to address the problem of channel estimation in massive MIMO systems by
leveraging radar sensing information. The authors retrieve the scatterers
location and velocity in the surrounding environment from the measurements
collected by a co-located radar at the base station (gNB). Then multi-path
parameters such as delays and angles are extracted from the radar sensing
information. These inferred multi-path parameters from the radar are then used
to initialize the dictionary in an orthogonal matching pursuit (OMP) based
channel estimation algorithm. However, the extracted multi-path parameters
from the radar are assumed to be error-free. The limitations of existing
literature on radar-based sensing for channel estimation in massive MIMO
systems include the following. Firstly, the sensing based channel estimation
algorithms may have limited angle and delay resolution, resulting in imprecise
channel estimation and affecting the system performance. Secondly, utilizing
separate spectrum for sensing and communication leads to inefficient resource
utilization. But this can be overcome by using full-duplex techniques [18],
which requires efficient signal processing to mitigate the self-interference
components. Thirdly, implementing sensing baded channel estimation in practice
can be difficult due to the increasing radio frequency (RF) components and
computational complexity, especially in massive MIMO systems with many
antennas. These limitations highlight the need for further research to improve
sensing based channel estimation’s accuracy, efficiency, and robustness in
massive MIMO systems. Specifically, this paper proposes novel signal
processing methods to bridge the gap on the limited angular and delay
resolution issue mentioned above.

In this work, we consider the problem of channel estimation in a wideband
mmWave MIMO system in which sensing information is obtained from a co-located
radar at the gNB as shown in Fig. 1 is used to reduce the pilot overhead. The
contributions of our paper are summarized as follows,

  1. 1.

Unlike [17], we assume that the sensing information from the radar can be
erroneous. We also consider cases in which scatterers detected from radar
might not be associated with the communication channel.

  2. 2.

To address these issues, a novel Simultaneous Weighting Orthogonal Matching
Pursuit (SWOMP) - Sparse Bayesian Learning (SBL) based channel estimation is
proposed that incorporates the imperfect sensing information from the radar.

  3. 3.

We also provide local identifiability analysis for the parameter estimation
using SBL by deriving Cramer-Rao bound (CRB) for the joint angle of arrival
(AoA) and path gain estimation using SBL.

##  II System Model

We consider a scenario where a user (UE) communicates with base station (gNB)
in an environment with the scatterers located between them. The scatterers are
represented by $\mathcal{S}_{r}$, and $|\mathcal{S}_{r}|=L_{r}$. Only a subset
of these scatterers, $\mathcal{S}_{c}\subseteq\mathcal{S}_{r}$,
$|\mathcal{S}_{c}|=L_{c}$, are assumed to affect the UE-gNB communication
channel. The set $\mathcal{S}_{c}$ is unknown, however, we assume that
location estimates of scatterers in $\mathcal{S}_{r}$ are provided by a
sensing system. This represents a scenario where the scatterers are present in
the blind zone to UE but can be detected by a sensing system co-located at the
gNB as shown in the Fig. 2.

###  II-A Sensing Information

We assume that the sensing is accomplished at the gNB either using a co-
located radar operating in a seperate spectrum [19] or through a joint
communication sensing framework [20]. The sensing information available at the
gNB is given by

$\\{(\tau_{\ell}^{rad}+e_{\tau},\theta_{l}+e_{\theta})~{}|~{}l=1,2,\ldots,L_{r}\\},$

where $\tau_{\ell}^{rad}$ and $\theta_{\ell}$ represent the round trip delay
and angle of the $l$-th scatterer from the gNB, respectively. The error in the
estimated parameters is assumed to be Gaussian distributed as
$e_{\theta}\sim\mathcal{N}(0,\sigma_{\theta}^{2})$ and
$e_{\tau}\sim\mathcal{N}(0,\sigma_{\tau}^{2})$. The error in the radar spatial
information can appear due to noise and the inability of the radar to resolve
delay and/or angles sufficiently.

###  II-B Communication Model

We consider a mmWave orthogonal frequency division multiplexing (OFDM) system
with a single antenna UE and $M$ antenna gNB. The gNB is equipped with a
uniform linear array (ULA) with half-wavelength spacing between consecutive
antennas.

The UE sends $P\ll K$ (narrowband) pilots, where $K$ is the total number of
subcarriers used for communication. The received complex baseband signal at
the $k$-th subcarrier after down-conversion, zero prefix removal, OFDM
demodulation, and correlation with the pilots is given by

$\displaystyle\bm{y}[k]=\bm{h}[k]+\bm{n}[k],$ (1)

where $k=0$ to $K-1$, $\bm{h}[k]\in\mathbb{C}^{M\times 1}$ represent the
baseband channel, $\bm{n}[k]\sim\mathcal{CN}(0,\sigma^{2}\bm{I}_{M})$ is a
circularly symmetric complex Gaussian distributed additive noise vector. We
define the received signal-to-noise-ratio (SNR) at subcarrier $k$ as
${\left\lVert{\bm{h}}[k]\right\rVert^{2}}/{\sigma^{2}}$. Next, we describe the
mmWave channel model generation that is a parametric function of the multipath
components.

###  II-C Channel Model

A frequency-selective geometric channel model with $N_{c}$ delay taps and
$L_{c}+1$ paths[7] is considered. The channel consists of a line-of-sight
(LoS) component, and $L_{c}$ (yet unknown) reflections resulting from the
scatterers as described earlier. The $d$-th delay tap is modeled as

$\displaystyle\bm{h}_{d}=\sqrt{\frac{M}{L_{c}+1}}\sum\limits_{\ell=0}^{L_{c}}\alpha_{\ell}p(dT_{s}-\tau_{\ell})\bm{a}(\theta_{\ell}),$
(2)

where $p(.)$ is the pulse-shaping filter, $T_{s}$ is the sampling interval,
$\alpha_{\ell}$, $\tau_{\ell}$, $\theta_{\ell}$ represent the path gain, delay
and the angle-of-arrival (AoA) of the $l$-th path, respectively. The receiver
array steering vector for the $l$-th path is denoted by
$\bm{a}(\theta_{\ell})\in\mathbb{C}^{M\times 1}$. The index $\ell{=}0$ is
always associated with the LoS path. We can compactly represent the channel as
$\bm{h}_{d}{=}\bm{A}\bm{\Delta}_{d},$ where $\bm{A}{=}[\bm{a}(\theta_{0})\
\bm{a}(\theta_{1})\ \dots\
\bm{a}(\theta_{L_{c}})]\in\mathbb{C}^{M\times\left(L_{c}+1\right)}$ contains
the receiver side steering vectors and

$\displaystyle\bm{{\Delta}}_{d}=\left[{\begin{array}[]{c}\alpha_{0}p(dT_{s}-\tau_{0}),\cdots,\alpha_{L_{c}}p(dT_{s}-\tau_{L_{c}})\end{array}}\right]^{{\operatorname{T}}}.$
(4)

We obtain the frequency domain channel representation by taking a $K$-point
DFT of the delay-domain channel, and the channel at subcarrier $k$ can be
written as

$\displaystyle\bm{h}[k]=\sum\limits_{d=0}^{N_{c}-1}\bm{h}_{d}\exp\left({-\frac{j2\pi
kd}{K}}\right)=\bm{A}\bm{{\Delta}}[k],$ (5)

and $\bm{{\Delta}}[k]$ is given by
$\bm{{\Delta}}[k]=\sum\limits_{d=0}^{N_{c}-1}\bm{{\Delta}}_{d}\exp\left({-\frac{j2\pi
kd}{K}}\right).$ Further substituting for $\bm{{\Delta}}_{d}$ from (4), we
obtain

$\displaystyle\bm{\Delta}[k]=\left[{\beta_{k,0}\alpha_{0}},{\beta_{k,1}\alpha_{1}},\dots,{\beta_{k,L_{c}}\alpha_{L_{c}}}\right]^{{\operatorname{T}}},$
(6)

where
$\beta_{k,\ell}{=}\sum_{d=0}^{N_{c}-1}p(dT_{s}-\tau_{\ell})\exp\left({-\frac{j2\pi
kd}{K}}\right)$. Substituting $\bm{{\Delta}}[k]$ in (5), a compact form of the
frequency domain channel $\bm{h}[k]$ can be obtained as

$\displaystyle\bm{h}[k]=\bm{A}\bm{\beta}_{k}\bm{\alpha},$ (7)

where
$\bm{\beta}_{k}{=}\textrm{diag}\big{(}\beta_{k,0},\beta_{k,1},\dots,\beta_{k,{L_{c}-1}}\big{)}$,
and
$\bm{\alpha}{=}[\alpha_{0},\alpha_{1},\dots,\alpha_{L_{c}}]^{{\operatorname{T}}}$.
Further substituting $\bm{h}[k]$ in (1), the received frequency domain signal
$\bm{y}[k]$ can be written as

$\displaystyle\bm{y}[k]=\bm{\Psi}_{k}\bm{\alpha}+\bm{n}[k],$ (8)

where
$\bm{\Psi}_{k}=\bm{A}\bm{\beta}_{k}\in\mathbb{C}^{M\times\left(L_{c}+1\right)}$.

Figure 2: Scatterer environment along with the sensing information.

##  III Sensing Aided Channel Estimation

In this section, we provide a channel estimation framework that incorporates
the sensing information available at the gNB. From (8), the received $P$
pilots in vectorized form is given by

$\displaystyle\bm{y}=\left[\bm{y}^{{\operatorname{T}}}[0]\,\bm{y}^{{\operatorname{T}}}[1]\,\ldots\,\bm{y}^{{\operatorname{T}}}[P-1]\right]^{{\operatorname{T}}},$
(9)
$\displaystyle\bm{y}=\underbrace{\left[{\bm{\Psi}_{0}^{{\operatorname{T}}}}{\bm{\Psi}_{1}^{{\operatorname{T}}}}\cdots{\bm{\Psi}_{P-1}^{{\operatorname{T}}}}\right]^{{\operatorname{T}}}}_{{\bm{\Omega}}}\bm{\alpha}+{\bm{n}},$
(10)

where the matrix $\bm{\Omega}\in\mathbb{C}^{MP\times(L_{c}+1)}$ carries the
delay-angle information of the multipath components and ${\bm{n}}$ is the
vectorized noise
${\bm{n}}=[{\bm{n}}^{{\operatorname{T}}}[0]{\bm{n}}^{{\operatorname{T}}}[1]\cdots{\bm{n}}^{{\operatorname{T}}}[P-1]]^{{\operatorname{T}}}$.

Moreover, the sensing information can be used as an initial estimate of the
multipath delays and angles. Let
$\tilde{\bm{\theta}}=[{\theta}_{0},\tilde{\theta}_{1},\tilde{\theta}_{2},\ldots,\tilde{\theta}_{L_{r}}]$,
where ${\theta}_{0}$ is the angle associated with the LoS path and
$\tilde{\theta}_{l}$, $l\in[1,L_{r}]$ is the AoA of the $l$-th path obtained
from the sensing information. The round-trip propagation delay between the gNB
and the $l$-th, $l\in[1,L_{r}]$, scatterer is denoted by $\tau^{rad}_{\ell}$.
Let us define
$\tilde{\bm{\tau}}=[{\tau_{0}},\tilde{\tau}_{1},\tilde{\tau}_{2},\ldots,\tilde{\tau}_{L_{r}}]$,
where $\tau_{0}$ is the delay between the UE and the gNB, and the delay of the
$\ell$-th communication path can be estimated using the radar delay
$\tau^{rad}_{\ell}$ as

$\tilde{\tau}_{\ell}={\tau^{rad}_{\ell}}/{2}+\tau^{\prime}_{\ell},$ (11)

where $\tau^{\prime}_{\ell}$ is obtained using triangle laws of cosines as
shown in Fig. 3,

$\displaystyle\tau^{\prime}_{\ell}=\sqrt{\tau_{0}^{2}+\left({\tau^{rad}_{\ell}}/{2}\right)^{2}-\tau_{0}(\tau^{rad}_{\ell})cos(\tilde{\theta}_{\ell}-\theta_{0})}.$
(12)

Similar to the matrix $\bm{\Omega}$ in (10), using the sensing information
$(\tilde{\bm{\tau}},\tilde{\bm{\theta}})$, we can construct a matrix
$\bar{\bm{\Omega}}\in\mathbb{C}^{MP\times(L_{r}+1)}$ that captures the delay-
angle information of the $L_{r}+1$ paths. As we described earlier, only a
subset of $L_{c}$ among the $L_{r}$ scatterers are included in the
communication channel, and $L_{c}$ is unknown. This can be mathematically
represented as,

$\vspace{-1mm}\bm{\Omega}=\bar{\bm{\Omega}}\bm{B}+\bm{E},\vspace{-1mm}$ (13)

where $\bm{B}\in\mathbb{R}^{(L_{r}+1)\times(L_{c}+1)}$ is obtained by
selecting $L_{c}+1$ columns of the identity matrix $\bm{I}_{L_{r}+1}$. The
indices of the columns that are included in $\bm{B}$, correspond to the paths
that are present both in the communication channel and sensing information.
The unknown error term is denoted by $\bm{E}$.

###  III-A Problem Formulation

Utilizing the received pilot signal (10) and the sensing information in the
form of (13), the maximum a posteriori (MAP) based channel estimation problem
is formulated as:

$\displaystyle\vspace{-1mm}\left[\bm{\Omega}^{*},\bm{\alpha}^{*}\right]=\arg\max\limits_{\bm{\Omega},\bm{\alpha}}p(\bm{\Omega},\bm{\alpha}\mid\bm{y}),\vspace{-1mm}$
(14)

where $p(.)$ represents the probability distribution and $\bm{\alpha}$ is the
channel gain vector.

The optimization problem at hand is difficult to solve in general as a) it is
hard to obtain the distribution $p(\bm{\Omega},\bm{\alpha}\mid\bm{y})$ b) the
combinatorial nature of the path association matrix $\bm{B}$ and the unknown
error. A conventional approach to relax this problem and solve it using
compressed sensing schemes, such as SBL, by considering a joint dictionary
matrix consisting of finely spaced angles and delays. However, such a solution
results in cubic complexity with respect to the dictionary dimensions, which
has to be finely spaced to alleviate the off-grid errors. Hence, we propose a
two-stage SWOMP-SBL algorithm to overcome such high complexity.

Figure 3: Communication delay estimation from radar delay.

##  IV SWOMP-SBL Algorithm

The proposed algorithm works in two stages. In the first stage, based on the
sensing information, a SWOMP based algorithm is used to find the paths that
are associated with the communication and their respective AoAs. Based on
these selected paths, a SBL inference algorithm is used to obtain finer
estimate of the delays and corresponding channel gains
$\widehat{\bm{\alpha}}$. A schematic describing this two-stage algorithm is
shown in Fig 4.

Figure 4:  SWOMP-SBL algorithm.

###  IV-A SWOMP Stage

The algorithm is initialized with assuming that all the $L_{r}+1$ paths from
the sensing information are present in the communication channel. The AoA’s
$\tilde{\bm{\theta}}$ are used to form the angle dictionary $\bm{A}^{\prime}$
as described in steps 2 and 3 of the Algorithm 1. The SWOMP algorithm [21]
outputs the maximum correlated paths $\hat{\bm{\theta}}$ corresponding to the
angle dictionary $\bm{A}^{\prime}$ with the received signal $\bm{y}$. The
noise variance $\sigma^{2}$ is utilized as a stopping condition in SWOMP,
where all the refined angles associated with the channel and their
corresponding path indices $\bm{\chi}$ are estimated. However, a dictionary
matrix $\widehat{\bm{\Omega}}$ is needed to refine the delays further and
estimate the channel gains. The path association matrix $\bm{B}$ can be
obtained from the estimated $\bm{\chi}$, but it’s avoided since the path
indices are enough to create the dictionary matrix $\widehat{\bm{\Omega}}$.
$\widehat{\bm{\Omega}}$ is constructed using the refined AoA
$\hat{\bm{\theta}}$ obtained using SWOMP and a finely space dictionary matrix
of the associated delays. The association of the path is given by the path
indices $\bm{\chi}$ and maps the refined angles to their corresponding delays.
The details of our algorithm are discussed in Algorithm 1. The refinement of
the delays $\tilde{\bm{\tau}}$ and their corresponding channel gains
$\bm{\alpha}$ are estimated using SBL with the obtained
$\widehat{\bm{\Omega}}$ in the next stage. The computational complexity of
SWOMP in each iteration is
$MP(d_{\theta}L^{\prime})^{2}+MPd_{\theta}L_{r}+MPd_{\theta}L^{\prime}$.

###  IV-B SBL Stage

Recalling the measurement equation with the obtained $\widehat{\bm{\Omega}}$,
we write $\bm{y}=\widehat{\bm{\Omega}}{{{\bm{\alpha}}}}+{{{\bm{n}}}}$. We
formulate the estimation method of $(\bm{\alpha},\tilde{\bm{\tau}})$ using SBL
as follows.

SBL is a type-II maximum likelihood (ML) estimation procedure to obtain the
channel estimate [22, 23]. In this method, ${\bm{\alpha}}$ is considered as a
hidden variable, and posterior statistics are obtained given the observations.
SBL assumes a complex Gaussian prior distribution for the entries of
${\bm{\alpha}}$, which gets written as
$p(\alpha_{i}){=}\frac{\gamma_{i}}{\pi}^{-\gamma_{i}(\alpha_{i})^{2}}.$ The
hyperparameters $\alpha_{i}$ also estimated using the inference procedure.
$\gamma_{i}$ is assumed to follow a Gamma distribution,
$\mathcal{G}(\gamma_{i};a,b){=}\Gamma((\gamma_{i}))^{-1}(\gamma_{i})^{a-1}e^{-b\gamma_{i}}b^{\gamma_{i}}$.
Defining, $\bm{\Gamma}{=}\mbox{diag}(\bm{\gamma})$, where $\bm{\gamma}$ is the
vector of $\gamma_{i}$. Noise is assumed to be complex Gaussian,
$\mathcal{CN}(\bf{0},\frac{1}{\zeta}{\bm{I}})$. $\zeta$ is assumed to have
Gamma as a prior distribution such that
$p_{\zeta}(\zeta){=}\mathcal{G}(\zeta;c,d)$, where $c,d$ are known. Note that
in the case of an uninformative prior, the values of $a$ and $b$ corresponds
to 1 and 0 respectively. Now, the posterior distribution of $\bm{\alpha}$ and
the hyper-parameters $\bm{\gamma},\zeta$ needs to be obtained. Since the prior
and the noise are both Gaussian, obtaining the posterior statistics of
$\bm{\alpha}$ is straightforward. But, the computation of $\bm{\gamma}$
requires the computation of the marginal probability distribution
$p(\bm{y};\bm{\gamma},\zeta)$ and maximizing it (alternatively) w.r.t.
$\bm{\gamma},\zeta$. This procedure is known as evidence maximization or type-
II ML estimation.

To solve this, expectation-maximization (EM) algorithm is used, which proceeds
by lower bounding the logarithm of the evidence $p(\bm{y};\bm{\gamma},\zeta)$,
and maximizing it iteratively. Treating $\bm{\alpha}$ as a hidden variable, In
the expectation (E) step, expectation of the log likelihood of
$(\bm{y},\bm{\alpha})$ w.r.t. $p(\bm{\alpha}|\bm{y},\bm{\gamma},\zeta)$ is
computed. In the maximization (M) step, the hyper-parameters
$\bm{\gamma},\zeta$ are computed by maximizing the function obtained in the E
step. More details of SBL and type-II ML estimation can be found in [22].
Detailed steps for the channel estimation are provided in Algorithm 2. The SBL
algorithm outputs the estimate of the channel gains $\widehat{\bm{\alpha}}$.
Using step 17 in Algorithm 1, the channel estimate $\hat{\bm{h}}$ at the
$k$-th subcarrier can be obtained by
$\hat{\bm{h}}_{k}{=}\hat{\bm{\Psi}}_{k}\widehat{\bm{\alpha}}$ for all the $K$
subcarriers. The convergence properties of the SBL algorithm are well
understood in the literature [22]. In short, using similar arguments in [22],
we can show that the proposed SBL converges to the sparsest solution when the
noise variance is zero and to a sparse local minimum, irrespective of the
noise variance. The computational complexity of each iteration of SBL is
$(MP)^{3}+2(MP)^{2}(d_{\tau}L^{\prime})+2(d_{\tau}L^{\prime})^{2}MP+(d_{\tau}L^{\prime})^{2}+4MP(d_{\tau}L^{\prime})+(d_{\tau}L^{\prime})$.

Algorithm 1 SWOMP Stage

1:$\bm{y},\tilde{\bm{\theta}},\tilde{\bm{\tau}},d_{\theta},d_{\tau},\sigma_{\theta},\sigma_{\tau},\sigma^{2}$

2:Initialize: $\bm{\chi}=\\{\\}$

3:$\bm{\theta}_{l}^{\prime}=\tilde{\theta}_{l}-2\sigma_{\theta}:\frac{4\sigma_{\theta}}{d_{\theta}}:\tilde{\theta}_{l}+2\sigma_{\theta}\in\mathbb{R}^{1\times
d_{\theta}}$

4:$\bm{A}^{\prime}$=$[a(\bm{\theta}_{0}^{\prime})\
a(\bm{\theta}_{1}^{\prime})\ \cdots\
a(\bm{\theta}_{L_{r}}^{\prime})]\in\mathbb{C}^{M\times d_{\theta}(L_{r}+1)}$

5:$\hat{\bm{\theta}}=\textbf{SWOMP}(\bm{y},\bm{A}^{\prime},\sigma^{2})$

6: $\hat{\bm{\theta}}\in\mathbb{R}^{1\times L^{\prime}}$ is
$\\{\hat{\theta}_{\ell}\ |\ \ell=0,1,\cdots,L^{\prime}-1\\}$

7: Path association:

8:for $\ell=0:L^{\prime}-1$ do

9: $p=\arg\min\limits\\{|\theta_{\ell}\bm{1}-\bm{\theta}|\\}$ $\triangleright$
$\bm{1}\in 1^{1\times(L_{r}+1)}$

10: $\bm{\chi}=\bm{\chi}\cup p$ $\triangleright$ $p{\ =}\ $path index

11:end for

12:$\tilde{\bm{\tau}}(\bm{\chi})\in\mathbb{R}^{1\times L^{\prime}}$ is
$\\{\tilde{\tau}_{\ell}(\bm{\chi})\ |\ \ell=0,1,\cdots,L^{\prime}-1\\}$
$\triangleright$ delays of the corresponding path index obtained in step 9

13:$\hat{\bm{\tau}}_{\ell}=\tilde{\tau}_{\ell}(\bm{\chi})-2\sigma_{\tau}:\frac{4\sigma_{\tau}}{d_{\tau}}:\tilde{\tau}_{\ell}(\bm{\chi})+2\sigma_{\tau}\in\mathbb{R}^{1\times
d_{\tau}}$

14:$\hat{\bm{\tau}}=[\hat{\bm{\tau}}_{0}\ \hat{\bm{\tau}}_{1}\ \cdots\
\hat{\bm{\tau}}_{L^{\prime}-1}]\in\mathbb{R}^{1\times d_{\tau}L^{\prime}}$

15:The resulting $\bm{\beta}_{k}$ obtained using $\hat{\bm{\tau}}$ is denoted
as $\hat{\bm{\beta}}_{k}\in\mathbb{C}^{d_{\tau}L^{\prime}\times
d_{\tau}L^{\prime}}$

16:$\hat{\bm{A}}_{\ell}=[a(\hat{\theta}_{\ell})a(\hat{\theta}_{\ell})\cdots
a(\hat{\theta}_{\ell})]\in\mathbb{C}^{M\times d_{\tau}}$$\triangleright$
Repeat $d_{\tau}$ times

17:$\hat{\bm{A}}=[\hat{\bm{A}}_{0}\ \hat{\bm{A}}_{1}\ \cdots\
\hat{\bm{A}}_{L^{\prime}-1}]\in\mathbb{C}^{M\times d_{\tau}L^{\prime}}$

18:$\hat{\bm{\Psi}}_{k}=\hat{\bm{A}}\hat{\bm{\beta}}_{k}\in\mathbb{C}^{M\times
d_{\tau}L^{\prime}}$

19:$\widehat{\bm{\Omega}}=[\hat{\bm{\Psi}}_{0}^{{\operatorname{T}}}\hat{\bm{\Psi}}_{1}^{{\operatorname{T}}}\cdots\hat{\bm{\Psi}}_{{P-1}}^{{\operatorname{T}}}]^{{\operatorname{T}}}\\!\in\\!\mathbb{C}^{MP\\!\times
d_{\tau}L^{\prime}}$

Algorithm 2 SBL Stage

1:$\bm{y}$, $\widehat{\bm{\Omega}}$,
$p_{\bm{y}}(\bm{y}\mid\widehat{\bm{\Omega}},\bm{\alpha})$

2:Initialize: $t=0$, $(\widehat{\bm{\alpha}})^{t}$ using LS estimate.

3:repeat

4: [Estimate $\bm{\alpha}$]

5:
$\bm{\Sigma}_{y}^{t}=\frac{1}{\zeta^{t-1}}{\bm{I}}+\widehat{\bm{\Omega}}\bm{\Gamma}^{-1}\widehat{\bm{\Omega}}^{H}$.

6:
$\widehat{\bm{\Sigma}}^{t}=\bm{\Gamma}^{-1}-\bm{\Gamma}^{-1}\widehat{\bm{\Omega}}^{H}\left(\bm{\Sigma}_{y}^{t}\right)^{-1}\widehat{\bm{\Omega}}\bm{\Gamma}^{-1}$.

7:
$(\widehat{\bm{\alpha}})^{t}=\zeta^{t-1}\widehat{\bm{\Sigma}}^{t}\widehat{\bm{\Omega}}^{H}\bm{y}$.

8: [Hyper-parameters Update]

9:
$\gamma_{i}^{t}=\frac{2a-1}{\mathbb{E}\left((\widehat{\alpha}_{i})^{2}\right)+2b}$.

10:
$\zeta^{t}=\frac{2c-1}{\frac{\mathbb{E}(\lvert\bm{y}-{\bm{z}}\rvert^{2})}{MP}+2c},z_{i}=\widehat{\bm{\Omega}}\widehat{\bm{\alpha}}$.

11:until Convergence

###  IV-C Identifiability of the Proposed SBL: Minimum Narrowband Pilots
Required?

This subsection provides conditions under which the sensing-aided channel
estimation using SBL becomes locally identifiable. Furthermore, the analysis
herein provides the minimum pilots required for the respective channel
estimation algorithm to be identifiable. The distribution of $\bm{y}[k]$,
after marginalizing w.r.t $\bm{\alpha}$ can be written as

$\displaystyle
p_{\bm{y}}(\bm{y}[k])=\mathcal{CN}(\bm{0},{\bm{\Psi}}_{k}\bm{\Gamma}^{-1}({\bm{\Psi}}_{k})^{H}+\zeta^{-1}{\bm{I}}).\vspace{-2mm}$
(15)

The signal model in (8) is non-identifiable if
${\bm{\Psi}}_{k}\bm{\Gamma}^{-1}_{1}({\bm{\Psi}}_{k})^{H}{=}{\bm{\Psi}}_{k}\bm{\Gamma}^{-1}_{2}({\bm{\Psi}}_{k})^{H}$
for some $\bm{\Gamma}^{-1}_{1}\neq\bm{\Gamma}^{-1}_{2}$. The rank of
${\bm{\Psi}}_{k}^{{\operatorname{T}}}\otimes{\bm{\Psi}}$ is denoted as
$R\leq\left(MP\right)$, where $\otimes$ represents the Khatri-Rao product.
Following similar analysis[24], we can show that the SBL algorithm is
identifiable as long as $L$ (number of nonzero elements in $\bm{\alpha}$) is
$\mathcal{O}(R^{2})$ (${=}\mathcal{O}(\left(MP\right)^{2})$ for suitable
${\bm{\Psi}}_{k}^{{\operatorname{T}}}$). For a mmWave system, this would be
just fewer pilots, compared to using number of pilots of the $\mathcal{O}(K)$
as in existing 5G-NR algorithms.

Next, we look at the CRB of the estimation model here. The local
identifiability (upto permutation ambiguity) of the SBL based parameter
estimation is ensured if the Fisher information matrix (FIM) is non-singular
[25]. First, the estimated parameters are defined in a vector as
$\bm{\Theta}{=}[\bm{\theta},\bm{\alpha},\bm{\gamma},\zeta,\bm{\tau}]$. The FIM
can be partitioned as

$\displaystyle\vspace{-2mm}\small{\bm{J}}_{\bm{\Theta}\bm{\Theta}}=\begin{bmatrix}{\bm{J}}_{\bm{\theta}\bm{\theta}}&{\bm{J}}_{\bm{\theta}\bm{\alpha}}&{\bm{J}}_{\bm{\theta}\bm{\gamma}}&{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\\\
{\bm{J}}_{\bm{\alpha}\bm{\theta}}&{\bm{J}}_{\bm{\alpha}\bm{\alpha}}&{\bm{J}}_{\bm{\alpha}\bm{\gamma}}&{\bm{J}}_{\bm{\alpha}\zeta}&{\bm{J}}_{\bm{\alpha}\bm{\tau}}\\\
{\bm{J}}_{\bm{\gamma}\bm{\theta}}&{\bm{J}}_{\bm{\gamma}\bm{\alpha}}&{\bm{J}}_{\bm{\gamma}\bm{\gamma}}&{\bm{J}}_{\bm{\gamma}\zeta}&{\bm{J}}_{\bm{\gamma}\bm{\tau}}\\\
{\bm{J}}_{\zeta\bm{\theta}}&{\bm{J}}_{\zeta\bm{\alpha}}&{\bm{J}}_{\zeta\bm{\gamma}}&{\bm{J}}_{\zeta\zeta}&{\bm{J}}_{\zeta\bm{\tau}},\\\
{\bm{J}}_{\bm{\tau}\bm{\theta}}&{\bm{J}}_{\bm{\tau}\bm{\alpha}}&{\bm{J}}_{\bm{\tau}\bm{\gamma}}&{\bm{J}}_{\bm{\tau}\zeta}&{\bm{J}}_{\bm{\tau}\bm{\tau}}\end{bmatrix},\vspace{-5mm}$
(16)

where ${\bm{J}}_{{\bm{x}}{\bm{y}}}{=}\mathbb{E}\left(\frac{\partial\ln
p({\bm{y}},{\bm{x}})}{\partial{\bm{x}}}\frac{\partial\ln
p({\bm{y}},{\bm{x}})}{\partial{\bm{y}}}^{{\operatorname{T}}}\right)$. Each of
the FIM blocks can be derived as (detailed derivations are skipped since those
follows classical results in estimation theory)

$\displaystyle\vspace{-4mm}{\bm{J}}_{\bm{\theta}\bm{\theta}}$
$\displaystyle=\mathbb{E}(\zeta)({\bm{\beta}}_{k})^{H}\mathbb{E}(\frac{\partial{\bm{A}}(\bm{\theta})}{\partial\bm{\theta}}^{H}\frac{\partial{\bm{A}}(\bm{\theta})}{\partial\bm{\theta}}){\bm{\beta}}_{k}\mathbb{E}(\bm{\Gamma}^{-1}),$
(17) $\displaystyle{\bm{J}}_{\bm{\theta}\zeta}$
$\displaystyle=\mbox{diag}\left(\Re\\{({\bm{\beta}}_{k})^{H}\frac{\partial{\bm{A}}(\bm{\theta})}{\partial\bm{\theta}}^{H}{\bm{A}}{\bm{\beta}}_{k}\\}\mathbb{E}(\bm{\Gamma}^{-1})\right),$
(18) $\displaystyle{\bm{J}}_{\bm{\gamma}\bm{\gamma}}$
$\displaystyle=-\mathbb{E}(\bm{\Gamma}^{-1})+(a-1)\mathbb{E}(\bm{\Gamma}^{-1}),$
(19) $\displaystyle{\bm{J}}_{\zeta\zeta}$
$\displaystyle=-MP\mathbb{E}(\zeta^{-2})+(c-1)\mathbb{E}(\zeta^{-1}),$ (20)
$\displaystyle{\bm{J}}_{\bm{\alpha}\bm{\alpha}}$
$\displaystyle=-\mathbb{E}(\bm{\Gamma})-({\bm{\beta}}_{k})^{H}({\bm{A}})^{H}{\bm{A}}{\bm{\beta}}_{k}\mathbb{E}(\zeta),$
(21) $\displaystyle{\bm{J}}_{\zeta\bm{\tau}}$
$\displaystyle=\frac{\partial({\bm{\beta}}_{k})^{H}}{\partial\bm{\tau}}\mathbb{E}\left({\bm{A}}(\bm{\theta})^{H}{\bm{A}}(\bm{\theta})\right){\bm{\beta}}_{k}\mathbb{E}(\bm{\Gamma}^{-1}),$
(22) $\displaystyle{\bm{J}}_{\bm{\tau}\bm{\tau}}$
$\displaystyle=\mathbb{E}(\zeta)\frac{\partial({\bm{\beta}}_{k})^{H}}{\partial\bm{\tau}}\mathbb{E}\left({\bm{A}}(\bm{\theta})^{H}{\bm{A}}(\bm{\theta})\right)\frac{\partial{\bm{\beta}}_{k}}{\partial\bm{\tau}}\mathbb{E}(\bm{\Gamma}^{-1}),\vspace{-4mm}$
(23) $\displaystyle\vspace{-3mm}{\bm{J}}_{\bm{\theta}\bm{\tau}}$
$\displaystyle=\mathbb{E}(\zeta)\frac{\partial({\bm{\beta}}_{k})^{H}}{\partial\bm{\tau}}\mathbb{E}\left({\bm{A}}(\bm{\theta})^{H}\frac{\partial{\bm{A}}(\bm{\theta})}{\partial\bm{\theta}}\right){\bm{\beta}}_{k}\mathbb{E}(\bm{\Gamma}^{-1}),\vspace{-5mm}$
(24)

and rest of the terms result to be zero.

$\displaystyle\vspace{-3mm}\small{\bm{J}}_{\bm{\Theta}\bm{\Theta}}=\begin{bmatrix}{\bm{J}}_{\bm{\theta}\bm{\theta}}&\bm{0}&\bm{0}&{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\\\
\bm{0}&{\bm{J}}_{\bm{\alpha}\bm{\alpha}}&\bm{0}&\bm{0}&\bm{0}\\\
\bm{0}&\bm{0}&{\bm{J}}_{\bm{\gamma}\bm{\gamma}}&\bm{0}&\bm{0}\\\
{\bm{J}}_{\zeta\bm{\theta}}&\bm{0}&\bm{0}&{\bm{J}}_{\zeta\zeta}&{\bm{J}}_{\zeta\bm{\tau}}\\\
{\bm{J}}_{\bm{\tau}\bm{\theta}}&\bm{0}&\bm{0}&{\bm{J}}_{\bm{\tau}\zeta}&{\bm{J}}_{\bm{\tau}\bm{\tau}}\end{bmatrix}.\vspace{-2mm}$
(25)

$\displaystyle
CRB(\bm{\theta},\bm{\alpha})=\left(\begin{bmatrix}{\bm{J}}_{\bm{\theta}\bm{\theta}}&\bm{0}\\\
\bm{0}&{\bm{J}}_{\bm{\alpha}\bm{\alpha}}\end{bmatrix}-\begin{bmatrix}\tiny\bm{0}&{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\\\
\bm{0}&\bm{0}&\bm{0}\end{bmatrix}\begin{bmatrix}\tiny{\bm{J}}_{\bm{\gamma}\bm{\gamma}}^{-1}&\bm{0}\\\
\bm{0}&\underbrace{\begin{bmatrix}{\bm{J}}_{\zeta\zeta}&{\bm{J}}_{\zeta\bm{\tau}}\\\
{\bm{J}}_{\bm{\tau}\zeta}&{\bm{J}}_{\bm{\tau}\bm{\tau}}\end{bmatrix}^{-1}}_{{\bm{F}}_{\zeta,\bm{\tau}}}\end{bmatrix}\begin{bmatrix}\tiny\bm{0}&{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\\\
\bm{0}&\bm{0}&\bm{0}\end{bmatrix}^{{\operatorname{T}}}\right)^{-1},\vspace{-2mm}$

The CRB for $\bm{\Theta}$ can be expressed as
$CRB(\bm{\Theta}){=}{\bm{J}}_{\bm{\Theta}\bm{\Theta}}^{-1}$. The CRB for AoA
estimates and $\bm{\alpha}$ can be written using Schur-complement for
inverting a block matrix as derived in (IV-C), which can be simplified as

$\displaystyle CRB(\bm{\theta},\bm{\alpha})=$ (26)
$\displaystyle\begin{bmatrix}\left({\bm{J}}_{\bm{\theta}\bm{\theta}}-\begin{bmatrix}{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\end{bmatrix}{\bm{F}}_{\zeta\bm{\tau}}^{-1}\begin{bmatrix}{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\end{bmatrix}^{{\operatorname{T}}}\right)^{-1}&\bm{0}\\\
\bm{0}&{\bm{J}}_{\bm{\alpha}\bm{\alpha}}^{-1}\end{bmatrix}.\vspace{-6mm}$

Following similar derivations, the CRB can be computed for
$\bm{\alpha},\bm{\gamma},\zeta$. From (26), we can conclude that for local
identifiability of $\bm{\theta},\bm{\alpha}$,
${\bm{J}}_{\bm{\theta}\bm{\theta}}-\begin{bmatrix}{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\end{bmatrix}{\bm{F}}_{\zeta\zeta}^{-1}\begin{bmatrix}{\bm{J}}_{\bm{\theta}\zeta}&{\bm{J}}_{\bm{\theta}\bm{\tau}}\end{bmatrix}^{{\operatorname{T}}}$,
and ${\bm{J}}_{\bm{\alpha}\bm{\alpha}}$ should be invertible, respectively.

##  V Simulation Results

In this section, the performance of our novel SWOMP-SBL sensing aided channel
estimation algorithm is evaluated through numerical simulations. The system
parameters considered here are, subcarrier spacing${=}120\,\textrm{KHz}$,
center frequency${=}28\,\textrm{GHz}$, sampling rate${=}30.72\,\textrm{MHz}$,
sampling period $T_{s}{=}32.552\,\textrm{ns}$, fft size $K{=}256$, cyclic
prefix length $N_{cp}{=}34$ and the number of receive antennas $M{=}32$. The
pilots are generated similar to the sounding reference signals (SRS) in 5G
standards [26]. The location of the pilots in the OFDM grid are arranged in a
comb fashion as defined in the 3GPP standard[26] i.e., one pilot for every
$K_{c}$ subcarriers as shown in Fig. 5. The channel is generated using a ray-
tracing tool using the locations of gNB and UE with the number of delay taps
$N_{c}{=}N_{cp}$. The erroneous sensing information is generated with
$\sigma_{\theta}{=}3^{\circ}$ and $\sigma_{\tau}{=}\frac{Ts}{6}$.

In our scenario, pilots of size $P{=}16$ are transmitted with a comb size
$K_{c}{=}16$. The erroneous AoA from the sensing information are refined using
SWOMP with a dictionary matrix considering $d_{\theta}{=}500$. Further, the
channel gains $\widehat{\bm{\alpha}}$ are estimated using SBL considering
$d_{\tau}{=}50$. The channel estimation procedure with erroneous sensing
information is denoted by SWOMP-SBL + Sensing Info Error in the plot. The
channel gains $\widehat{\bm{\alpha}}$ are also estimated using LS assuming
perfect sensing information is available at the gNB denoted by Ideal Sensing
Info + LS.

The performance of our proposed algorithms is evaluated by comparing the
normalized mean squared error (NMSE) of the channel using fewer $P$ pilots to
the NMSE of the channel obtained by transmitting all the $K=256$ pilots
(wideband). The channel estimation using the wideband pilots is performed
using the classical LS method and SWOMP denoted as WB+LS and WB+SWOMP
respectively. The dictionary used for both algorithms is of size 500,
discretized in the angular domain in $[0^{\circ},180^{\circ}]$.

We consider a scenario where all the scatterers provided by the sensing
information in $\mathcal{S}_{r}$ might not be associated with the
communication channel. The parameters of the scatterers chosen for the
simulations are $L_{r}{=}10$ and $L_{c}{=}6$. From Fig. 6, we can see that
with sensing information, SWOMP-SBL based channel estimation algorithm has a
significant gain in the NMSE compared to the wideband classical LS and greedy
SWOMP algorithm with fewer pilots and robust to the errors in the sensing
information. Hence, we reduce the pilot overhead from $100\%$ to $6.25\%$.

Figure 5: Uplink SRS comb structure in a OFDM symbol.

Figure 6: SNR vs NMSE of the channel estimates in the case of point
scatterers.

##  VI Conclusion

In this paper, the uplink channel estimation aided by sensing information for
mmWave MIMO systems has been studied. The proposed SWOMP-SBL algorithm, along
with the sensing information, uses fewer uplink pilots compared to
conventional state-of-the-art systems. The proposed scheme is also robust to
erroneous sensing information, including unassociated paths in the sensing
information. Simulation results have validated the superior performance using
reduced uplink pilots for the proposed SWOMP-SBL scheme compared to
conventional state-of-the-art algorithms. Finally, the CRB for the unknown
parameters is derived, and local identifiability analysis has been presented.

## References

  * [1] W. Saad, M. Bennis, and M. Chen, “A vision of 6G wireless systems: Applications, trends, technologies, and open research problems,” _IEEE Network_ , vol. 34, no. 3, pp. 134–142, 2020.
  * [2] S. Rangan, T. S. Rappaport, and E. Erkip, “Millimeter-wave cellular wireless networks: Potentials and challenges,” _Proceedings of the IEEE_ , vol. 102, no. 3, pp. 366–385, 2014.
  * [3] Akdeniz _et al._ , “Millimeter wave channel modeling and cellular capacity evaluation,” _IEEE JSAC_ , vol. 32, no. 6, pp. 1164–1179, 2014.
  * [4] P. Schniter and A. Sayeed, “Channel estimation and precoder design for millimeter-wave communications: The sparse way,” in _IEEE Asilomar Conference on Signals, Systems and Computers_ , 2014.
  * [5] A. Alkhateeb, O. El Ayach, G. Leus, and R. W. Heath, “Channel estimation and hybrid precoding for millimeter wave cellular systems,” _IEEE JSTSP_ , vol. 8, no. 5, pp. 831–846, 2014.
  * [6] J. Lee _et al._ , “Channel estimation via orthogonal matching pursuit for hybrid MIMO systems in millimeter wave communications,” _IEEE Trans on Comm_ , vol. 64, no. 6, pp. 2370–2386, 2016.
  * [7] K. Venugopal, A. Alkhateeb, N. González Prelcic, and R. W. Heath, “Channel estimation for hybrid architecture-based wideband millimeter wave systems,” _IEEE JSAC_ , vol. 35, no. 9, pp. 1996–2009, 2017.
  * [8] H. Tang, J. Wang, and L. He, “Off-grid sparse Bayesian learning-based channel estimation for mmwave massive MIMO uplink,” _IEEE Wireless Communications Letters_ , vol. 8, no. 1, pp. 45–48, 2019.
  * [9] A. Mishra, A. Rajoriya, A. K. Jagannatham, and G. Ascheid, “Sparse Bayesian learning-based channel estimation in millimeter wave hybrid MIMO systems,” in _IEEE SPAWC_ , 2017.
  * [10] D. Vasisht, S. Kumar, H. Rahul, and D. Katabi, “Eliminating channel feedback in next-generation cellular networks,” in _Proceedings of the ACM SIGCOMM Conference_ , 2016.
  * [11] A. Ali, N. González-Prelcic, and R. W. Heath, “Millimeter wave beam-selection using out-of-band spatial information,” _IEEE Trans on Comm_ , vol. 17, no. 2, pp. 1038–1052, 2018.
  * [12] A. Ali, N. González-Prelcic, and A. Ghosh, “Millimeter wave V2I beam-training using base-station mounted radar,” in _IEEE Radar Conference (RadarConf)_ , 2019.
  * [13] F. Liu, W. Yuan, C. Masouros, and J. Yuan, “Radar-assisted predictive beamforming for vehicular links: Communication served by sensing,” _IEEE Trans on Comm_ , vol. 19, no. 11, pp. 7704–7719, 2020.
  * [14] Y. Wang, M. Narasimha _et al._ , “Mmwave beam prediction with situational awareness: A machine learning approach,” in _IEEE SPAWC_ , 2018.
  * [15] A. Klautau, N. González-Prelcic, and R. W. Heath, “LIDAR data for deep learning-based mmwave beam-selection,” _IEEE Wireless Communications Letters_ , vol. 8, no. 3, pp. 909–912, 2019.
  * [16] C. Chaccour, W. Saad _et al._ , “Joint sensing and communication for situational awareness in wireless thz systems,” in _IEEE ICC_ , 2022.
  * [17] S. Jiang and A. Alkhateeb, “Sensing aided OTFS channel estimation for massive mimo systems,” _arXiv preprint arXiv:2209.11321_ , 2022.
  * [18] S. A. Hassani, V. Lampu, K. Parashar, L. Anttila, A. Bourdoux, B. v. Liempd, M. Valkama _et al._ , “In-band full-duplex radar-communication system,” _IEEE Systems Journal_ , vol. 15, no. 1, pp. 1086–1097, 2021.
  * [19] S. Rao, “Introduction to mmwave sensing: FMCW radars,” _Texas Instruments (TI) mmWave Training Series_ , pp. 1–11, 2017.
  * [20] J. A. Zhang, F. Liu, C. Masouros, R. W. Heath _et al._ , “An overview of signal processing techniques for joint communication and radar sensing,” _IEEE JSTSP_ , vol. 15, no. 6, pp. 1295–1315, 2021.
  * [21] J. Rodríguez-Fernández _et al._ , “Frequency-domain compressive channel estimation for frequency-selective hybrid millimeter wave MIMO systems,” _IEEE Trans on Comm_ , vol. 17, no. 5, pp. 2946–2960, 2018.
  * [22] D. Wipf and B. Rao, “Sparse Bayesian learning for basis selection,” _IEEE Transactions on Signal Processing_ , vol. 52, no. 8, pp. 2153–2164, 2004.
  * [23] Z. Zhang and B. D. Rao, “Sparse signal recovery with temporally correlated source vectors using sparse Bayesian learning,” _IEEE JSTSP_ , vol. 5, no. 5, pp. 912–926, 2011.
  * [24] P. Pal and P. P. Vaidyanathan, “Parameter identifiability in sparse bayesian learning,” in _IEEE ICASSP_ , 2014.
  * [25] M. Boizard, R. Boyer, G. Favier, J. E. Cohen, and P. Comon, “Performance estimation for tensor CP decomposition with structured factors,” in _IEEE ICASSP_ , 2015.
  * [26] “5G-NR; Physical Channels and Modulation,” _3GPP TS 38.211 version 15.2.0 Release 15_.