# Unbiased Cascade Bandits: Mitigating Exposure Bias in Online Learning to
Rank Recommendation

Masoud Mansoury  University of AmsterdamAmsterdamthe Netherlands
<EMAIL_ADDRESS>,  Himan Abdollahpouri  Northwestern UniversityEvanstonUSA
<EMAIL_ADDRESS>,  Bamshad Mobasher  DePaul
UniversityChicagoUSA<EMAIL_ADDRESS>,  Mykola Pechenizkiy  Eindhoven
University of TechnologyEindhoventhe Netherlands<EMAIL_ADDRESS>,  Robin
Burke  University of Colorado BoulderBoulderUSA<EMAIL_ADDRESS>and
Milad Sabouri  DePaul UniversityChicagoUSA<EMAIL_ADDRESS>
(2021)

###### Abstract.

Exposure bias is a well-known issue in recommender systems where items and
suppliers are not equally represented in the recommendation results. This is
especially problematic when bias is amplified over time as a few popular items
are repeatedly over-represented in recommendation lists. This phenomenon can
be viewed as a recommendation feedback loop: the system repeatedly recommends
certain items at different time points and interactions of users with those
items will amplify bias towards those items over time. This issue has been
extensively studied in the literature on model-based or neighborhood-based
recommendation algorithms, but less work has been done on online
recommendation models such as those based on multi-armed Bandit algorithms. In
this paper, we study exposure bias in a class of well-known bandit algorithms
known as Linear Cascade Bandits. We analyze these algorithms on their ability
to handle exposure bias and provide a fair representation for items and
suppliers in the recommendation results. Our analysis reveals that these
algorithms fail to treat items and suppliers fairly and do not sufficiently
explore the item space for each user. To mitigate this bias, we propose a
discounting factor and incorporate it into these algorithms that controls the
exposure of items at each time step. To show the effectiveness of the proposed
discounting factor on mitigating exposure bias, we perform experiments on two
datasets using three cascading bandit algorithms and our experimental results
show that the proposed method improves the exposure fairness for items and
suppliers.

recommender system, cascading bandit, exposure bias, feedback loop

††copyright: acmcopyright††journalyear: 2021††doi:
10.1145/1122445.1122456††booktitle: ††price: 15.00††isbn:
978-1-4503-XXXX-X/18/06††ccs: Information systems Users and interactive
retrieval††ccs: Information systems Recommender systems

##  1\. Introduction

Exposure bias in recommender systems refers to the fact that some items and
suppliers are over-represented in recommendation results, while other items
and suppliers are not adequately represented (Abdollahpouri and Mansoury,
2020; Chen et al., 2020; Singh and Joachims, 2018). Due to the interactive
nature of recommendation systems, this bias can be amplified over time as
users interact with the recommended items at each time and their interactions
would be used as input for recommendation algorithm in the subsequent times
(Mansoury et al., 2020b). In this way, recommendation system and users form a
feedback loop (Sinha et al., 2016; Jiang et al., 2019). Various research
efforts have studied the exposure bias in offline recommendation algorithms
(Sun et al., 2019; Mansoury et al., 2020b, 2021), but less work has been done
on interactive recommendation algorithms such as online learning to rank
algorithms where the system iteratively learns from users’ interaction
feedback by balancing exploration versus exploitation (Barraza-Urbina, 2017).

Multi-armed bandit (MAB) algorithms are often used to systematically balance
exploitation and exploration. Simply put, given $L$ arms as the items in the
system, the learning agent pulls $K$ arms at a time to form the recommendation
list and tries to maximize the reward based on the utility associated with
items selected by the user. There are different MAB algorithms for managing
the exploration and exploitation trade-off (Sutton and Barto, 2018). For
example, $\epsilon$-Greedy MAB algorithms always exploit the best items that
have been explored so far with the probability of $1-\epsilon$, and explores
other items with probability of $\epsilon$. Upper Confidence Bound (UCB)
algorithms approach the problem differently by making their selections based
on how uncertain the algorithm is about a given selection.

Recently, a variant of MAB algorithms known as Cascading Bandits (Kveton et
al., 2015a, b; Li et al., 2016; Li et al., 2020; Zong et al., 2016;
Hiranandani et al., 2020) have been proposed that employ UCB for their
operations and show superior performance in comparison to other MAB
algorithms. We focus our analysis on this class of MAB algorithms. Besides
their strength in increasing the click-through rate, these algorithms are also
able to address position bias (Craswell et al., 2008; Chuklin et al., 2015)
and improve the topical diversity of the recommendations. We specifically
focus on three cascading bandit algorithms: CascadeLSB (Hiranandani et al.,
2020), CascadeLinUCB (Zong et al., 2016), and CascadeHybrid (Li et al., 2020).

Our goal in this work is to investigate the degree to which these cascading
bandit algorithms are vulnerable to exposure bias. We investigate the extent
to which these algorithms successfully explore the full list of available
items and how fairly they recommend different items or suppliers. In other
words, we determine how successful they are in terms of 1) giving enough
choice to different users over time (user-level exploration) and 2) giving a
fair exposure to different items or suppliers over the course of interactions
with the users for a given period of time. We observe that these algorithms,
despite their high emphasize on exploration, still fail to fully explore the
available items and have high exposure bias both for items and suppliers. To
address these issues, we propose a simple approach by discounting the items
that have been recommended too frequently in the past by the algorithm.
Experimental results using two datasets show that our approach can
significantly improve the exposure and fairness of the recommendations while
keeping the relevance of the recommendations as high as possible.

##  2\. Background

In this section, we provide some background on the notion of exposure bias
(Singh and Joachims, 2018) and summarize the key aspects of cascading bandit
algorithms (Zong et al., 2016; Hiranandani et al., 2020; Li et al., 2020).

###  2.1. Exposure bias

The concept of exposure bias has been studied by many researchers and
different approaches are proposed to address this issue (Singh and Joachims,
2018; Geyik et al., 2019; Mehrotra et al., 2018; Mansoury et al., 2020a;
Abdollahpouri et al., 2020). Singh and Joachims in (Singh and Joachims, 2018)
discussed how exposure bias can lead to unfair treatment of different groups
of users in ranking systems. They proposed a general framework for addressing
the exposure bias when maximizing the utility for users in generating the
ranked results under various fairness constraints.

In domains like job recommendation where the task is recommending people (job
seekers or qualified candidates), besides giving a fair exposure to each item
(applicants), the solutions need to also take into account the fair exposure
for people belonging to different sensitive groups (e.g. male versus female).
In this line of research, Zehlike et al. (Zehlike et al., 2017) proposed a re-
ranking algorithm to improve the ranked group fairness in recommendations.
Their approach optimizes to achieve a predefined exposure for the minority
group as much as possible, while maintaining the accuracy of the system. In
another work, Geyik et al. in (Geyik et al., 2019) explored the exposure bias
in LinkedIn Talent Search where the distribution of applicants belong to
different groups of sensitive attributes in recommendation lists do not follow
the distribution of applicants’ group in the initial search results. They
showed that applicants belonging to the protected group are often under-
recommended. To address this issue, the authors proposed an algorithm to
achieve the desired distribution of applicants’ group with respect to
sensitive attributes in top-n recommendation results.

Some research efforts have tried to mitigate exposure bias by improving
aggregate diversity (Adomavicius and Kwon, 2011a, b; Mansoury et al., 2020a)
seeking to increase the number of unique items in the recommendation lists.
Antikacioglu and Ravi in (Antikacioglu and Ravi, 2017) proposed the idea of
aggregate diversity maximization using a maximum flow approach. They showed
that the minimum-cost network flow method can be efficiently used for finding
recommendation subgraphs that optimizes the aggregate diversity. Mansoury et
al. in (Mansoury et al., 2020a) proposed a graph-based algorithm that finds
high quality items that have low visibility in the recommendation lists by
iteratively solving the maximum flow problem on the recommendation graph.

In addressing the exposure bias for suppliers, Abdollahpouri in
(Abdollahpouri, 2020) analyzed the popularity bias as the main source of
exposure bias from the perspective of multiple stakeholders and proposed the
idea of popularity calibration for mitigating this bias. Mehrotra et al.
(Mehrotra et al., 2018) investigated the trade-off between the relevance of
recommendations for users and supplier fairness, and their impacts on users’
satisfaction. To determine the supplier fairness in recommendation list,
first, suppliers are grouped into several bins based on their popularity in
rating data and then the supplier fairness of a recommendation list is
measured as how diverse the list is in terms of covering different supplier
popularity bins.

###  2.2. Cascading bandit models

In a cascading bandit (Hiranandani et al., 2020; Li et al., 2020; Zong et al.,
2016), the learning agent interacts with users by delivering the
recommendations to them and receiving feedback. In these algorithms the
Cascade model (Craswell et al., 2008; Chuklin et al., 2015) is used for
modeling user’s feedback behavior. The system presents the user with a list of
recommended items. The user examines each recommended item one by one from the
first position to the last, selects the first attractive item, and does not
examine the rest of the items. This way, the items above the selected item are
considered unattractive, the selected item is considered attractive. The rest
of the items are considered as unobserved (neither attractive nor
unattractive). Cascade model is effective in addressing well-known position
bias (Collins et al., 2018; Hofmann et al., 2014) where lower ranked items in
the recommendation list are less likely to be clicked than the higher ranked
items. By not considering the lower ranked (unobserved) items as unattractive,
the model can still give them a chance to appear in the higher positions in
the recommendations lists later interaction rounds.

The cascading bandit algorithms compute the probability of the target user
liking a target item called attraction probability using known item features
and unknown user preferences in each iteration $t$. Since user preference is
unknown and needs to be learned by interacting with the user, the attraction
probability is estimated by solving a ridge regression problem over past
observations on item features as independent variables and their attraction
probabilities as dependent variable in $t-1$ time steps. Due to the
uncertainty in estimating user preference toward items, cascading bandit
algorithms employ UCB approach to model this uncertainty by computing an upper
bound for the expected weight of each item. Given weights computed for each
item, $K$ items with the largest estimated weights are returned to form the
recommendation list $\mathcal{A}_{t}$. In the next step, given recommendation
list $\mathcal{A}_{t}$, the agent receives feedback according to the cascade
model and finally, for each examined item, the agent updates the model
parameters.

The reward of the agent at each iteration is computed by
$f(\mathcal{A},w)=1-\prod_{k=1}^{K}(1-w(a_{k}))$ which is 1 if user clicks on
an item and 0 otherwise. Then, the performance of the agent’s policy is
evaluated by its n-step regret:

(1) $R(n)=\sum_{t=1}^{n}\mathbb{E}[f(A^{*},w_{t})-f(\mathcal{A},w_{t})]$

where $A^{*}$ is the optimal list of items that maximizes the reward at any
time $t$. Cascading bandits proposed in (Zong et al., 2016; Hiranandani et
al., 2020; Li et al., 2020) generally follow the above process for their
operations and only differ in how they define the item features.

####  2.2.1. CascadeLSB

Yue and Guestrin in (Yue and Guestrin, 2011) proposed the notion of topic
coverage of an item to describe the probability that the item covers each
topic. Similarly, the topic coverage of a list is defined as the probability
of the items in the list covering each topic. Assuming one or more topics are
associated with each item and topic coverage for each item, the feature vector
for each item is defined as the gain in topic coverage by adding that item to
the recommendation list created so far. Therefore, the feature vector for each
item is dependent to the items already added to the list. If the target item
is diverse in terms of topics with the items previously added to the
recommendation list, the target item will have higher chance to be added to
the list. Hiranandani et. al. in (Hiranandani et al., 2020) adapted this
algorithm by considering Cascade model to address the position bias and
proposed CascadeLSB.

####  2.2.2. CascadeLinUCB

Wen et al. in (Wen et al., 2015) proposed this algorithm that uses item
features derived from the user-item interaction data and, unlike CascadeLSB
that only seeks to generate diversified recommendation, this algorithm aims at
recommending items that are relevant to the user’s preferences. Various ways
can be considered for deriving item features. One way proposed in (Li et al.,
2020) that we also used for our experiment is performing singular-value
decomposition (SVD) on user-item interaction data. Zong et. al. in (Zong et
al., 2016) adapted this algorithm by considering Cascade model for
interpreting user feedback behavior and proposed CascadeLinUCB.

####  2.2.3. CascadeHybrid

Li et. al. in (Li et al., 2020) proposed CascadeHybrid by combining CascadeLSB
and CascadeLinUCB for generating recommendations that are both relevant to the
users’ preferences and diverse in terms of topics.

##  3\. Bias analysis

In this section, we analyze the cascading bandits described in section 2 in
terms of their ability in representing different items in recommendation lists
over time.

###  3.1. Methodology

For the experiments, we follow the experimental setting and data pre-
processing used in (Li et al., 2020). We perform our experiments on two
publicly-available datasets: MovieLens (Harper and Konstan, 2015) and Last.fm
(Schedl, 2016).

####  3.1.1. Data pre-processing

On MovieLens dataset, we extract 1K most active users and 1K most rated items
from the user-item interaction data. Then, we convert the ratings to binary
values where rating 5 is converted to 1 and others to 0. For our analysis on
supplier-side exposure bias, we consider the movie-maker of each movie as the
supplier. Since this dataset does not originally contain information about the
movie-makers, we used the API provided by OMDB
website111http://www.omdbapi.com/ to extract the information about movie-
makers associated with different movies. Overall, there are 92,114
interactions between users and items, 18 genres associated to the items, and
513 suppliers (i.e. movie-makers) in this sample.

On Last.fm dataset, we extract 2K most active users and 2K most rated items
from the user-item interaction data. To binarize the data, for items
interacted more than 50 times by a user, we assign rating 1 and 0 to the
others. Since the data does not have genre information, we extracted genre
information using API on TheAudioDB222https://www.theaudiodb.com/. Overall,
there are 137,587 interactions between users and items, 88 genres associated
to the items, and 967 suppliers (i.e. artists).

####  3.1.2. Simulation

For simulating the interaction between learning agent and users, following the
setting in (Li et al., 2020; Hiranandani et al., 2020), we randomly divide
users profile into 50% as training set and 50% as test set. Training set is
used for computing attraction probability of each item and generating the
recommendation list to each user. Test set is used for modeling user feedback
on recommendation list and generating the optimal recommendation list for
evaluating the performance of the model. We perform the experiments for
$n=10k$ steps and recommendations are generated for 100 randomly selected
users.

####  3.1.3. Evaluation metrics

We use the following metrics for measuring the exposure bias and the
performance of recommendation results:

  * •

n-step regret: Cumulative difference between the optimal and agent’s rewards
computed by equation 1.

  * •

Item Coverage (IC): The fraction of items recommended at least once to all
users over $n$ interactions.

  * •

Supplier Coverage (SC): The fraction of suppliers recommended at least once to
all users over $n$ interactions.

  * •

Average User-level Item Coverage (UIC): Average item coverage for each user.

  * •

Gini Index (G): Uniformity measure of the frequency distribution of
recommended items. Gini Index is in $[0,1]$ where 0 indicates uniform
distribution (fair outcome) and 1 indicates skewed distribution (unfair
outcome).

(a) Last.fm

(b) MovieLens

Figure 1. The performance of cascading bandit algorithms in addressing
exposure bias for $c=1$.

###  3.2. Experimental Results

Figure 1 compares these algorithms from four different perspectives on Last.fm
(Figure 1(a)) and MovieLens (Figure 1(b)) datasets and shows how they handle
exposure bias in recommendation results.

On Last.fm, in terms of n-step regret, CascadeLinUCB achieved better
performance (lower n-step regret), while on MovieLens, CascadeLSB outperformed
other algorithms. This is mainly because CascadeLSB behaved differently on
each dataset. Item-topic data on Last.fm is much sparser than the item-topic
data on MovieLens. Since CascadeLSB only uses topic information, highly sparse
topic data can lead to poor performance. Also, the performance of
CascadeHybrid is between CascadeLinUCB and CascadeLSB which is expected as it
is a combination of those algorithms.

Cumulative item coverage simply shows how each algorithm explores item space
over time by recommending new items to users. A flat line in the plot
indicates no new item was added to the recommendation lists over time. On both
datasets, we can see that all algorithms explore the item space more in the
early iterations and then they focus on exploitation in the later iterations.
CascadeLSB stops exploring the item space after few iterations which leads to
not covering sufficient items in the recommendation lists. Also, the plot
shows that even running the algorithm for more iterations does not improve the
item coverage. CascadeLinUCB is performing well by covering majority of the
items in the recommendation lists and the ascending slope of the plot suggests
that it may further improve item coverage if we run the algorithm for a longer
time period. Although CascadeLinUCB showed performing well so far, our next
analysis reveals its weaknesses.

Cumulative user-level item coverage shows the item coverage for each user and
how many new items are exposed to each user at each iteration. Analogous to
what we observed for cumulative item coverage, CascadeLinUCB outperformed
other algorithms, CascadeLSB is the weakest by recommending only a small
fraction of the items, and CascadeHybrid is performing in between. An
interesting pattern in these results is that even though CascadeLinUCB
performs well in terms of item coverage by covering more than 80% of the items
on both datasets in the whole recommendation lists for all users, its user-
level item coverage is very low by covering only less than 18% of the items on
Last.fm dataset and less than 22% of the items on MovieLens dataset. This
shows that these algorithms are unable to explore sufficiently the item space
for an individual user and clearly they recommend the same items repeatedly to
that user over time. This issue is more severe for CascadeLSB on both datasets
and CascaseHybrid on MovieLens which the algorithms even stop exploring the
item space and recommending new items to each user.

Table 1. Comparison of original and unbiased cascading bandit algorithms in terms of item-side and supplier-side fairness for $c=1$. The bolded entries show the best values and the underlined entries show the statistically significant change with $p<0.01$. algorithms | Last.fm |  | MovieLens
---|---|---|---
$IC$ | $SC$ | $UIC$ | $G$ |  | $IC$ | $SC$ | $UIC$ | $G$
CascadeLSB | 4.6% | 9.1% | 4.5% | 0.964 |  | 4.1% | 5.7% | 2.8% | 0.987
CascadeLinUCB | 90.4% | 96.3% | 17.7% | 0.759 |  | 94.4% | 96.5% | 21.3% | 0.639
CascadeHybrid | 30.9% | 44.7% | 9.7% | 0.937 |  | 54.2% | 65.7% | 14.1% | 0.954
UnbiasedCascadeLSB | 5.7% | 10.5% | 5% | 0.962 |  | 5.2% | 7.4% | 3.7% | 0.985
UnbiasedCascadeLinUCB | 96.7% | 99.2% | 35.4% | 0.630 |  | 97.5% | 98.6% | 35% | 0.550
UnbiasedCascadeHybrid | 36.2% | 49.3% | 11% | 0.929 |  | 60.8% | 72.1% | 18.1% | 0.942

Finally, exposure (the rightmost plot) shows the distribution of the
recommendation frequency for different items (number of times each item
appeared in the recommendation lists). Since the distribution for all
algorithms are extremely long-tailed and their plots overlap, for the
presentation purposes, we performed log-transformation on the values. These
results also confirm that all three algorithms extremely suffer from exposure
bias by not fairly representing the items in the recommendation lists. The
long-tail distribution of the recommendation frequency for the items shows
that few items frequently appeared in the recommendation lists (over-
recommendation), while majority of the items rarely appeared in the
recommendation lists (under-recommendation).

Overall, the results in Figure 1 reveal that the cascading bandit algorithms
described in section 2 suffer from exposure bias where items are not equally
represented in the recommendation lists. In the following sections, we show
that how this unfair treatment of items can lead to supplier-side unfairness.
Therefore, a solution is needed to address the issue of exposure bias in
cascading bandits. In the next section, we propose our solution for tackling
exposure bias for items and suppliers, and we show how it can improve exposure
fairness.

##  4\. Mitigating Bias in Cascading Bandits

In this section we integrate our proposed discount factor into the cascading
bandit algorithms and empirically examine the degree to which the extended
algorithms can mitigate exposure bias.

###  4.1. Cascading Bandits with Exposure Discount

The utility function in bandit algorithms described in section 2 are in the
following form:

(2) $utility_{t}(i)=AttractionProb_{t}(i)+c\times UpperBound_{t}(i)$

where $c$ controls the degree of exploration. Higher value for $c$ allows the
algorithm to explore more which, as a result, will degrade the performance of
the algorithm in terms of n-step regret. This utility function does not
differentiate between frequently and rarely exposed items and assigns the same
degree of exploration to all items. To better control the degree of
exploration for items with different exposure level, we introduce a
discounting factor that computes the degree of exploration for each item
according to its exposure until time $t$. Thus, the utility function would be
as follows:

(3) $utility_{t}(i)=AttractionProb_{t}(i)+c\times(1-\frac{N_{i}}{t})\times
UpperBound_{t}(i)$

where $N_{i}$ is the number of times item $i$ is recommended in the past $t-1$
time steps for a particular user to avoid unnecessary repetitions. For items
that are already highly exposed, this discounting factor decreases the degree
of the exploration, while for rarely exposed items, it increases the degree of
exploration to give them more chance to be recommended. For the rest of the
paper, for each algorithm that uses equation 3 for computing the utility, we
name it the unbiased version of that algorithm: UnbiasedCascadeLSB,
UnbiasedCascadeLSB, and UnbiasedCascadeHybrid.

###  4.2. Experimental results

In this section, we evaluate our unbiased cascading bandits in terms of their
ability in addressing item-side and supplier-side exposure fairness and
compare them with the original cascading bandits described in section 2.

####  4.2.1. The Impact on Item and Supplier Fairness

Table 1 presents the experimental results for original and unbiased cascading
bandit algorithms for $c=1$ as it resulted in a better performance in terms of
n-step regret and exposure fairness on original cascading bandits. The results
show that on both datasets and across all metrics, our unbiased cascading
bandits significantly outperform the original ones.

One important consideration for analyzing the fairness of an algorithm is how
many unique items or suppliers are represented in the recommendation lists. In
particular, in bandit algorithms where the algorithm is running for a long
period of time, it is expected that more items or suppliers to have a chance
to appear in the recommendation lists. This is measured by $IC$ and $SC$ in
Table 1.The proposed unbiased algorithms significantly improved the coverage
for items and suppliers with respect to these two metrics as compared to the
original algorithms. For instance, UnbiasedCascadeHybrid yielded 36.2% $IC$
and 49.3% $SC$ on Last.fm compared to 30.9% and 44.7% for original algorithms,
respectively, and 60.8% $IC$ and 72.1% $SC$ on MovieLens compared to 54.5% and
65.7% for original algorithms, respectively.

Another important consideration is the coverage of items for each user. Low
item coverage for a user means that few distinct items are repeatedly
recommended to that user in different time steps. This will have negative
impacts on users experience as users are not exposed to new items and the
algorithm may not be able to properly learn various preferences. This is
evaluated by $UIC$ in Table 1. As shown, the proposed unbiased cascading
bandits achieved fairer results in terms of $UIC$ compared to the original
algorithms. For instance, UnbiasedCascadeLinUCB yielded 35.4% and 35% $UIC$ on
Last.fm and MovieLens datasets compared to 17.7% and 21.3% $UIC$ for
CascadeLinUCB, respectively.

Finally, fair distribution of recommended items is another important factor
that shows how the system equitably treats items when generating
recommendation lists for users. This is shown by $G$ in Table 1. Again, the
proposed unbiased bandits outperformed the original algorithms in terms of
$G$. For instance, UnbiasedCascadeLinUCB outperformed CascadeLinUCB by
achieving $G$ of 0.630 compared to 0.759 on Last.fm and 0.550 compared to
0.639 on MovieLens.

(a) Last.fm

(b) MovieLens

Figure 2. n-step regret for the original (dashed) and unbiased (solid)
algorithms for different values of $c$. Lower regret indicates that more
clicks are received by the algorithm during the online learning. The numbers
in the tuple show the item coverage for the corresponding $c$ values. The
first and second numbers are the item coverage for original and unbiased
algorithm, respectively.

####  4.2.2. The Effect of Varying Exploration Coefficient

Coefficient $c$ in the utility function controls the degree of exploration in
the cascading bandits. This means that we should be able to explore more by
increasing $c$. Thus, an important question is: can we tune coefficient $c$ to
achieve the highest possible exposure fairness for items and suppliers without
the proposed discounting factor in equation 3? In this section, we empirically
answer this question by performing experiments with different values of $c$
and the experimental results show that tuning $c$ does not necessarily improve
the exposure fairness for items and suppliers.

Figure 2 shows the performance of original and unbiased cascading bandit
algorithms for $c\in\\{0.5,1,2,5\\}$ in terms of n-step regret. The numbers
shown as a tuple in each plot show the item coverage ($IC$) where the first
and second numbers indicate the item coverage of original and unbiased
algorithm, respectively. Looking at the plots for original algorithms (dashed
line) reveals that when we increase $c$, the n-step regret increases over time
(lower performance). However, higher $c$ (higher weight for exploration) does
not necessarily improve $IC$. The same pattern is also observed for other
metrics. This shows that even tuning $c$ may not necessarily increase the
exposure of items (and suppliers).

Comparing the performance of original and unbiased bandits in Figure 2 shows
that for $c=0.5$ (red line), both original and unbiased bandits achieved
almost the same n-step regret (even better performance by
UnbiasedCascadeLinUCB on Last.fm), but the item coverage achieved by our
unbiased bandit is significantly higher than the original one. Also, the same
pattern can be observed for $c=1$ in most cases. For $c\in\\{2,5\\}$, although
the unbiased cascading bandits achieved higher n-step regret than original
bandit, the performance for both versions is not practical as both n-step
regret and item coverage are worse than the ones achieved from
$c\in\\{0.5,1\\}$. In practice, the hyperparameters are tuned to achieve the
best performance. In our experiments, $c\in\\{0.5,1\\}$ yielded the best
performance in terms of both n-step regret and item coverage in all cases,
therefore, we concentrate our analysis on the results by $c\in\\{0.5,1\\}$.

On Last.fm dataset in Figure 2(a), the proposed unbiased cascading bandits
resulted in the highest item coverage with insignificant change in n-step
regret as 6.3% ($c=0.5$) for UnbiasedCascadeLSB, 97% ($c=1$) for
UnbiasedCascadeLinUCB, and 53% ($c=0.5$) for UnbiasedCascadeHybrid. On
MovieLens, the same pattern is observed with the highest item coverage as 5.5%
($c=0.5$), 98% ($c=1$), and 68% ($c=0.5$), respectively. Also, another
interesting pattern is that for all $c$ values on both datasets, the proposed
unbiased cascading bandits yielded higher item coverage than the original
ones, except for CascadeLinUCB for $c\in\\{2,5\\}$. All these results show
that the proposed discounting factor is effective in addressing exposure bias
and is able to significantly improve the exposure fairness of items and
suppliers.

##  5\. Conclusion and Future Work

In this paper, we studied exposure bias in three well-known cascading bandit
algorithms. Our results showed that these algorithms do not fairly represent
items and suppliers in the recommendation lists over time. To address this
issue, we introduced a discounting factor and incorporated it into the utility
function of those algorithms. This discounting factor is able to adjust the
degree of exploration for different items based on their exposure in previous
time steps. Our experiments on two datasets showed that the proposed
discounting factor significantly improved the exposure of items and suppliers
with negligible loss on reward.

In future work, following the idea proposed in (Li et al., 2016), we plan to
investigate the effectiveness of discounting the feature vector of highly
exposed items when updating the model parameters at each iteration, instead of
incorporating the discounting factor into the utility function, for mitigating
the exposure bias. Another interesting future work is taking into account the
position of each item in the proposed discounting factor. This means that
while trying to balance the exposure for different items, we also need to
provide equal opportunity for different items to be shown on the top of the
list. This is in particular important in the domains like job recommendation
where people belonging to different sensitive groups must have chance to
appear on top of the lists (Zehlike et al., 2017; Geyik et al., 2019).

We also plan to investigate the relationship between the popularity of the
items in the input data and their exposure in the recommendation lists in
bandit algorithms. This helps to better understand the factors leading to
unfairness in these algorithms. Finally, as another future work, we are
interested in reproducing the experiments conducted in this paper on other
variants of MAB algorithms and analyzing those algorithms in handling exposure
bias.

## References

  * (1)
  * Abdollahpouri (2020) Himan Abdollahpouri. 2020\.  _Popularity Bias in Recommendation: A Multi-stakeholder Perspective_.  Ph.D. Dissertation. University of Colorado Boulder, https://arxiv.org/pdf/2008.08551.pdf.
  * Abdollahpouri and Mansoury (2020) Himan Abdollahpouri and Masoud Mansoury. 2020.  Multi-sided exposure bias in recommendation.  _ACM KDD Workshop on Industrial Recommendation Systems_ (2020).
  * Abdollahpouri et al. (2020) Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. 2020.  The connection between popularity bias, calibration, and fairness in recommendation. In _Fourteenth ACM Conference on Recommender Systems_. 726–731.
  * Adomavicius and Kwon (2011a) Gediminas Adomavicius and YoungOk Kwon. 2011a.  Improving aggregate recommendation diversity using ranking-based techniques.  _IEEE Transactions on Knowledge and Data Engineering_ 24, 5 (2011), 896–911.
  * Adomavicius and Kwon (2011b) Gediminas Adomavicius and YoungOk Kwon. 2011b.  Maximizing aggregate recommendation diversity: A graph-theoretic approach. In _Proc. of the 1st International Workshop on Novelty and Diversity in Recommender Systems (DiveRS 2011)_. Citeseer, 3–10.
  * Antikacioglu and Ravi (2017) Arda Antikacioglu and R. Ravi. 2017.  Post processing recommender systems for diversity. In _Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining_. 707–716.
  * Barraza-Urbina (2017) Andrea Barraza-Urbina. 2017\.  The exploration-exploitation trade-off in interactive recommender systems. In _Proceedings of the Eleventh ACM Conference on Recommender Systems_. 431–435.
  * Chen et al. (2020) Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020\.  Bias and Debias in Recommender System: A Survey and Future Directions.  _arXiv preprint arXiv:2010.03240_ (2020).
  * Chuklin et al. (2015) Aleksandr Chuklin, Ilya Markov, and Maarten de Rijke. 2015\.  Click models for web search.  _Synthesis lectures on information concepts, retrieval, and services_ 7, 3 (2015), 1–115.
  * Collins et al. (2018) Andrew Collins, Dominika Tkaczyk, Akiko Aizawa, and Joeran Beel. 2018.  Position bias in recommender systems for digital libraries. In _International Conference on Information_. Springer, 335–344.
  * Craswell et al. (2008) Nick Craswell, Onno Zoeter, Michael Taylor, and Bill Ramsey. 2008.  An experimental comparison of click position-bias models. In _Proceedings of the 2008 international conference on web search and data mining_. 87–94.
  * Geyik et al. (2019) Sahin Cem Geyik, Stuart Ambler, and Krishnaram Kenthapadi. 2019\.  Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In _Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2221–2231.
  * Harper and Konstan (2015) F Maxwell Harper and Joseph A Konstan. 2015.  The movielens datasets: History and context.  _Acm transactions on interactive intelligent systems (tiis)_ 5, 4 (2015), 1–19.
  * Hiranandani et al. (2020) Gaurush Hiranandani, Harvineet Singh, Prakhar Gupta, Iftikhar Ahamath Burhanuddin, Zheng Wen, and Branislav Kveton. 2020.  Cascading linear submodular bandits: Accounting for position bias and diversity in online learning to rank. In _Uncertainty in Artificial Intelligence_. PMLR, 722–732.
  * Hofmann et al. (2014) Katja Hofmann, Anne Schuth, Alejandro Bellogin, and Maarten De Rijke. 2014.  Effects of position bias on click-based recommender evaluation. In _European Conference on Information Retrieval_. Springer, 624–630.
  * Jiang et al. (2019) Ray Jiang, Silvia Chiappa, Tor Lattimore, András György, and Pushmeet Kohli. 2019\.  Degenerate feedback loops in recommender systems. In _Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society_. 383–390.
  * Kveton et al. (2015a) Branislav Kveton, Csaba Szepesvari, Zheng Wen, and Azin Ashkan. 2015a.  Cascading bandits: Learning to rank in the cascade model. In _International Conference on Machine Learning_. PMLR, 767–776.
  * Kveton et al. (2015b) Branislav Kveton, Zheng Wen, Azin Ashkan, and Csaba Szepesvari. 2015b.  Combinatorial cascading bandits.  _arXiv preprint arXiv:1507.04208_ (2015).
  * Li et al. (2020) Chang Li, Haoyun Feng, and Maarten de Rijke. 2020.  Cascading Hybrid Bandits: Online Learning to Rank for Relevance and Diversity. In _Fourteenth ACM Conference on Recommender Systems_. 33–42.
  * Li et al. (2016) Shuai Li, Baoxiang Wang, Shengyu Zhang, and Wei Chen. 2016\.  Contextual combinatorial cascading bandits. In _International conference on machine learning_. PMLR, 1245–1253.
  * Mansoury et al. (2020a) Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020a.  FairMatch: A Graph-based Approach for Improving Aggregate Diversity in Recommender Systems. In _Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization_. 154–162.
  * Mansoury et al. (2020b) Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020b.  Feedback loop and bias amplification in recommender systems. In _Proceedings of the 29th ACM International Conference on Information & Knowledge Management_. 2145–2148.
  * Mansoury et al. (2021) Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2021\.  A Graph-based Approach for Mitigating Multi-sided Exposure Bias in Recommender Systems.  _arXiv preprint arXiv:2107.03415_ (2021).
  * Mehrotra et al. (2018) Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and Fernando Diaz. 2018\.  Towards a fair marketplace: Counterfactual evaluation of the trade-off between relevance, fairness & satisfaction in recommendation systems. In _Proceedings of the 27th acm international conference on information and knowledge management_. 2243–2251.
  * Schedl (2016) Markus Schedl. 2016\.  The lfm-1b dataset for music retrieval and recommendation. In _Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval_. 103–110.
  * Singh and Joachims (2018) Ashudeep Singh and Thorsten Joachims. 2018.  Fairness of exposure in rankings. In _Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2219–2228.
  * Sinha et al. (2016) Ayan Sinha, David F Gleich, and Karthik Ramani. 2016\.  Deconvolving feedback loops in recommender systems.  _Advances in neural information processing systems_ 29 (2016), 3243–3251.
  * Sun et al. (2019) Wenlong Sun, Sami Khenissi, Olfa Nasraoui, and Patrick Shafto. 2019.  Debiasing the human-recommender system feedback loop in collaborative filtering. In _Companion Proceedings of The 2019 World Wide Web Conference_. 645–651.
  * Sutton and Barto (2018) Richard S Sutton and Andrew G Barto. 2018.  _Reinforcement learning: An introduction_.  MIT press.
  * Wen et al. (2015) Zheng Wen, Branislav Kveton, and Azin Ashkan. 2015\.  Efficient learning in large-scale combinatorial semi-bandits. In _International Conference on Machine Learning_. PMLR, 1113–1122.
  * Yue and Guestrin (2011) Yisong Yue and Carlos Guestrin. 2011.  Linear submodular bandits and their application to diversified retrieval. Neural Information Processing Systems.
  * Zehlike et al. (2017) Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017.  Fa* ir: A fair top-k ranking algorithm. In _Proceedings of the 2017 ACM on Conference on Information and Knowledge Management_. 1569–1578.
  * Zong et al. (2016) Shi Zong, Hao Ni, Kenny Sung, Nan Rosemary Ke, Zheng Wen, and Branislav Kveton. 2016\.  Cascading bandits for large-scale recommendation problems. In _Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence_. 835–844.


[table]capposition=top

# Advancing O-RAN to Facilitate Intelligence in V2X

Eugenio Moro,, Francesco Linsalata, Maurizio Magarini, Umberto Spagnolini, and
Antonio Capone The authors are with the Dipartimento di Elettronica
Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy, (e-mails:
{eugenio.moro, francesco.linsalata, maurizio.magarini, umberto.spagnolini,
antonio.capone}@polimi.it).

###### Abstract

Vehicular communications at high frequencies are envisioned to be a
breakthrough application for the 6th generation (6G) cellular systems.
Traditional RANs lack the flexibility to enable sophisticated control
mechanisms that are demanded by the strict performance requirements of the
dynamic vehicular environment. In contrast, the features of Open RAN (O-RAN)
can be exploited to support advanced use cases. Indeed, the emerging paradigm
of O-RAN represents an ideal framework for the orchestration of vehicular
communication. Although the high potential stemming from their integration can
be easily seen and recognized, the effective combination of the two ecosystems
is an open issue. This article pioneers the integration of the two strategies
for seamlessly incorporating vehicle-to-everything (V2X) control within
O-RAN’s ecosystem. We propose and discuss an enabling architecture that
tightly integrates V2X and O-RAN. In the proposed solution, an O-RAN-based
control plane operates at low frequencies to achieve reliable and efficient
connectivity among autonomous vehicles at higher frequencies. The
technological feasibility of this integrated architecture is investigated. A
detailed case study is presented and analyzed to demonstrate the design of an
xApp to showcase a practical example of O-RAN solution for a specific V2X
scenario.

###### Index Terms:

O-RAN, V2X, 6G, dynamic control

##  I Introduction

The current RAN paradigm does not easily support network intelligence due to
network components - mainly Base Station (BS)\- being operated as monolithic
and inflexible black-boxes [1]. To address this limitation and bridge the gap
between real-world RAN deployments and cutting-edge network intelligence, a
consortium of vendors, operators, and research institutions have proposed
O-RAN as an architectural overhaul of RANs.

O-RAN is a disaggregated and open architecture that separates the hardware and
software components of the RAN, enabling interoperability, modularity, and
flexibility [1]. RAN Intelligent Controllers (RICs) are one of the major
innovations of O-RAN: softwarized control loops that enable data collection
and dynamic control implemented as micro-services over large-scale and
heterogeneous RAN deployments.

Specialized O-RAN-based control solutions have been successfully applied to
optimize different aspects of 5th generation (5G) cellular systems, confirming
the disruptive effects of this architecture. However, O-RAN still supports
only the most traditional of 5G deployments, where the network components are
only BSs and User Equipments (UEs). There is a case to be made for advancing
O-RAN to support emerging 6G use cases, where network intelligence plays an
even stronger role [2]. In this work, we argue that Connected and Autonoums
Vehicles (CAVs) exploiting high data-rate links represents one of these
fundamental use cases, and we propose our vision on the matter.

Vehicular communication is likely to be a key driving force in the future 6G
wireless networks, as it will enable advanced vehicular mobility paradigms
such as autonomous driving, coordinated sensing, and enhanced navigation. At
the core of vehicular communications lies V2X, a communication technology that
facilitates the interconnection among vehicles and infrastructure. V2X
realizes direct and multi-hops links among CAVs, namely vehicle-to-vehicle
(V2V) or Sidelink communications. Direct vehicular links reduce the network
infrastructure involvement, facilitating communication even in out-of-coverage
areas and considerably decreasing latency [3]. Moreover, since most of the
CAVs use cases require high data rates, V2V will make use of higher carrier
frequencies, such as millimeter wave (mmWave)\- currently standardized in 5G
as Frequency Range 2 (FR2)\- or sub-THz, with the introduction of beam-based
highly directive communication to counteract the considerable pathloss that
characterizes propagation in these bands [3, 4].

The unique challenges posed by CAV scenario, the dynamic nature of the
vehicular environment, the harsh propagation conditions at high frequencies,
as well as the hybrid nature of V2X necessitate the development of
sophisticated control mechanisms to ensure the success of this disruptive
technology. Albeit currently being limited to traditional RAN deployments
support, O-RAN represents the ideal candidate to enable management and
orchestration in the challenging scenario mentioned above. This article
focuses on integrating a next-generation O-RAN with V2X communications. It
addresses the challenges and opportunities associated with this integration
towards the ultimate goal of enhancing the performance, reliability, and
adaptability of RAN-based vehicular communications. To this end, we identify
fundamental key challenges of V2X, elaborating on how these can be
successfully addressed through O-RAN-based solutions to present some potential
research directions yet to be explored in the literature.

In addition, this article proposes a next-generation O-RAN architecture that,
dfor the first time, embodies a tight integration of V2X within the O-RAN
concepts. Through a proper extension of the O-RAN interfaces, we show how it
is possible to support the additional network components of V2X and let them
be managed by the O-RAN control loops.

Most notably, we allow the communication stack of connected vehicles to be
part of the entire O-RAN architecture. The result is a comprehensive vehicular
communication solution where O-RAN RICs acts as the orchestrator of a hybrid
network where vehicles are connected both to the RAN and among themselves.
Reliable and pervasive low frequency RANs (i.e., incumbent 5G Frequency Range
1 (FR1) deployments) are used to support a control plane where O-RAN messages
can be exchanged between vehicles and the RIC. This, in turn, will manage
high-frequency V2V links to effectively create a high-performance data plane,
namely a Vehicular Ad-hoc Network (VANET), to cater to the communication needs
of autonomous driving, infotainment, and other vehicular communication
applications.

Finally, a case study is presented, where we use the proposed architecture to
demonstrate the impact of a next-generation O-RAN in addressing a specific V2X
challenge. In particular, we design an O-RAN micro-service (i.e., an xApp) and
test it in a simulated environment to showcase the capabilities and benefits
of leveraging O-RAN in solving real-world V2X problems. Numerical results in
terms of network connectivity and control overhead are provided to demonstrate
the superior performance of the xApp-controlled V2X network compared to the
unmanaged solution. At the same time, we positively confirm the feasibility of
the proposed next-generation architecture.

(a)

(b)

Figure 1: V2X challenges that can benefit from O-RAN architecture intelligent
and programmable control: a) optimal beam selection and b) augmented network
connectivity

##  II An O-RAN approach to the V2X challenges

The O-RAN architecture features the possibility of applying centralized
control to the RAN through the so-called RICs, as exemplified in Fig. 1. These
functional components can implement arbitrary data collection and control
logic by communicating with the network infrastructure (i.e., BSs) thanks to
open and standardized interfaces. In particular, O-RAN introduced a Near Real-
time RAN Intelligent Controller (near-RT RIC), which operates on a $1\,$ms to
$1\,$s time scale and, thus, it is capable of operating under stringent V2X
latency requirements. Arbitrary data collection and control mechanisms are
then implemented through the so-called xApps, which are network microservices
that run on the primitives exposed by the near-RT RIC. Additionally, O-RAN has
also standardized the Non-Real-Time RIC (non-RT RIC), which is a centralized
control loop operating on a slower time scale, but with broader network
visibility. As such, it enables large-scale orchestration and policing
mechanisms implemented as network applications called rApps. When applied to
V2X, these two control loops can potentially unlock significant optimization
and orchestration gains with respect to the current architecture.

We now discuss a key set of fundamental V2X challenges, where O-RAN-based
solutions are expected to have a disruptive impact.

###  II-A Beam selection and management at mmWave

In the context of V2X, mmWave provides the communication capabilities required
to support most of the core concepts of Intelligent Transportation Systems
(ITS) [5]. To establish effective directional communication at mmWave, beams
have to be aligned both at the transmitter and the receiver, as shown in Fig.
1a. This critical operations are costly in terms of beam training overhead and
become even more challenging due to the relative mobility of the vehicles [6]
that requires tight beam tracking. As such, traditional beam management
mechanisms are considered inadequate and V2X-tailored solutions are required
instead. Data-driven approaches are proven to be effective in providing fast
beam alignment and tracking for vehicular communication. Locally sourced data
coming from on-board sensors can assist the vehicle in autonomously
identifying ideal beam direction candidates. However, centralized solutions
based on a fusion of reported vehicle positions and planned path, blockage
prediction, urban layout information, and past successful beam alignment show
the potential of orchestrating large-scale beam alignment to improve
performance and reduce interference [7]. While promising, such sophisticated
solutions require access to a large amount of fresh network data coming from
heterogeneous sources, which is hardly practical for the traditional RAN
architecture.

With the capability of abstracting the physical equipment idiosyncrasies and
enabling large-scale data collection, O-RAN represents an ideal enabler for
these solutions. By tapping into the wealth of up-to-date data that a near-RT
RIC can expose, an xApp could effectively host a well-informed beamforming
management function based on arbitrary algorithms, i.e., Machine Learning
(ML)-based, that operate down to the ms timescale. Concurrently, an rApp
running in the non-RT RIC can update beam management policies to fine-tune the
overall objective of the beam management solution according to specific
policies. In particular, the rApp can select a particular beam-width and
codebook size to further reduce alignment overhead (fewer larger beams) or
reduce interference and increase single-link performance (more but narrower
candidate beams). Overall, the potential of an O-RAN-based beam management has
been proven in the most general settings [8]. Nonetheless, there is still a
lack of V2X-dedicated studies on this matter where the capabilities of O-RAN
are applied to beam management and massive MIMO for vehicular communications.

###  II-B Radio resource management.

ITS are characterized by a large set of diverse services that present
extremely challenging communication requirements. Efficiently managing the
scarce radio resources in the RAN represents a critical challenge. 5G foresees
the use of network slices, which can be briefly described as bundles of
virtualized resources dedicated to providing specific connectivity services to
a subset of network users. Owing to the possibility of activating flexible and
service-tailored slices, network slicing is considered a natural enabler of
the diverse V2X use cases [9]. However, physical resources still need to be
efficiently allocated so that slices can support the communication
requirements, and slice isolation needs to be guaranteed. Static resource
partitioning is hardly viable due to the fast-changing state of the wireless
network. Dynamic slice resource allocation based on continuous monitoring of
the network parameters is to be preferred. This is all the more true for V2X
due to the aforementioned challenging environment [10, 3].

Thanks to its extensive data collection and control capabilities, O-RAN is
considered the fundamental enabler of slicing resource management [11].
Nonetheless, the problem of practically enabling the slicing for V2X in the
real world has yet to be satisfactorily addressed. In this case, an O-RAN-
based approach can fuse network measurements and external information (i.e.,
vehicle localization and planned path) to detect any potential criticality,
reallocate slice resources accordingly, and allow for more efficient spectrum
use. For instance, an xApp could monitor vehicular traffic and proactively
allocate slice resources in those cells that will soon be subject to increased
vehicular activity. If such resources are unavailable, xApps can unload the
receiving cells by triggering handovers, reducing foreign slices’ allotments,
or disconnecting some users in the extreme. Additionally, by dynamically
controlling Semi-persistent Scheduling (SPS) and Configured Grant (CG), the
xApp can guarantee resource-efficient low latency and reliable communications
both at the slice and single vehicle levels, ultimately providing isolation
for safety-critical services. Given the complexity of the problem, ML-based
approaches are likely to be required. In this case, an rApp could monitor and
fine-tune the mechanism put in place by the xApps and build or retrain
appropriate ML models, further adapting the V2X slicing management to long-
term environmental variations.

The problem of resource allocation is also relevant for direct V2V
connections. According to the 3rd Generation Partnership Project (3GPP)
standard, a central entity (i.e., a BS or a Road Side Unit (RSU)) is expected
to allocate the radio resources [10]. This mechanism is hindered by the
limited perception of the central entity with respect to each V2V link
condition and traffic requirement [12]. In this context, an xApp could gather
data about the vehicle’s position and mobility, as well as channel status and
interference profile. This information can be processed to adapt the
allocation strategies to the fast-varying vehicular environment.

###  II-C Enhanced Vehicle-to-vehicle connectivity.

Fundamental ITS services such as cooperative awareness, augmented reality, and
coordinated autonomous driving require extensive data exchange among CAVs that
are in close proximity to each other [13]. However, relying on base stations
to forward the entirety of this traffic is impractical due to inefficiency and
increased burden on the traditional RAN infrastructure. High-frequency
Sidelink communications enabled by 5G New Radio (NR) are thus inevitable for
reliable low latency and high throughput V2V links. The challenges presented
in the previous paragraphs interest V2V communications as well, making a case
for addressing them through an O-RAN approach. Additionally, the VANET created
through the activation of direct links needs to be properly managed in order
to support the ITS services [13].

Figure 2: Next-generation O-RAN architecture details.

The high mobility of the vehicles and the harsh mmWave propagation creates a
challenging twist on the traditional problems of Ad-hoc networks, which
include link selection, network graph and routing optimization, and congestion
control, as shown in Fig. 1b. By tapping into the control and monitoring
capabilities of the O-RAN architecture, an xApp could select which V2V links
to activate according to the expected channel quality and probability of
blockage. The overall link selection strategy could optimize the underlying
VANET graph to meet different objectives. For instance, different superimposed
graphs - low latency graphs prioritizing short paths or high throughput graphs
prioritizing high-quality links - can be precomputed and dynamically activated
according to the instantaneous communication needs or the policies dictated by
an rApp.

###  II-D Programmable and up-to-date V2X digital twin.

The 6G V2X communications will exploit the cooperation among CAVs to augment
environment perception and to enable the creation of a Digital Twin (DT) of
the surrounding environments [14]. To obtain an accurate real-time digital
reproduction of the physical environment, the envisioned digital twin-enabled
V2X system has to use high-definition 3D maps and combine multi-modal sensory
data from several vehicles’ onboard sensor data, as well as a detailed
description of the communication network state. The O-RAN architecture is
well-positioned to source the network information required to build high-
fidelity V2X DTs, reducing the amount of data that the network nodes should
manage and exchange. At the same time, O-RAN applications can exploit the DT
itself to run inference on the overall V2X scenario without causing
communication overhead with the infrastructure. In particular, xApps can
exploit the DT to obtain reasonably precise information on current and future
vehicle positions without directly interrogating them. Proactive and optimized
traffic forecasting capabilities from the V2X DT can also be exploited. rApps
can retrain ML models on the virtual V2X environment recreated by the DT to
ensure that the xApp data-driven approaches always employ up-to-date models.

##  III A next-generation O-RAN architecture for V2X

In the previous section, we have detailed how some relevant V2X challenges can
be successfully addressed by exploiting the network programmability enabled by
O-RAN. However, the specifications of the O-RAN are currently designed around
traditional access networks. Due to the peculiarities of the V2X environment,
some extensions to the O-RAN architecture are required such that all of these
solutions can be practically realized. We now focus on this matter, proposing
our vision of an enabling architecture where O-RAN and V2X are integrated to
unlock the aforementioned opportunities. As shown in Fig. 2 for the case of
using 5G as the Radio Access Technology (RAT), a typical O-RAN deployment
includes a non-RT RIC and a near-RT RIC embodied as software components in the
Edge Computing Cluster (ECC) [1]. A DT could also be deployed inside the same
ECC, allowing for undisturbed data exchanges with the RICs. O-RAN apps running
on top of both RICs communicate with the network infrastructure through open
interfaces: the E2 interface for the non-RT RIC and the O1 and O2 interfaces
for the near-RT RIC. In light of this, creating an O-RAN empowered V2X
requires all the V2X devices to be reachable by the O-RAN RICs through these
interfaces. BSs are normally equipped with interface terminations to enable
data collection and control, as shown in Fig. 2; thus, no modifications are
required from the communication standpoint. However, the interface definitions
will likely require extensions to support the specifics of data collection and
control required by vehicular use cases. RSUs are currently not supported by
O-RAN specifications. Nonetheless, integrating the O1, O2, and E2 terminations
in RSUs would be a relatively straightforward operation, as the communication
between the devices and the RICs in the ECC could take place by using the
already existing RSU control plane. As previously mentioned, proper extensions
to the O-RAN interface definitions will enable RSUs to be subjected to data
collection and control. On the other hand, direct communication between the
RICs and the CAVs results in being more challenging. However, allowing the NR
Sidelink stack of CAVs to be centrally controlled opens the opportunity of
addressing a key issue in V2X networks: orchestrating the V2V communication.

###  III-A O-RAN-based control plane for V2V

In our integrated architecture proposition, we envision a VANET which still
retains its decentralized nature, but it is supported by a next-generation
O-RAN to enhance its performance to the point of supporting the stringent
requirements of the safety-critical ITS services. As Fig. 2 shows in the
exemplary scenario on the left, the data plane of the V2V ad-hoc network is
represented by direct NR sidelink mmWave connections established among CAVs,
offering high throughput without occupying BS resources. For the control
plane, Below $6\,$GHz (sub-$6$GHz) links are employed, allowing for reliable
and efficient communication between vehicles and the centralized controller
through the existing RAN infrastructure. The use of sub-$6$GHz frequencies in
the control plane offers a wider coverage and better penetration capabilities,
leveraging on the ubiquitous coverage of modern cellular network deployments.
The role of this out-of-band control plane is to relay O-RAN messages between
the RICs and the interface terminations of the CAV 111In Fig. 2, only the E2
termination is included in CAVs for the sake of simplicity.. According to this
architecture, a CAV would require a 5G FR1 UE to connect to the 5G BS and an
FR2 NR Sidelink UE to connect with other vehicles. After an FR1 connection is
established with the BS, dedicated radio bearers for the O-RAN-based
management plane are established. In particular, at least one is required to
create a GTP tunnel with a User Plane Function (UPF) co-located with the RIC
in the ECC. As Fig. 2 shows in detail, the GTP tunnel can be then used to
transparently connect the near-RT RIC and the E2 termination in the CAV.
Thanks to the capability of GTP tunnels of maintaining IP endpoint
connectivity through handovers, the high mobility of CAV is not expected to
disrupt the proposed control plane. In other terms, the burden of managing the
mobility of E2 terminations is left to the 5G connection, while O-RAN
microservices obtain a reliable connection with the CAVs as they navigate
through the coverage area.

###  III-B Technological feasibility

The proposed architecture is feasible from a technological realization
standpoint, with minimal modifications required. Integrating a 5G UE and a
Sidelink UE into each vehicle allows for the establishment of both the FR1
connection with the 5G BS and the FR2 NR Sidelink connections with other
vehicles. This integration can be achieved without significant challenges, as
vehicles generally have flexible energy consumption constraints, making it
feasible to accommodate the necessary communication modules. Moreover, the
proposed architecture does not require substantial modifications to the
existing 5G stack, as it relies on standard 5G communication modes.

However, although realizable, the architecture’s effectiveness and performance
must be thoroughly analyzed. Factors such as latency and control plane
overhead (i.e., BS resource utilization) must be carefully considered to
ensure the architecture’s viability. Such analysis should be conducted on a
case-by-case basis, representing another open question in the context of O-RAN
for V2X. In the following section, we conduct a preliminary analysis of the
effectiveness and viability of addressing a challenging V2V goal through the
proposed O-RAN-enabled architecture.

##  IV O-RAN for V2V: a case study

We conduct a case study based on a typical vehicular communication scenario
where multiple CAVs traverse a busy and blind urban intersection. The scenario
is compatible with the architecture proposed in the previous section.

###  IV-A Problem definition

CAVs traversing the urban intersection in the scenario require uninterrupted
V2V connectivity to exchange safety-critical information throughout the entire
navigation area. However, due to the nature of mmWave propagation, V2V links
are often interrupted by blockages caused by buildings, urban obstacles, and
vehicles themselves. In case of Line of Sight (LoS) obstruction, RSUs and
other CAVs can act as relays to maintain connectivity between two vehicles.
However, a central coordinator is required to optimally chose the path and set
up the new connection.

Figure 3: xApp message diagram. The implementation of New path computation is
arbitrary.

###  IV-B O-RAN solution

To address the problem defined above, we propose an xApp that optimally
selects relays (namely RSUs or other CAVs) among those available to establish
a multi-hop path between two vehicles whose direct link is in a Non-Line of
Sight (NLoS) condition.

Fig. 3 shows how such an xApp can be implemented by detailing the exchange of
messages between the involved components. The exchange starts with a V2V link
failure report originating from the CAVs. This report travels through the low-
frequency control plane and reaches the xApp, which is now tasked with finding
an alternative relayed path. To do so, the xApp first needs to reconstruct the
vehicular network graph based on the quality of all the V2V links that can be
established at that moment. Such information can be obtained, for instance, by
knowing the relative position of the vehicles and their communication
characteristics to extract the channel quality according to an appropriate
propagation model [13]. Alternatively, this information can be acquired
through measures carried out by CAVs and periodically reported through the
control plane. We now assume the presence of a DT that the xApp can
interrogate to obtain an up-to-date and reliable representation of the
vehicular scenario to also demonstrate the role of this technology.

Once the data-plane graph is reconstructed, the xApp finds an alternative
route between the two vehicles according to an arbitrary logic represented by
the New path computation in Fig. 3. Each vehicle involved in the path is then
informed to establish a new V2V link through control messages from the xApp
and delivered through the low-frequency control plane.

Figure 4: Network connectivity and average number of hops versus required SNR

Finally, this approach can be generalized to allow any two vehicles to
communicate by requesting a route to the xApp when a direct link is
unavailable.

###  IV-C Effectiveness

With the goal of investigating the attainable performance of the proposed
solution, we replicated the actions of the xApp in a simulated environment.

We recreated the study scenario through a ray tracer simulator as in [13]. We
simulated the action of a sample xApp by computing the alternative route
between any pair of vehicles using a shortest path algorithm. We repeated the
experiments by deactivating links to guarantee an increasingly stringent
minimum Signal-to-Noise-Ratio (SNR). This was done to explore the capability
of an xApp to select a path based on specific communication performance
requirements.

Results are shown in Fig. 4. The baseline approach, considering only the
direct links, shows that no more than 25% of the vehicles can establish a
connection throughout the entire time window. On the other hand, it is shown
how the proposed xApp has the potential of guaranteeing full vehicular
connectivity even for high levels of minimum guaranteed SNR. Please note that
our intention here was not to propose a state-of-the-art solution but to
evaluate the impact of a simplified, centralized approach with respect to an
unmanaged V2V scenario. Fig. 4 also plots the average number of hops required
to ensure vehicular connectivity. This measure shows a trade-off between
minimum SNR\- affecting the throughput - and path length - affecting the
latency - suggesting that a more refined xApp should be capable of exploiting
it.

Overall, these results confirm how even such a simple xApp can be highly
beneficial in addressing one of the fundamental challenges of V2X.

###  IV-D Control plane overhead

Figure 5: Control plane traffic required by the V2X xApp

In the previous analysis, we replicated the action of a sample xApp to confirm
its effectiveness. We now focus on the feasibility of the proposed approach.
According to the enabling architecture proposed in Sec. III, the RICs and the
CAVs communicate through a low-frequency control plane that utilizes the
resources of the existing 5G infrastructure. Consequently, a feasible xApp
must not generate excessive traffic while operating.

As shown in Fig. 3, the proposed V2V xApp requires two control messages in
uplink and as many downlink control messages as the number of hops. Based on
this, it is possible to measure the control traffic that the xApp would
generate in the simulated scenario by counting the path computation events.
Fig. 5 reports the results averaged over the 5 minutes time window and with a
pessimistic O-RAN packet size of $1\,$Kb, showing that the downlink traffic
constitutes the largest part of the control plane overhead, as expected.
Nonetheless, the worst-case traffic of $160\,$Kbps is negligible for the 5G
RAT. This confirms the feasibility of the proposed xApp in terms of
communication overhead.

###  IV-E Control latency

Figure 6: Vehicular link outage duration statistic. $n$ represents the outage
event count.

A reactive xApp, as the one proposed, must be able to apply the control
solution fast enough to be effective. The timescale is highly dependent on the
application. For our case study, the xApp must be able to compute and
establish an alternative route well before the LoS obstruction is cleared. It
is possible to measure the control delay of the proposed xApp in the simulated
scenario. We consider negligible the communication delay between the BS and
the xApp, as it happens in the edge cluster.

Additionally, due to the extreme computational efficiency of the shortest path
algorithm, we again consider the alternative route computation negligible.
Consequently, the control latency boils down to the latency of the 5G system
supporting the control plane, which we fix to a conservative $30\,$ms. We then
compare this value with the duration of the direct link outage events, whose
statistic is reported in Fig. 6. Here we have filtered out outage events
lasting less than $1\,$s - supposed to be recovered by the beam tracking
mechanism - and more than $30\,$s - as they mostly represent outliers.
Overall, the statistics show how the outage duration is centered around values
going from $3$ to $10\,$s.

The comparison with the previously computed control delay shows the xApp’s
high feasibility and quick response to disruptions in the communication
system. In other words, the xApp promptly adapts to changing conditions and
maintains communication between vehicles when direct links are blocked. This
highlights the xApp effectiveness in mitigating link failures and minimizing
communication outages.

##  V Concluding Remarks

As the world moves towards a more connected and automated future, the need for
reliable and efficient communication between vehicles and network
infrastructure has become increasingly important. In this article, we focused
on the use of the O-RAN architecture for 5G and beyond V2X communication. We
highlighted how O-RAN has the potential to provide a more flexible, scalable,
and cost-effective solution compared to current solutions for V2X systems.
Also, we discussed integration points, proposing our envisioned architectural
solution. A first set of simulations demonstrated numerically the benefits of
a managed, controlled, and programmable V2X system compared with an unmanaged
one.

## Acknowledgment

This article was supported by the European Union under the Italian National
Recovery and Resilience Plan (NRRP) of NextGenerationEU, partnership on
“Telecommunications of the Future” (PE00000001 - program “RESTART”, Structural
Project 6GWINET).

## References

  * [1] M. Polese, L. Bonati, S. D’Oro, S. Basagni, and T. Melodia, “Understanding O-RAN: Architecture, Interfaces, Algorithms, Security, and Research Challenges,” _IEEE Communications Surveys & Tutorials_, pp. 1–1, 2023\.
  * [2] E. Moro, G. Gemmi, M. Polese, L. Maccari, A. Capone, and T. Melodia, “Toward open integrated access and backhaul with O-RAN,” in _2023 21st Mediterranean Communication and Computer Networking Conference (MedComNet) (MedComNet 2023)_ , Island of Ponza, Italy, Jun. 2023, p. 9.
  * [3] M. Noor-A-Rahim, Z. Liu, H. Lee, M. O. Khyam, J. He, D. Pesch, K. Moessner, W. Saad, and H. V. Poor, “6g for vehicle-to-everything (v2x) communications: Enabling technologies, challenges, and opportunities,” _Proceedings of the IEEE_ , vol. 110, no. 6, pp. 712–734, 2022.
  * [4] A. Molina-Galan, B. Coll-Perales, and J. Gozalvez, “C-v2x assisted mmwave v2v scheduling,” in _2019 IEEE 2nd Connected and Automated Vehicles Symposium (CAVS)_ , 2019, pp. 1–5.
  * [5] J. Choi, V. Va, N. Gonzalez-Prelcic, R. Daniels, C. R. Bhat, and R. W. Heath, “Millimeter-wave vehicular communication to support massive automotive sensing,” _IEEE Communications Magazine_ , vol. 54, no. 12, pp. 160–167, 2016.
  * [6] J. Gu, B. Salehi, D. Roy, and K. R. Chowdhury, “Multimodality in mmWave MIMO Beam Selection Using Deep Learning: Datasets and Challenges,” _IEEE Communications Magazine_ , vol. 60, no. 11, pp. 36–41, 2022.
  * [7] M. Mizmizi, F. Linsalata, M. Brambilla, F. Morandi, K. Dong, M. Magarini, M. Nicoli, M. N. Khormuji, P. Wang, R. A. Pitaval _et al._ , “Fastening the initial access in 5G NR sidelink for 6G V2X networks,” _Vehicular Communications_ , vol. 33, p. 100402, 2022.
  * [8] O.-R. Alliance, “Use cases analysis report r003-v10.00,” Tech. Rep., 2023.
  * [9] C. Campolo, A. Molinaro, A. Iera, and F. Menichella, “5g network slicing for vehicle-to-everything services,” _IEEE Wireless Communications_ , vol. 24, no. 6, pp. 38–45, 2017.
  * [10] M. H. C. Garcia, A. Molina-Galan, M. Boban, J. Gozalvez, B. Coll-Perales, T. Şahin, and A. Kousaridas, “A tutorial on 5G NR V2X communications,” _IEEE Communications Surveys & Tutorials_, vol. 23, no. 3, pp. 1972–2026, 2021.
  * [11] C.-L. I, S. Kuklinskí, and T. Chen, “A perspective of o-ran integration with mec, son, and network slicing in the 5g era,” _IEEE Network_ , vol. 34, no. 6, pp. 3–4, 2020.
  * [12] C. Shin, E. Farag, H. Ryu, M. Zhou, and Y. Kim, “Vehicle-to-everything (v2x) evolution from 4g to 5g in 3gpp: Focusing on resource allocation aspects,” _IEEE Access_ , vol. 11, pp. 18 689–18 703, 2023.
  * [13] F. Linsalata, S. Mura, M. Mizmizi, M. Magarini, P. Wang, M. N. Khormuji, A. Perotti, and U. Spagnolini, “Los-map construction for proactive relay of opportunity selection in 6G V2X systems,” _IEEE Transactions on Vehicular Technology_ , pp. 1–15, 2022.
  * [14] C. Ding and I. W.-H. Ho, “Digital-Twin-enabled City-model-aware Deep Learning for Dynamic Channel Estimation in Urban Vehicular Environments,” _IEEE Transactions on Green Communications and Networking_ , vol. 6, no. 3, pp. 1604–1612, 2022.

## Biographies

Eugenio Moro is a researcher at Politecnico di Milano. His research area is
wireless networks, with a focus on optimization, programmability and smart
propagation.
---
Francesco Linsalata is a researcher at Politecnico di Milano. His main
research interests focus on V2X communications and the next generation of
wireless networks.
---
Maurizio Magarini is an Associate Professor at Politecnico di Milano. His
research interests are in the broad area of communication and information
theory, with focus on molecular communications, massive MIMO, vehicular
communications, study of advanced waveforms for 6G, and wireless networks
using unmanned aerial vehicles and high-altitude platforms.
---
Antonio Capone (Fellow, IEEE) is currently the Dean of the School of
Industrial and Information Engineering, Politecnico di Milano (Technical
University of Milan). His main research activities include radio resource
management in wireless networks, traffic management in software defined
networks, network planning, and optimization.
---
Umberto Spagnolini is Professor of Statistical Signal Processing, Director of
Joint Lab Huawei-Politecnico di Milano and Huawei Industry Chair, scientific
coordinator of 6G Wireless Networks and Technologies, a large Eu-National
project. Recent interests are on MIMO channel estimation, cooperative and
distributed inference, vehicular systems (V2X and radar), integrated
communication and sensing.
---