Neighbor Auto-Grouping Graph Neural Networks for Handover Parameter
  Configuration in Cellular Network
The mobile communication enabled by cellular networks is the one of the main
foundations of our modern society. Optimizing the performance of cellular
networks and providing massive connectivity with improved coverage and user
experience has a considerable social and economic impact on our daily life.
This performance relies heavily on the configuration of the network parameters.
However, with the massive increase in both the size and complexity of cellular
networks, network management, especially parameter configuration, is becoming
complicated. The current practice, which relies largely on experts' prior
knowledge, is not adequate and will require lots of domain experts and high
maintenance costs. In this work, we propose a learning-based framework for
handover parameter configuration. The key challenge, in this case, is to tackle
the complicated dependencies between neighboring cells and jointly optimize the
whole network. Our framework addresses this challenge in two ways. First, we
introduce a novel approach to imitate how the network responds to different
network states and parameter values, called auto-grouping graph convolutional
network (AG-GCN). During the parameter configuration stage, instead of solving
the global optimization problem, we design a local multi-objective optimization
strategy where each cell considers several local performance metrics to balance
its own performance and its neighbors. We evaluate our proposed algorithm via a
simulator constructed using real network data. We demonstrate that the handover
parameters our model can find, achieve better average network throughput
compared to those recommended by experts as well as alternative baselines,
which can bring better network quality and stability. It has the potential to
massively reduce costs arising from human expert intervention and maintenance.
Mehrtash Mehrabi, Walid Masoudimansour, Yingxue Zhang, Jie Chuai, Zhitang Chen, Mark Coates, Jianye Hao, Yanhui Geng
Neighbor Auto-Grouping Graph Neural Networks for Handover Parameter Configuration in Cellular Network

###### Abstract

The mobile communication enabled by cellular networks is the one of the main foundations of our modern society. Optimizing the performance of cellular networks and providing massive connectivity with improved coverage and user experience has a considerable social and economic impact on our daily life. This performance relies heavily on the configuration of the network parameters. However, with the massive increase in both the size and complexity of cellular networks, network management, especially parameter configuration, is becoming complicated. The current practice, which relies largely on experts' prior knowledge, is not adequate and will require lots of domain experts and high maintenance costs. In this work, we propose a learning-based framework for handover parameter configuration. The key challenge, in this case, is to tackle the complicated dependencies between neighboring cells and jointly optimize the whole network. Our framework addresses this challenge in two ways. First, we introduce a novel approach to imitate how the network responds to different network states and parameter values, called auto-grouping graph convolutional network (AG-GCN). During the parameter configuration stage, instead of solving the global optimization problem, we design a local multi-objective optimization strategy where each cell considers several local performance metrics to balance its own performance and its neighbors. We evaluate our proposed algorithm via a simulator constructed using real network data. We demonstrate that the handover parameters our model can find, achieve better average network throughput compared to those recommended by experts as well as alternative baselines, which can bring better network quality and stability. It has the potential to massively reduce costs arising from human expert intervention and maintenance.

1 Huawei Noah's Ark Lab, \({}^{2}\) University of Alberta, \({}^{3}\) McGill University, \({}^{4}\) Tianjin University

{mehrtash.mehrabi, walid.masoudimansour, yingxue.zhang, chuaijie, chenzhitang2, haojianye, geng.yanhui}@huawei.com, mark.coates@mcgill.ca

## 1 Introduction

The rapid growth in the number of devices that need real time, high quality connection to the internet (e.g., internet of things (IoT) devices, health monitoring equipment, devices used for online education and remote working, autonomous vehicles, etc.) makes it essential to improve cellular network performance. Unsatisfactory user experience and network interruption have negative impacts in our modern society. Thus, improving the cellular network has both economic and social impact towards achieving United Nations Sustainable Development Goals (UNSDGs) [22, 13]. Moreover, it can highly contribute to enhancing infrastructure, promoting sustainable industrialization, fostering innovation, responsible consumption, enabling sustainable cities and communities, and promoting decent work and economic growth [1, 1, 16].

The performance of a cellular network relies heavily on its parameter configurations and it is becoming more crucial, as the number of mobile users continues to grow rapidly [2]. These parameters govern access control, handover, and resource management [1, 1]. One of the factors that has a significant impact on the quality of service (QoS) in such networks is the handover parameter [10]. We provide more details concerning this parameter and its effects in the supplementary materials, Sec. A.1.

Optimizing handover parameters is one of the most common approaches to guarantee minimum service delay or interruption and improve coverage and throughput [15]. However, with the massive increase in both the size and complexity of cellular networks, parameter configuration is becoming complicated. The current practice, which relies largely on experts' prior knowledge, is inadequate, requiring many domain experts and leading to high maintenance costs.

One of the key challenges in the network parameter optimization problem is the complex spatial and temporal dependencies in the cellular network. Any employed algorithm should be capable of tracking the non-stationary changes in the environment, i.e., the fluctuations of user number, network load, etc. [1]. Also, due to the diverse characteristics of cells across the network, the best parameter configuration for one cell may not be optimal for another and parameter configuration of one cell not only affects its own performance, but also affects its neighbors' [1]. Therefore, there are strong interactions between neighboring cells which become extremely complicated in heterogeneous network. Consequently, developing an algorithm that can adapt to the temporal dynamics and cell diversity in real networks is essential for parameter configuration [11].

The current cellular network deployments are highly de
pendent on human designed rules or analytical models based on domain knowledge and assumptions about the network dynamics which is far from optimal. They only consider a limited number of network states (e.g., user distribution, channel quality, etc.) and parameters, and cannot capture the complex relationships between network states, parameter configurations and network performance. Also, the assumptions of the network dynamics, based on which the rules/models are developed, are often simplified without considering the non-stationary changes in real environments, which degrades their performance. Finally, these rules/models may not be able to deal with the cell diversities in the network which makes them sub-optimal [12].

Recently, data-driven approaches based on machine learning (ML) have been extensively used for parameter configuration and network management in cellular networks [20, 14, 15, 16]. It has been shown that the multi-layer perceptron (MLP) can be considered as a universal function approximator [1]. Thus, in environments such as cellular networks where there is lack of an accurate analytical model and the network is highly dynamic, neural-network-based methods can be used to achieve high-accuracy prediction. ML models can utilize high dimensional information and approximate complex functions to fully describe the relationship between network states, parameter configurations and network performance metrics, which cannot be achieved by human experience.

In order to address the above challenges, we investigate two important questions: 1) _Modeling_: how to model the spatial and temporal dependencies of the cellular network? 2) _Decision-making_: how to choose the parameter values to jointly optimize the overall performance of interconnected and interacting cells? We first, propose a ML-based model to precisely imitate the cellular network environment and then, use it to configure the parameters.

We demonstrate that the handover parameters recommended by our model can achieve better average network throughput compared to the existing methods and our approach can massively reduce costs from human expert intervention and maintenance. It opens up the potential for high-quality internet access to geographical areas that are currently under-served by the cellular network. Besides, this framework can bring new possibilities for important applications to under-developed regions including online education, health monitoring devices by improving their real time connection [1].

Our main contributions are summarized as follows.

* We propose a novel method to model the impact from the neighbors of each cell in a distinguishable way to capture the complex spatial dependencies of the network.
* We consider the changing dynamics of the network in our reward model to better reflect the temporal dependencies.
* We introduce a multi-objective optimization strategy based on the model to consider several performance metrics and improve the overall network throughput, which has the potential for high social impact applications.

## 2 Background and Related Work

The adjustment of handover parameters helps to balance the traffic load in the network and it can dramatically affect the network throughput. During the handover process in cellular networks, in order to guarantee an acceptable service quality, a user equipment (UE) must monitor the reference signal received power (RSRP) of the serving cell (3GPP TS36.331 2016). As soon as the RSRP drops below a pre-defined threshold (called A2-threshold), the UE starts to report measurements to its serving cell and prepares for handover. Increasing the value of A2-threshold decreases the number of UEs in the serving cell in which the handover is triggered, and this spreads the serving cell's load to its neighbors, resulting in a significant change of throughput for the serving cell and its neighbors. While improving the load balance of the network, this can have adverse effects on the network performance since it forces frequent handovers which requires a considerable amount of bandwidth for measurement reporting and causes a drop in network throughput. Decreasing the value of A2-threshold, on the other hand, may cause a poor experience for edge UEs and lead to repeated connection loss due to weak signal. In attempt to solve the problem of optimization of the parameters of a wireless network, different techniques such as fuzzy systems, deep reinforcement learning (DRL), and contextual bandit have been used in the literature. (see Sec. A.2 for some details).

The use of graph convolutional networks (GCNs) [16, 17, 18] has also yielded significantly well-designed models to predict the network traffic and optimize the corresponding parameters. For example, in [15], the authors introduce a novel handover strategy based on GCNs. The handover process is modeled as a directed graph by which the user tries to predict its future signal strength. Other works such as [15] introduce novel methods of network traffic prediction combined with a greedy search or action configuration method to optimize handover parameters. However, these works fail to consider the heterogeneous aspect of the cellular networks.

Despite being effective, none of the above-mentioned methods uses the capacity of the neighbors' information to fully tailor the model to adapt to the spatial characteristics of a cellular network, where the interaction is complex and the network is heterogeneous. Also, despite the fact that these techniques consider some important measures of optimization, none of them approaches the problem at hand by considering two of the most important measures simultaneously (especially from the users' perspective): load balancing and throughput. In this article, we propose an effective and efficient framework that models the network as a heterogeneous graph where we learn an implicit interaction type for each neighboring cell. Then, it incorporates the impact of neighboring cells from each interaction group in a unique way. Moreover, in contrast to the available methods in the literature, we exploit two important measures in the network simultaneously, to configure the parameters effectively: throughput and load balancing, which are directly related to the user experience in the network.
Problem Formulation

Let us consider a network with \(N\) cells, and form \(N\) clusters each composed of one of the network cells as its center cell along with its neighboring cells. As an example, we choose the optimization of the A2-threshold to investigate the performance of our algorithm. According to the 3GPP standard (3GPP TS36.331 2016), an A2 event is triggered when the received power at user \(u\) from cell \(n\), \(P^{u,n}\), satisfies

\[P^{u,n}+H_{ys}<Thresh, \tag{1}\]

where \(H_{ys}\) is the hysteresis parameter to avoid frequent handovers and \(Thresh\) is the A2-threshold we are optimizing.

We consider an online optimization process. In real practice, network operators are often conservative and only allow a limited number of experiments. During the optimization period of \(L\) days, and the A2-threshold can be adjusted once for each cell at the beginning of each day. For day \(t\), let \(D_{t}\) be the total bits transmitted by all the cells, and \(T_{t}\) be the total transmission time. We would like to maximize the accumulated network throughput of the optimization period, i.e., \(\max\sum_{t=1}^{L}\frac{D_{t}}{T_{t}}\). Maximizing the overall network throughput by jointly optimizing the A2-threshold of all cells is difficult. The problem becomes even more complicated as the network size increases, which makes a centralized solution not scalable. The adjustment of the A2-threshold of one cell only affects its local neighborhood and thus, we convert the centralized problem into a local decision problem. That is, each cell only examines its local performance metrics and chooses its own parameter configuration value.

The adjustment of the A2-threshold affects the network throughput via two means: better resource utilization by load balancing, and improved cell throughput with less connection loss and measurement reporting. Consequently, in order to configure it, these two metrics must be considered in the local decision problem. The throughput of cell \(i\) on day \(t\) is highly dependent on its A2-threshold, formulated as \(a_{t}^{i}\), denoted as \(\alpha_{t}^{i}(a_{t}^{i})\). The load balancing factor in the \(i\)-th cluster with center cell \(i\) on day \(t\) with \(a_{t}^{i}\) is defined as the ratio of the center cell throughput to the average throughput of its neighboring cells, denoted by \(\beta_{t}^{i}(a_{t}^{i})\) and formulated as \(\beta_{t}^{i}(a_{t}^{i})=\alpha_{t}^{i}(a_{t}^{i})/\bar{\alpha}_{t}^{i}\), where \(\bar{\alpha}_{t}^{i}\) is the average throughput of the neighbors of cell \(i\) with action \(a_{t}^{i}\) and, denoting by \(\mathcal{N}_{t}(i)\) the set of all neighbors of cell \(i\) on day \(t\), it can be formulated as \(\bar{\alpha}_{t}^{i}=\frac{1}{|\mathcal{N}_{t}(i)|}\sum_{j\in\mathcal{N}_{t}(i )}\alpha_{t}^{j}(a_{t}^{j})\). The throughput ratio (rather than traffic/user ratio) is used since different cells have different capacities. This value approaches \(1\) when loads of different cells match their capacities.

Our goal is to maximize the overall network throughput by optimizing the two important network performance metrics, namely, throughput ratio \(\beta_{t}^{i}(a_{t}^{i})\) and cell throughput \(\alpha_{t}^{i}(a_{t}^{i})\) for each cell \(i\in[1,N_{t}]\), where \(N_{t}\) is the total number of cells on day \(t\), at the same time. Therefore, we propose the following optimization problem for tuning the A2-threshold for cell \(i\):

\[\operatorname*{arg\,max}_{a_{t}^{i}\in\mathcal{A}}\Big{(}-\sqrt{ \big{|}1-\beta_{t}^{i}(a_{t}^{i})\big{|}},\alpha_{t}^{i}(a_{t}^{i})\Big{)}, \tag{2}\]

where \(\mathcal{A}\) is the set of all possible values for the A2-threshold in the cellular network.

The challenge of solving the above problem lies in several folds. _First_, since the network performance function is complex, dynamic and unknown, obtaining accurate \(\beta_{t}^{i}(a_{t}^{i})\) and \(\alpha_{t}^{i}(a_{t}^{i})\) is difficult. Instead, in this work, we adopt a data-driven approach to learn reward models and estimate the performance metrics. _Second_, in real-world cases, only a limited experimental budget is allowed by network operators leading to insufficient diverse historical data (state, action pairs) to train a data-driven learning model. In our design, we use a data augmentation technique in the form of neighbor cell augmentation to enrich the features from each cell. _Third_, the handover parameter configuration is affected by adjacent cells. Thus, it is essential to model the information coming from the adjacent cells to achieve accurate reward modeling. _Lastly_, optimizing one performance metric greedily might hinder another, thus, how to jointly optimize different performance metrics needs careful consideration.

## 4 Temporal Auto-Grouping GCN for Reward Modeling

In order to better capture the dependency between each cell and its neighboring cells, we _first_ introduce our novel method for neighboring cell feature aggregation. _Second_, we propose a temporal feature aggregation step with recurrent neural networks (RNN) to model the temporal correlation from the historical sequence of the network states. _Third_, we elaborate the overall training process, considering the impact from the neighboring cells, the temporal correlation in the network and the action we aim to optimize.

### Spatial Feature Modeling

The handover parameters heavily impact the learning problem on the graph of the center cells as well as the neighboring cells, hence, we aim to capture the neighboring cells information during our modeling process. Recently, message-passing neural networks (MPNNs) in the form of graph neural networks (GNNs) have been introduced and showed to be effective in modeling real world applications with structural information. The dependencies in the dataset are modeled using a graph (Hamilton et al., 2017; Ying et al., 2018; Wang et al., 2019). In each layer of a GNN, each node's representation includes the features from itself as well as the features from its neighboring nodes (messages sent from the neighborhood). We believe the GNN framework is suitable for handling the dependencies between the center cell and the neighboring cells in cellular networks. We present more details on GNN and recent works on homogeneous and heterogeneous graphs in the Sec. A.3.

Graph-Based Cellular Network ModelingWe construct a graph \(\mathcal{G}_{t}\!=\!(\mathcal{V}_{t},\mathcal{E}_{t},\mathbf{X}_{t})\) for day \(t\), where each node \(v\!\in\!\mathcal{V}_{t}\) represents one cell and is associated with a feature vector \(\mathbf{x}_{t}^{v}\!\in\!\mathbb{R}^{d}\) (\(v\)-th column of \(\mathbf{X}_{t}\!\in\!\mathbb{R}^{d\times|\mathcal{V}_{t}|}\)), including the statistical properties of node \(v\) measured on day \(t\). The statistical properties could include several features such as the antenna transmission power, physical resource block (PRB) usage ratio, the amount of data traffic, and the transmission bandwidth. These features serve as the node attributes. The edge set \(\mathcal{E}_{t}\) encodes the interactions between cells based on
the handover events between pairs of cells. Based on historical data, if any pair of cells has an average number of handover events above a threshold \(\tau\), we assume an edge between those two cells. The neighboring set for node \(v\) is denoted as \(\mathcal{N}_{t}^{g}(v)\)=\(\{u|u\in\mathcal{V}_{t},(u,v)\in\mathcal{E}_{t}\}\).

Due to the heterogeneous nature of the cellular network, the relationships between the neighboring cells can be complex. Concretely, there might be an implicit \(M\) latent relationship types \(\mathcal{R}=\{r_{1},r_{2},\cdots,r_{M}\}\) that can be learned to better handle the complex interactions in the cellular networks. Assuming each cell is represented by its states such as PRB usage, traffic, etc. in the network graph, we aim at dividing the neighboring cells into different groups, each of which will provide some information that is shared between the neighbors in that group and help to better capture the rich information from neighboring cells in a distinguishable way. Thus, inspired by the above motivation and a recent work [20], we propose a novel GCN approach called auto-grouping GCN (AG-GCN) to characterize this special property of cellular networks when handling the interactions between neighboring cells. In the following, we elaborate upon the detailed steps to realize our design.

Neighborhood AugmentationIn cellular network modeling, since the experiment budget is limited, the historical data (state-action pairs) is not diverse enough to train our data-driven model. Besides, since we construct the graph based on the handover events, there are cells that have a very limited number of neighboring cells. Thus, in our design, we use a data augmentation technique in the form of neighbor cell augmentation based on the similarity between cells in a latent space, to enrich the features of each cell.

We define a feature transformation function \(f(\cdot):\mathbb{R}^{d}\rightarrow\mathbb{R}^{l}\) which maps the input node feature \(\mathbf{x}_{t}^{v}\in\mathbb{R}^{d}\) to a latent space \(\mathbf{y}_{t}^{v}=f(\mathbf{x}_{t}^{v})\in\mathbb{R}^{l}\). In order to capture the long-range dependencies and similarity in the cellular network, we design an additional neighborhood in the latent representation space based on Euclidean distance. For each node \(v\in\mathcal{V}_{t}\), we form the augmented neighborhood \(\mathcal{N}_{t}(v)=\mathcal{N}_{t}^{g}(v)\cup\mathcal{N}_{t}^{s}(v)\), where \(\mathcal{N}_{t}^{g}(v)\) and \(\mathcal{N}_{t}^{s}(v)\) are the neighbors of node \(v\) in the original graph and in the latent space, respectively. The neighbors in the latent space are selected based on their Euclidean distance to the center cell. The \(n\) nearest nodes in the latent space are selected to create \(\mathcal{N}_{t}^{s}(v)\) for cell \(v\), where the number of nodes we select based on the feature similarity is equal to the neighborhood size in the original graph \(|\mathcal{N}_{t}^{g}(v)|\)=\(|\mathcal{N}_{t}^{s}(v)|\)=\(n\). The neighbor augmentation module in Fig. 1 illustrates this process.

Neighborhood Auto-GroupingOnce we have obtained the augmented neighborhood set, the neighbors in the augmented neighborhood \(\mathcal{N}_{t}(v)\) are divided into different groups by a geometric operator \(\gamma\). Consider node \(v\) and its neighbor node \(u\in\mathcal{N}_{t}(v)\). The relation between them on day \(t\) is denoted as \(\gamma(\mathbf{y}_{t}^{v},\mathbf{y}_{t}^{u}):(\mathbb{R}^{l},\mathbb{R}^{l}) \rightarrow\mathcal{R}=\{r_{1},r_{2},\cdots,r_{M}\}\). This grouping aims at combining neighbors' information in groups with similar inter-group features. For each group \(r_{i}\in\mathcal{R}\), the neighborhood feature set on day \(t\) is defined as \(\mathcal{N}_{t}^{r_{i}}(v)=\{u|u\in\mathcal{N}_{t}(v),\gamma(\mathbf{y}_{t}^{v },\mathbf{y}_{t}^{u})=r_{i}\}\). The auto-grouping module in Fig. 1 demonstrates this process. Note that yellow neighbors (marked with *) are the projected counterparts of the neighbors in the graph space, while the green neighbors (marked with +) correspond to the augmented neighbors from the latent space.

Conditional Message PassingSince the order within each neighbor group should not impact the output of the representation, we apply a permutation invariant function \(\pi(\cdot)\) on the neighbors within each group (mean pooling across each feature dimension) and aggregate them separately. Fig. 1 shows an example of the AG-GCN, where \(l=2\) and \(|\mathcal{R}|=4\) and the representation after the permutation invariant function \(\pi(\cdot)\) is shown by black dashed arrows ended to nodes 1, 2, 3, and 4. Then for each group \(r_{i}\in\mathcal{R}\), a non-linear transform is further applied as:

\[\mathbf{z}_{t}^{v,r_{i}}=\sigma\Big{(}\mathbf{W}_{t}^{v,r_{i}}\cdot\pi\big{(} \{\mathbf{x}_{t}^{u}|u\in\mathcal{N}_{t}^{r_{i}}(v)\}\big{)}\Big{)}, \tag{3}\]

where \(\mathbf{W}_{t}^{v,r_{i}}\) is a learnable weight matrix for the neighbors in group \(r_{i}\) of node \(v\) on day \(t\), and \(\sigma(\cdot)\) is a non-linear function, e.g., \(\tanh\). Then for each node we aim to aggregate the transformed neighborhood features from their different groups of neighbors in a distinguishable way. The vectors \(\mathbf{z}_{t}^{v,r_{i}}\) for \(r_{i}\in\mathcal{R}\) are further aggregated as \(\mathbf{h}_{t}^{v}=[\mathbf{z}_{t}^{v,r_{1}};\cdots;\mathbf{z}_{t}^{v,r_{M}}]\), where \([\ ;\ ]\) represents concatenation.

### Temporal Feature Modeling

Capturing the trend in the evolution of the states of each cell within a day properly can benefit the prediction of the performance metric for the following day. We propose to use additional temporal features for each center cell to extract the changing dynamic pattern of its states within each day to further improve the reward model performance. We assume the samples of the center cell \(v\) on day \(t\) can be divided into \(K\) groups by their temporal order. For all the samples in each group \(k\), we take the average network state for each group of samples and denote it as \(\mathbf{x}_{t,k}^{v}\). We use an RNN layer to capture this temporal dependency of the features from different groups by feeding all the network states as an input sequence \(\mathbf{P}_{t}^{v}=\big{[}\mathbf{x}_{t,1}^{v}\cdot\mathbf{x}_{t,2}^{v}\cdot \cdots;\mathbf{x}_{t,K}^{v}\big{]}^{T}\in\mathbb{R}^{K\times d}\), to obtain

\[\mathbf{c}_{t}^{v}=\mathrm{RNN}(\mathbf{P}_{t}^{v},\delta)\in\mathbb{R}^{d^{ \prime}}, \tag{4}\]

where \(\delta\) and \(d^{\prime}\) are the set of trainable parameters and the output dimension of the RNN layer, respectively.

### Overall Training Pipeline

The main purpose of the model is to estimate the real network's response, and predict the throughput ratio and throughput of the center cell for the next day based on the observed network states in the current day. These performance metrics are not only affected by the current day's states, but also highly correlated with the action we choose to configure for the next day. Thus, we also consider the actions of the next day. Furthermore, the throughput ratio and throughput of the next day are highly dependent on the previous performance metrics. Hence, we consider the current throughput ratio, i.e., \(\beta_{t}^{v}\), in the prediction process.

To make the final prediction, the learned representation of the neighborhood by the AG-GCN aggregation, the temporal features of the center cell, and the throughput ratio of the
current day, i.e. \(\beta^{v}_{t}\), are concatenated to form the state vector of cell \(v\) as \(\mathbf{s}^{v}_{t}=\Psi(\mathbf{W}^{v}_{t}\cdot\left[\beta^{v}_{t};\mathbf{c}^{v }_{t};\mathbf{h}^{v}_{t}\right])\), where \(\mathbf{W}^{v}_{t}\) is a learnable weight matrix for node \(v\) on day \(t\), and \(\Psi(\cdot)\) is a non-linear function, e.g., tanh. Since the final representation should be sensitive to the chosen input action (of which the decision making process will be elaborated in Sec. 5), the throughput ratio and throughput of the next day for cell \(v\) are formulated as the output of a non-linear transformation \(\Lambda(\cdot)\) function of state and action:

\[\hat{\beta}^{v}_{t+1} =\Lambda\big{(}\mathbf{W}^{v}_{\beta}\cdot([\mathbf{s}^{v}_{t}; a^{v}_{t}])\big{)}, \tag{5}\] \[\hat{\alpha}^{v}_{t+1} =\Lambda\big{(}\mathbf{W}^{v}_{\alpha}\cdot([\mathbf{s}^{v}_{t}; a^{v}_{t}])\big{)}, \tag{6}\]

where \(\mathbf{W}^{v}_{\beta}\) and \(\mathbf{W}^{v}_{\alpha}\) are trainable matrices of node \(v\) for throughput ratio and throughput models, respectively. The overall flow of data from the graph structure to the final prediction is represented in Fig. 1. Note that we train two separate models for predicting the throughput and the throughput ratio simultaneously.

In order to properly use the A2-threshold for the prediction, we use the change in this parameter compared to the previous day as the action \(a^{v}_{t}=A2^{v}_{t+1}-A2^{v}_{t}\), where \(A2^{v}_{t+1}\) and \(A2^{v}_{t}\) are the A2-thresholds for cell \(v\) on day \(t+1\) and \(t\), respectively. The reason for this design choice has twofold. First, the original action space of A2 is large, but the range of the change of action can be smaller by controlling the adjustment steps, making it easier for the model to learn and conduct the decision making step. Besides, the delta action directly reflects the change in the cell coverage/loads, so they are more sensitive to the performance metrics. To form the training objective, we consider data of \(T+1\) consecutive days and form the pairs \((t,t+1)\), \(t\in\{1,2,\cdots,T\}\), to predict the throughput ratio and throughput of the center cell in day \(T+1\), trained by minimizing the following loss functions respectively:

\[\frac{1}{T}\sum_{t=1}^{T}\frac{1}{N_{t}}\sum_{v=1}^{N_{t}}(\hat{ \beta}^{v}_{t+1}-\beta^{v}_{t+1})^{2}+\lambda_{1}||\Theta_{1}||^{2}, \tag{7}\] \[\frac{1}{T}\sum_{t=1}^{T}\frac{1}{N_{t}}\sum_{v=1}^{N_{t}}(\hat{ \alpha}^{v}_{t+1}-\alpha^{v}_{t+1})^{2}+\lambda_{2}||\Theta_{2}||^{2}, \tag{8}\]

where \(\lambda_{1}\) and \(\lambda_{2}\) are the hyperparameters chosen for regularization. \(\Theta_{1}\) and \(\Theta_{2}\) represent all the trainable parameters in the models. The trained reward model is now able to mimic the real network and predict both throughput ratio and throughput of each center cell for the coming day and can be used to check the impact of actions towards the performance metrics we are considering.

## 5 Action Configuration

As discussed in the earlier sections the main objectives to consider in the action configuration process are load balancing, identified by the throughput ratio, and the cell throughput. Hence, the best action for cell \(v\) on day \(t\), i.e. \(a^{v}_{t}\in\mathcal{A}\), is the one that optimizes the problem in (2). In general, when dealing with a multi-objective problem, different objectives are often conflicting, and we may not be able to optimize them simultaneously. One common way to tackle this problem is to give different objectives weights and optimize the weighted objective value. However, in our scenario, it is difficult to determine the weights and different clusters may require cluster-specific weights. Here we break the problem in (2) into two sub-problems, and solve them sequentially. We first optimize the action with respect to the predicted throughput ratio, i.e., \(\hat{\beta}^{v}_{t+1}(a^{v}_{t})\) for cell \(v\) on day \(t\), where \(a^{v}_{t}\in\mathcal{A}\), and then optimize the throughput \(\hat{\alpha}^{v}_{t+1}(a^{v}_{t})\).

Specifically, the throughput ratio is optimized and we find the set of best \(c\) values for \(a^{v}_{t}\), denoted \(\mathcal{A}^{v}_{c}\), such that

\[\underset{a^{v}_{t}\in\mathcal{A}^{v}_{c}}{\min}\!\!-\!\sqrt{\left|1-\hat{ \beta}^{v}_{t+1}(a^{v}_{t})\right|}\!\!\geq\!\!\!\max_{a^{v}_{t}\in\mathcal{A} -\mathcal{A}^{v}_{c}}\!\!\sqrt{\left|1-\hat{\beta}^{v}_{t+1}(a^{v}_{t})\right|}. \tag{9}\]

Then, our goal is to achieve the maximum possible throughput for cell \(v\) on day \(t\) with this is through

\[\hat{a}^{v}_{t}=\underset{a^{v}_{t}\in\mathcal{A}^{v}_{c}}{\arg\max}\ \hat{\alpha}^{v}_{t+1}(a^{v}_{t}). \tag{10}\]

\(\hat{a}^{v}_{t}\) is then the final recommended action for cell \(v\) on day \(t\). This procedure for all the \(N_{t}\) cells of the network on day \(t\) is presented in Algorithm 1.

## 6 Experimental Results

The experiments are conducted on a large-scale cellular network simulator constructed from real-world data which presented in Sec. B.1. We use principal component analysis

Figure 1: The flow of information from graph structure to final prediction, used to form the training pipeline of two models for predicting the throughput \(\hat{\alpha}\) and the throughput ratio \(\hat{\beta}\) for \(\mathcal{T}_{in}=\{1,2,\cdots,t-1\}\). In this demo example, the auto-grouping module constructs \(M=4\) groups of neighbors, where \(l=2\). Empty groups are filled with the average of other groups.
(PCA) as the mapping function \(f(\cdot)\), as defined in Sec. 4, for obtaining the latent representation in the AG-GCN step. It transforms the original feature into a 2-dimensional space to perform the neighborhood augmentation and the neighbors group assignment process. After this transformation, the relationship operator \(\gamma\) for the auto-grouping assigns a group to each subset of points in each quadrant of this two dimensional space presented in Table 1. The permutation-invariant function \(\pi\) applied on each group of neighbors is average in our experiments.

### Datasets

To perform our experiments and evaluate the proposed model, two datasets are used in this study (see Sec. B.2 for more details):

Dataset-A:A real metropolitan cellular network containing around 1500 cells sampled hourly and collected from Oct. 17 to Oct. 31, 2019. Each data sample contains information such as the cell ID, sample time, configuration of cell parameters, and measurements of the cell states.

Dataset-B:Also a real metropolitan cellular network. The network contains 1459 cells, and the data is collected from Sep. 1 to Sep. 29, 2021. Each data sample contains similar information as above.

### Reward Model Accuracy Evaluation

Dataset GenerationIn order to evaluate the prediction accuracy of our model, we use a simulator to modify Dataset-A with a random policy to diversify our network configuration. On each day, the A2-threshold for each cell is randomly selected around the default action -100 dBm within the range \([-105,-95]\). This approach provides us a fix data buffer with diverse action dataset to train all models and have a fair comparison of their accuracy. For Dataset-B, there exists a reasonable amount of the diversity in the handover parameter configuration, thus we directly use the raw dataset from the live network to perform the training and evaluation.

Training Process and MetricsAs samples generated hourly, we aggregate them within each day as described in Sec. 4. To evaluate the model accuracy in predicting cell throughput and throughput ratio, we train the model with the generated pairs \(\{(1,2),\cdots,(t-1,t)\}\) for \(t=9\) and 12 days for Dataset-A and B, respectively. At each day \(t>2\), data pairs \(\{(1,2),\cdots,(t-2,t-1)\}\) are used as training and validation sets, and \((t-1,t)\) serves as testing set for evaluation across different models. We report the mean square error (MSE) to measure the reward model performance.

Comparison with Benchmark ModelsIt is important to note that, the social impact of this work has not been addressed by ML approaches the same way as we propose. Due to the uniqueness of our problem, existing solutions for optimizing handover parameters are either not appropriate to solve it or there is no apparent way to directly adapt them to our problem. For instance, traditional handover optimization methods rely on designing fuzzy rules based on different measures of QoS in the network [20], however, designing proper rules is complex and cannot handle the change in highly dynamic systems well. Instead, we hope to use a data-driven approach, among which the (deep) RL method gains the most attention [1, 23]. However, in this type of problems, the network provider only allows limited exploration of the parameter values (e.g., allows changing the A2 value once a day) to ensure the stability of the network. Thus, we only have limited days for exploring the best action. RL models, nevertheless, usually need longer episodes to optimize the accumulated return. Despite this fact, we made some preliminary attempt, presented in Sec. B.3, to adapt the RL paradigm from the literature into our problem which did not show any advantage over our simpler design.

In order to show the effectiveness of our proposed reward model, we compare it with alternative designs for the prediction model. It should be mentioned that all of these models are our contribution. In the Sec. B.4, we summarize our benchmarks and the properties of each model. The first model is MLP, where we only use the features of the center cells and ignore the neighboring cells' features. In GCN model, we follow the typical GCN formulation [1] and process the network as a homogeneous

\begin{table}
\begin{tabular}{|c|c|c|} \hline \(\gamma(\mathbf{y}_{t}^{v},\mathbf{y}_{t}^{u})\) & \(\mathbf{y}_{t}^{v}[0]>\mathbf{y}_{t}^{u}[0]\) & \(\mathbf{y}_{t}^{v}[0]\leq\mathbf{y}_{t}^{u}[0]\) \\ \hline \(\mathbf{y}_{t}^{v}[1]\leq\mathbf{y}_{t}^{u}[1]\) & 2 & 1 \\ \(\mathbf{y}_{t}^{v}[1]>\mathbf{y}_{t}^{v}[1]\) & 3 & 4 \\ \hline \end{tabular}
\end{table}
Table 1: The relationship operator \(\gamma\)

\begin{table}
\begin{tabular}{|c|c|c|} \hline \(\gamma(\mathbf{y}_{t}^{v},\mathbf{y}_{t}^{u})\) & \(\mathbf{y}_{t}^{v}[0]>\mathbf{y}_{t}^{u}[0]\) & \(\mathbf{y}_{t}^{v}[0]\leq\mathbf{y}_{t}^{u}[0]\) \\ \hline \(\mathbf{y}_{t}^{v}[1]>\mathbf{y}_{t}^{v}[1]\) & 2 & 1 \\ \(\mathbf{y}_{t}^{v}[1]>\mathbf{y}_{t}^{v}[1]\) & 3 & 4 \\ \hline \end{tabular}
\end{table}
Table 1: The relationship operator \(\gamma\)

Figure 2: Achieved MSE of the throughput for test data of (a) Dataset-A and (b) Dataset-B for different methods
graph where the neighbor information is aggregated jointly without distinction. The AG-GCN model ignores the temporal dependencies of the data which we consider in TAG-GCN model. In Fig. 2, we compare the prediction accuracy of these models for throughput in Dataset-A and B. We observe on average the best accuracy for the test set is achieved by AG-GCN and TAG-GCN, with TAG-GCN performing marginally better on the average rank metric across the evaluation days, indicating that our neighbor aggregation and temporal features extraction have a considerable impact on the reward modeling for cellular networks. The same results also achieved for the throughput ratio model. The same test is performed for throughput ratio and included in the supplementary materials in Fig. 10.

### Overall Parameter Optimization Performance

The Action Recommendation ProcessIn the following experiments we use the presented models to recommend the actions for Dataset-A. The actions in day 1, i.e., Oct. 17, has been set to the default action which is -100 dBm. Unless otherwise stated, the action for the second day, i.e., Oct. 18, is initialized by a set of random actions around the default action in the range of \([-105,-95]\). The model is trained iteratively on each day and used to recommend actions for the next day. The process is depicted in Fig. 3, where states of the cells on day \(t\) are given to the trained model to predict performance metrics of the network on day \(t+1\) and the action \(a_{t}^{v}\) is adjusted for each cell based on the predictions. Finally, the network states and performance measurements for day \(t+1\) are computed according to the new selected action by the cellular network simulator and used for model training and action recommendation in the following day.

Baseline Performance BoundsIn addition to the result achieved by the actions recommended by the models, we use three baseline performance bounds achieved by the _default A2-threshold_, the _expert rule_, and the _optimal actions_ of the simulator. As stated before the default A2-threshold value is \(-100\) dBm and this is used as the lower bound in the following experiments. The optimal actions in the simulator are obtained by brute-force search and it introduces the upper performance bound. The expert rule-based method provided by experienced network operator is a simple rule presented in the Sec. B.6. The performance achieved by the expert rule is better than the default action. We hope to use our proposed learning based framework to further fill the gap with the reward achieved by optimal actions.

ResultsIn the following experiments, we compare the performance of the network under the actions recommended by different models in terms of the network throughput as defined in Sec. 3. We plot the trajectory of the throughput difference to the default A2-threshold baseline (dash black line) in Fig. 4. We repeat all the experiments 20 times for all models, where each run uses the same set of random actions on the first action exploration day (Oct. 18) for all the models. We also show the performance achieved through the expert rule action recommendation, default action, and the optimal actions of the simulator (random actions are also used on Oct. 18 for the curve of the optimal action). The quantitative results for Fig. 4 are summarized in Table 6 in the supplementary materials. TAG-GCN can achieve better average throughput in the final days which indicates the importance of our auto-grouping GCN design to tailor the heterogeneous property of the cellular networks. Besides, as expected, all the learning-based models can beat the expert rule algorithm which is highly dependent on human experience and is unable to recover from the performance degradation due to bad random initialization on the first day. Furthermore, to show the effectiveness of our proposed model in terms of load balancing and enhancing cluster throughput ratio, we illustrate the progress of this ratio achieved by TAG-GCN for some selected severely unbalanced cells in Fig.5. As it can be seen, the throughput ratio of the clusters form a trajectory that converges to 1 which is the ideal target value. More ablation studies are presented in Sec. B.7.

Based on the above experiments, our proposed ML-based solutions can improve the network performance and optimize the handover process compared to the conventional methods such as using the default action or human experts rule-based methods. Moreover, the automation of the parameter optimization process achieved by our ML-based solutions reduces the domain expert's intervention and, hence, the management cost of network operators and improves the maintenance efficiency of cellular networks. Consequently, the proposed solutions open up the possibilities to provide reliable and high-quality network access even to geographical areas that are currently underserved by the cellular network. This can bring exciting new opportunities to these regions such as remote education, remote working, health monitoring, video streaming, etc.

## 7 Conclusion

In this paper, we study the handover parameter configuration problem in cellular networks. We propose a reward prediction model to accurately imitate the cellular network and estimate the performance metrics. Our proposed model, i.e.,

Figure 4: Performance comparison of different models along with optimal action curve initialized with random actions

Figure 3: The process of action recommendation by the trained model and the simulator
TAG-GCN, investigates the impact of the adjacent cells and differentiate their impact on the center cell of each cluster. We also consider the network changing dynamics in our model to learn the temporal dependencies in the data. Based on the reward model, a novel multi-objective parameter configuration strategy is proposed to perform the optimization for each cluster and balance the performance metrics in each neighborhood. The conducted simulations shows the superiority of TAG-GCN which has a huge potential social impact by improving the cellular network parameters and providing massive connectivity and high coverage with a balanced traffic across the network. Hence, this can help the widespread adaptation of new technologies to benefit many sectors such as health and education.
