# OLGA: One-cLass Graph Autoencoder

Marcos Paulo Silva Gôlo, José Gilberto Barbosa de Medeiros Junior,
Diego Furtado Silva & Ricardo Marcondes Marcacini
Institute of Mathematical and Computer Sciences
University of Sao Paulo
São Carlos, São Paulo, Brazil
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>

###### Abstract

One-class learning (OCL) comprises a set of techniques applied when real-world
problems have a single class of interest. The usual procedure for OCL is
learning a hypersphere that comprises instances of this class and, ideally,
repels unseen instances from any other classes. Besides, several OCL
algorithms for graphs have been proposed since graph representation learning
has succeeded in various fields. These methods may use a two-step strategy,
initially representing the graph and, in a second step, classifying its nodes.
On the other hand, end-to-end methods learn the node representations while
classifying the nodes in one learning process. We highlight three main gaps in
the literature on OCL for graphs: (i) non-customized representations for OCL;
(ii) the lack of constraints on hypersphere parameters learning; and (iii) the
methods’ lack of interpretability and visualization. We propose One-cLass
Graph Autoencoder (OLGA). OLGA is end-to-end and learns the representations
for the graph nodes while encapsulating the interest instances by combining
two loss functions. We propose a new hypersphere loss function to encapsulate
the interest instances. OLGA combines this new hypersphere loss with the graph
autoencoder reconstruction loss to improve model learning. OLGA achieved
state-of-the-art results and outperformed six other methods with a
statistically significant difference from five methods. Moreover, OLGA learns
low-dimensional representations maintaining the classification performance
with an interpretable model representation learning and results.

##  1 Introduction

Graphs offer a powerful structure for modeling real-world problems by
explicitly capturing node relations. Edges represent their connections,
providing a comprehensive view of interconnectedness and dependencies. This
explicit representation of relations is essential in domains such as social
networks, recommendation systems, and biological networks, enabling accurate
analysis and prediction (Zhou et al., 2020; Xia et al., 2021; Deng et al.,
2022).

Different classification problems involving an interest class are commonly
modeled using graphs. These include tasks such as classifying fake news (de
Souza et al., 2022), identifying events of interest (Nguyen & Grishman, 2018),
predicting hit songs (da Silva et al., 2022), detecting fraud (Dou et al.,
2020), and performing anomaly detection (Liu et al., 2022a). Notably, in many
of these tasks, humans can naturally detect the interest class solely by
observing instances of interest. In machine learning, this strategy is
represented by One-Class Learning (OCL) approaches, which enable the
classification of interest instances by training solely on samples from this
class (Emmert-Streib & Dehmer, 2022). OCL reduces the need for extensive
labeling efforts, is suitable for unbalanced scenarios, and does not require
comprehensive coverage of the non-interest class(es) (Tax, 2001; Khan &
Madden, 2014; Fernández et al., 2018; Alam et al., 2020).

OCL and graphs are attractive strategies for solving real-world problems with
an interest class (Wang et al., 2021; Feng et al., 2022; Deldar et al., 2022).
First, Two-step methods employed unsupervised graph neural networks (GNNs) to
generate representations for graph nodes, followed by OCL algorithms to
classify the interest nodes (Gôlo et al., 2022; da Silva et al., 2022; Huang
et al., 2022b; Ganz et al., 2023). Recent advancements have focused on end-to-
end one-class graph neural networks (OCGNNs), which simultaneously learn
representations and classify instances as belonging to the interest class
(Feng et al., 2022; Wang et al., 2021; Huang et al., 2021; Deldar et al.,
2022). A common strategy in both approaches is defining a hypersphere to
encapsulate the interest representations, aiming to separate them from non-
interest (Tax & Duin, 2004; Wang et al., 2021).

Significant gaps need to be addressed when OCL and GNNs are combined. Firstly,
two-step methods rely on pre-learning data representations, which are not
customized for the OCL step (Wang et al., 2021). On the other hand, current
end-to-end methods lack studies on graphs from different domains and lack
constraints on hypersphere parameters learning, often encapsulating all
instances of interest and non-interest (Ruff et al., 2018; Feng et al., 2022;
Wang et al., 2021). Another significant limitation in current research is the
representation learning process interpretability issue (Wu & Mooney, 2019; Liu
et al., 2022b).

This paper introduces OLGA (One-cLass Graph Autoencoder), a new end-to-end
one-class graph neural network method. OLGA achieves meaningful node
representations and state-of-the-art one-class node classification by
combining two loss functions. The first is based on the graph autoencoder
loss, which aims to reconstruct and preserve the graph topology by mapping
nodes into a new latent space. The second is a newly proposed hypersphere loss
that leverages instances from the interest class to enhance one-class-oriented
representation learning and classification. The GAE reconstruction loss
function works as a constraint so that the hypersphere loss function does not
completely bias learning since this total bias can leave all instances within
the hypersphere. Moreover, OLGA also learns low-dimensional representations to
improve the classification performance and introduce interpretability to the
learning process. In summary, our contributions are:

  1. 1.

We present OLGA, a new end-to-end one-class graph neural network method for
node classification that simultaneously learns meaningful representations and
classifies nodes.

  2. 2.

We propose a new hypersphere loss function for one-class graph neural
networks.

  3. 3.

We demonstrate that low-dimensional one-class representations in OLGA support
interpretability for model representation learning and improve classification
performance.

  4. 4.

We evaluate OLGA against other state-of-the-art methods on eight datasets from
various domains and sources to confirm its performance in practical
application scenarios.

We carried out an experimental evaluation using eight one-class datasets from
diverse domains and sources. We compared OLGA with six other methods,
including three OCGNNs with different GNN architectures and three strong
baselines: Deep-walk, Node2Vec, and Graph Autoencoder (GAE) combined with the
One-Class Support Vector Machine. OLGA achieved state-of-the-art performance
on most datasets considering textual, image, and tabular domains outperforming
the other six methods. Our method demonstrated a statistically significant
improvement over five of the compared methods. Moreover, OLGA demonstrated the
potential to learn low-dimensional representations enabling an interpretable
learning process and data visualizations for OCL.

##  2 Related Work

We divide the related work into two categories. The first category comprises
the two-step methods. These methods generate embeddings through unsupervised
graph neural networks and, in another step, apply a one-class learning
algorithm. The second category comprises the end-to-end methods that learn the
representations while classifying the nodes in one step (learning process).

###  2.1 Two-Step Methods

Two-step methods apply graph neural networks (GNNs) in the first step. GNNs
are state-of-the-art for representation learning on graphs. GNNs also capture
structural features of the graphs and aggregate information from neighboring
nodes associated with the interest class to generate embeddings for the nodes.
Finally, in the second step, these methods use OCL algorithms to solve the
one-class problem (da Silva et al., 2022; Huang et al., 2022b; Ganz et al.,
2023; Gôlo et al., 2022). In this sense, Gôlo et al. (2022) proposed to
perform the movie recommendations through one-class learning. The authors used
unsupervised graph neural networks with a link prediction loss to learn
representations for the movies and users in the graph. The study used the One-
Class Support Vector Machines (OCSVM) to classify.

da Silva et al. (2022) propose to detect hit songs through OCL. The authors
used an unsupervised heterogeneous graph neural network to learn
representations for the songs and, in the second step, used the OCSVM to
classify and measure how famous a song is. Furthermore, Huang et al. (2022b)
proposed to detect intrusions in systems through OCL. The authors proposed the
One-Class Directed Heterogeneous graph neural network. This method combines an
unsupervised heterogeneous GNN with the Deep Support Vector Data Description
(DeppSVDD) algorithm (Ruff et al., 2018) to detect the anomalies. Ganz et al.
(2023) detects backdoor software through OCL. Through collaborative graphs
with commit nodes, branches, files, developers, and methods (functions), the
authors represent the graph nodes through an unsupervised heterogeneous GNN,
specifically a variational graph autoencoder (Kipf & Welling, 2016). In the
second step, Ganz et al. (2023) uses DeppSVDD to detect backdoor software.

Two-step methods obtained state-of-the-art results in learning robust
representations through GNNs for the OCL. However, these methods generate non-
customized and agnostic representations for the OCL classification algorithm
since the representation is learned independently of the OCL algorithm. This
fact can limit the learned representation quality and negatively impact the
OCL algorithm. End-to-end methods overcome this limitation by combining these
two steps into one.

###  2.2 End-To-End Methods

End-to-end methods learn representations and classify instances in a single
learning process. In the OCL, this is challenging due to the lack of
counterexamples. In this sense, we need an appropriate loss function based on
one class. In this way, the methods learn embeddings customized and non-
agnostic that capture structural information from the graph, aggregating
information from neighboring nodes and learning the pattern of the interest
class while classifying instances (Feng et al., 2022; Wang et al., 2021; Huang
et al., 2021; Deldar et al., 2022). In this sense, Feng et al. (2022) proposed
to detect anomalies in IIoT Systems through OCL. The authors proposed using
GNNs through a graph autoencoder to learn representations, and for
classification, the authors used a threshold applied to the losses generated
by the graph autoencoder.

In a pioneering way, Wang et al. (2021) proposed the one-class graph neural
network (OCGNN), a graph neural network with a loss function similar to SVDD,
learning embeddings for the nodes while encapsulating the interest instances
through a hypersphere. The OCGNN was applied in graph anomaly detection (GAD).
Later, Huang et al. (2021) proposed the GAD through the one-class temporal
graph attention network method, a method similar to OCGNN that considers
temporal information, attention mechanisms, and dynamic graphs with de
hypersphere loss. Deldar et al. (2022) performs Android malware detection
through a graph autoencoder (GAE) with a customized loss via thresholds. The
authors used a loss function through an anomaly score generated by the
difference between the initial representation and the representation decoded
by the GAE.

Most end-to-end methods are based on the One-Class Graph Neural Network
(OCGNN) from Wang et al. (2021). Formally, the OCGNN from Wang et al. (2021)
minimizes Equation 1:

$\mathcal{L}(r,\bm{{W}})=\frac{1}{\nu|\bm{V}^{\text{in}}|}\sum_{i=1}^{|\bm{V}^{\text{in}}|}[\|g(\bm{{V}},\bm{A};\bm{{W}})_{i}-\bm{c}\|^{2}-r^{2}]^{+}+r^{2}+\frac{\lambda}{2}\sum_{l=0}^{L}\|\bm{{W}}^{l}\|^{2},$
(1)

in which $\bm{V}$ is the node set, $\bm{V}^{\text{in}}$ is the interest node
set, $\bm{A}$ is the adjacency matrix, $g(\bm{{V}},\bm{A};\bm{{W}})$ is a
traditional graph neural network, $r$ is the radius of the hypersphere that is
learned alternately with the neural network weights $\bm{{W}}$, $\nu\in[0,1)$
is an upper bound on the fraction of training errors and a lower bound of the
fraction of support vectors, $\bm{c}$ is the center of the hypersphere defined
as the mean of the embeddings of the nodes in the class of interest through an
initial forward propagation, $\lambda$ is the weight decay regularizer of the
OCGNN, and $L$ is the number of neural network layers. This approach can be
used with different GNNs, such as Graph Convolutional Networks (GCN) (Kipf &
Welling, 2017), Graph Attention Networks (GAT) (Velickovic et al., 2017), and
GraphSAGE (Hamilton et al., 2017), by modifying the output of the term
$g(\bm{{V}},\bm{A};\bm{{W}})$.

These end-to-end methods are applied in GAD, thus lacking studies on different
domains. These methods based on hyperspheres lack constraints on the
hypersphere loss function, often biasing the learning and harming the
performance. For instance, in the OCGNNs, by using only the hypersphere loss
function, all instances (interest and non-interest) will gradually converge to
the center as GNN aggregates neighbor representations. Furthermore, in the
GAEs with thresholds that use only the reconstruct loss, all instances
(interest and non-interest) can converge for one single region since
autoencoders are vulnerable to converge to a constant mapping onto the mean,
which is the optimal constant solution of the mean squared error (Ruff et al.,
2018). Another gap is the lack of methods exploited in low dimensionality to
interpret and explain the learning process and to be used as visualization
methods. Finally, GAEs are explorers in different studies (Deldar et al.,
2022; Feng et al., 2022), and the hyperspheres loss function (Wang et al.,
2021; Huang et al., 2021). However, GAEs were not exploited in OCGNNs that use
the hypersphere loss function. One reason is the challenge of combining the
loss function of the GAEs and hyperspheres since we have different scales for
the loss functions. In this sense, in the next section, we present OLGA, the
One-cLass Graph Autoencoder, a method to mitigate these gaps.

##  3 OLGA: One-cLass Graph Autoencoder

We propose a novel end-to-end method for classifying interest nodes called
One-cLass Graph Autoencoder (OLGA). OLGA learns node representations while
classifying the nodes using hypersphere-based modeling (Tax & Duin, 2004). We
base our method on a graph autoencoder to capture the structural properties of
the graph through a reconstruction loss function (Kipf & Welling, 2016).
Additionally, we propose a new loss function to encapsulate the interest
instances that encourage these interest instances to approach the center even
within the hypersphere. This characteristic encourages unlabeled interest
instances to go into the hypersphere since labeled interest instances will
remain updated even within the hypersphere. Otherwise, the learning process
stabilizes, harming learning because unlabeled interest instance
representations may not be updated.

We utilize the GAE architecture with our loss functions to improve the one-
class learning in a multi-task learning way (Zhang & Yang, 2021; Sami et al.,
2022). Multi-task learning aims to improve the learning for each task by
leveraging the relevant information contained in multiple solved tasks (Zhang
& Yang, 2018). In the context of OCL, multi-task learning has shown promise in
improving the learning process (Xue & Beauseroy, 2016; Liu et al., 2021).
Furthermore, promising multi-task GNNs indicate that performing multi-task
learning also boosts each individual task while improving representation
learning (Ma & Mei, 2019; Xie et al., 2020). In this context, we propose a
multi-task learning approach using our loss functions for OCL in GNNs. Figure
1 illustrates OLGA.

Figure 1: OLGA illustration with our new hypersphere and GAE loss functions.

Formally, a GAE utilizes a GNN encoder and a decoder represented by an inner
product of the latent representation (Kipf & Welling, 2016). Equation 2
describes a GAE (Kipf & Welling, 2016).

$GAE=\left\\{\begin{matrix}Encoder:\bm{H}^{(L)}=g(\bm{{V}},\bm{A};\bm{{W}})\,\,\
\\\
Decoder:\bm{\hat{A}}=\sigma(\bm{H}^{(L)}\cdot\bm{H}^{(L)^{\intercal}})\end{matrix}\right.,$
(2)

in which, GAE learns $\bm{H}^{(L)}$ with the mean squared error between
$\bm{A}$ and $\bm{\hat{A}}$, and $\sigma(.)$ is a logistic sigmoid function.

Our main task is the classification of instances as belonging to the interest
class or not, which we define as $\mathcal{T}_{1}$. We define two additional
reconstruction tasks to improve the learning and, consequently, the
classification. The first one regards the reconstruction of the labeled nodes,
defined as $\mathcal{T}_{2}$. The other task is the reconstruction of
unlabeled nodes, defined as $\mathcal{T}_{3}$.

Let’s define some sets of nodes according to $\bm{V}$.
$\\{\bm{V}^{\text{in}},\bm{V}^{\text{u}}\\}\in\bm{V}$, where
$\bm{V}^{\text{in}}$ is the set of nodes of interest, and $\bm{V}^{\text{u}}$
is the set of unlabeled nodes. Let’s define $d_{i}$ as the value indicating
whether the interest instance is within the hypersphere with radius $r$ and
center $\bm{c}$ given by Equation 3:

$d_{i}=\|\bm{h}^{(L)}_{i}-\bm{c}\|^{2}-r^{2},$ (3)

in which, $\bm{h}^{(L)}_{i}\in\bm{H}^{(L)}$. Negative values of $d_{i}$
indicate that the node is inside the hypersphere, and positive values indicate
that it is outside.

For each task addressed in this study, we define a loss function. For
$\mathcal{T}_{1}$, we propose a new loss function based on the hypersphere
paradigm (Tax & Duin, 2004; Wang et al., 2021). The proposed hypersphere loss
function penalizes instances of interest outside the hypersphere. Also, when
the instance is inside the hypersphere, it continues to be penalized but to a
lesser extent to encourage it to move closer to the center. We define our loss
function in Equation 4. Figure $\mathcal{L}_{1}$ illustrates the loss function
$\mathcal{L}_{1}$ in a one-dimensional space.

Figure 2: Our new hypersphere loss function ($\mathcal{L}_{1}$) Illustration.

$\mathcal{L}_{1}(\bm{{W}})=\frac{1}{|\bm{V}^{\text{in}}|}\sum_{i=1}^{\bm{V}^{\text{in}}}f(d_{i}),$
(4)

$f(d_{i})=\left\\{\begin{matrix}d_{i}+1,\text{ if }d_{i}>0\,\,\ \\\
\exp(d_{i}),\text{otherwise}\end{matrix}\right..$ (5)

For $\mathcal{T}_{2}$, we define $\mathcal{L}_{2}$ as the reconstruction error
of the labeled nodes ($\bm{V}^{\text{in}}$), defined in Equation 6. For
$\mathcal{T}_{3}$, we define the loss function $\mathcal{L}_{3}$, which can be
defined as shown in Equation 7.

$\mathcal{L}_{2}(\bm{{W}})=mse(\bm{A^{in}},\hat{\bm{A}}^{\text{in}}),$ (6)

$\mathcal{L}_{3}(\bm{{W}})=mse(\bm{A}^{\text{u}},\hat{\bm{A}}^{\text{u}}),$
(7)

in which $\bm{A}^{\text{u}}$ is the adjacency matrix of the unlabeled nodes in
the graph, $\hat{\bm{A}}^{\text{u}}$ is the reconstruction of this matrix
generated by OLGA, $\bm{A}^{\text{in}}$ is the adjacency matrix of the labeled
nodes, and $\hat{\bm{A}}^{\text{in}}$ is the reconstruction of the
$\bm{A}^{\text{in}}$.

If we only use the $\mathcal{L}_{1}$, all instances will converge towards the
center, regardless of whether they are interest nodes, as the GNN aggregates
representations at each iteration. Therefore, we propose multi-task learning
with additional loss functions to assist our main task solved by
$\mathcal{L}_{1}$, combining the loss functions. Equation 8 presents our final
loss. Our strategy combines the GAE loss function with the hypersphere loss
function to work as a constraint so that the $\mathcal{L}_{1}$ does not
entirely bias the learning and improve the learning of representations and
classification performance.

$\mathcal{L}(\bm{{W}})=\mathcal{L}_{1}*\alpha+\mathcal{L}_{2}*\beta+\mathcal{L}_{3}*\delta,$
(8)

in which $\alpha$, $\beta$, and $\delta$ are enablers and disablers, i.e.,
have values 0 or 1 for $\mathcal{L}_{1}$, $\mathcal{L}_{2}$, and
$\mathcal{L}_{3}$ losses in the training process.

The update of the network parameters is done based on the partial derivative
of the losses. In the end, we have an addition of the partial derivatives for
the
$\frac{\partial\mathcal{L}_{1}}{\partial\theta_{e}}+\frac{\partial\mathcal{L}_{2}}{\partial\theta_{d}}+\frac{\partial\mathcal{L}_{3}}{\partial\theta_{d}}$,
in which $\theta_{e}$ represents the encoder parameters, i.e., $\bm{{W}}$ and
bias, and $\theta_{d}$ the decoder, i.e., $\bm{{W}}$, bias, and the inner
product. Since the $\mathcal{L}_{2}$ and $\mathcal{L}_{3}$ are an MSE based
well defined in the literature, we focus on defining the $\mathcal{L}_{1}$
partial derivative (Equation 9). So that way, in Equation 10, we have a total
partial derivative of

$\frac{\partial\mathcal{L}_{1}}{\partial\theta_{e}}=\frac{1}{|\bm{V}^{\text{in}}|}\sum^{|\bm{V}^{\text{in}}|}_{i=1}\frac{\partial}{\theta_{e}}f(d_{i}),$
(9)

$\frac{\partial
L_{\text{total}}}{\partial\theta}=\frac{\partial\mathcal{L}_{1}}{\partial\theta_{e}}+\frac{\partial\mathcal{L}_{2}}{\partial\theta_{d}}+\frac{\partial\mathcal{L}_{3}}{\partial\theta_{d}}.$
(10)

However, when combining two different loss functions, one potential issue is
their scales, where one loss could have a stronger influence on the network’s
learning, causing the other task to be ignored entirely. However, using
hyperbolic tangents serves as a solution to this problem. Note that in the
adjacency matrix represented by zeros and ones, both $\mathcal{L}_{2}$ and
$\mathcal{L}_{3}$ return values on a similar scale to those found in the
matrix. In this sense, we need to focus on the one-class loss function to
solve the problem. The hyperbolic tangent restricts the problem space to $-1$
and $1$ across all its dimensions. Therefore, the distance used for
calculating $\mathcal{L}_{1}$ is on a similar scale as $\mathcal{L}_{2}$ and
$\mathcal{L}_{3}$, avoiding the issue of imbalanced losses.

Once each dimension of the learned representation ranges between $-1$ and $1$
due to the hyperbolic tangent, the hypersphere’s radius will be in the range
$(0,1]$. In this scenario, the hypersphere volume tends to $0$ the larger the
dimension explored (Smith & Vamanamurthy, 1989). Therefore, with high
dimensions, OLGA will learn representations for interest instances very close
to the center until OLGA learns representations at a single point to
encapsulate the interest instances. Even if the task is being solved in OCL,
using these representations in a subtask becomes unfeasible.

If OLGA explores low dimensionality in its last layer ($2$ or $3$), the volume
of the hypersphere will not tend to $0$, which makes it possible for OLGA to
learn representations that can be used in other tasks and still solve the node
classification. In addition, when using low dimensions, OLGA becomes an
interpretable representation learning method since we can visualize each
learning stage by plotting the representations at each epoch. This fact also
makes OLGA a method more interpretive. In addition, we can visualize the
circle or sphere in the classification step.

##  4 Experimental Evaluation

This section presents the experimental evaluation of this article. We present
the used datasets, experimental settings, results, and discussion. Our goal is
to demonstrate that our OLGA proposal outperforms other state-of-the-art
methods. Another goal is to demonstrate that our method learns low-dimensional
representations without losing classification performance and performing
better than other methods. The experimental evaluation codes are publicly
available111https://github.com/GoloMarcos/JMLR-OLGA.

###  4.1 Datasets

We used one-class datasets from the literature from different domains,
sources, and data types and used the $k$-nn modeling given the lack of
benchmarking homogeneous graph datasets for one-class learning. We explored
textual, image, and tabular datasets. We accurately choose one-class original
datasets, i.e., datasets that have an interest class. We explore two tabular
datasets collected from the UCI repository. The first is a dataset of musk and
non-musk molecules (Chapman & Jain, 1994; Dietterich et al., 1993). Our
interest class was musk molecules. The second is a malware detection dataset
(Tuandromd) (Borah et al., 2020). Our interest class was malware in the
Tuandromd dataset.

We also use three textual datasets. The first is a dataset about fake news
detection (de Souza et al., 2022; Gôlo et al., 2021a). This dataset has real
and fake classes, and our interest class was fake news. The dataset is called
Fact Checked News (FCN) and is publicly available. The second is a dataset for
detecting interest events (Gôlo et al., 2021b). This dataset has terrorism and
non-terrorism classes, and our interest class was terrorism events. This
dataset is also publicly available. The third textual dataset is for detecting
relevant software reviews (Stanik et al., 2019; Gôlo et al., 2022). Our
interest class was relevant reviews, while irrelevant reviews were considered
outliers. The dataset is called App Reviews in English (ARE) and is not
publicly available. However, the dataset creators (Stanik et al., 2019) make
the datasets available for academic purposes upon request.

In addition, we explore three image datasets. The first dataset was collected
from the Kaggle platform (Antonov, 2019). This dataset is a dataset with food
images and non-food images. We use food images as our interest class. We also
collected from Kaggle the second dataset with images of lungs with and without
pneumonia (Kermany et al., 2018). We used pneumonia images as an interest
class. Finally, we used a dataset with images of healthy and abnormal
strawberries (Choi et al., 2022). Healthy strawberries are our interest class.

We model the datasets using the similarity between nearest neighbors. In the
graph, each node is connected with its $k$ nearest neighbors in this modeling.
We use the values of $k=\\{1,2,3\\}$. The modeling that generated the best
result in the validation set was used in the test set. The graph nodes must be
represented to calculate the similarity between nodes. Tabular datasets have a
natural and initial representation. However, text and image datasets are
naturally unstructured.

For the text datasets, we use the pre-trained model Bidirectional Encoder
Representations from Transformers (BERT) in the Multilingual version (Reimers
& Gurevych, 2019) to represent the textual nodes since the use of BERT to
initialize the textual representations nodes in GNNs presented good
performances (Huang et al., 2022a). We used the Contrastive Language Image
Pre-training (Radford et al., 2021) pre-trained model for the image datasets
to represent the image nodes. In addition to using the initial representations
to model our graphs, we also use these initial representations as input to
methods based on OCGNNs, the unsupervised GAE, and our OLGA approach.

###  4.2 Experimental Settings

We used three two-step baselines. In the first step, we generate the
representations through an unsupervised graph-based method. In the second, we
use the OCL algorithm to classify the instances based on the generated
representations. As a representation method, we used DeepWalk (Perozzi et al.,
2014), Node2Vec (Grover & Leskovec, 2016), and Graph Autoencoder (GAE) (Kipf &
Welling, 2016). We used the One-Class Support Vector Machines (OCSVM)
(Schölkopf et al., 2001) to classify the representations. We also compare our
method with three state-of-the-art end-to-end algorithms based on the OCGNN
proposed by (Wang et al., 2021). We use OCGCN, OCGAT, and OCSAGE, three OCGNN
variations that consider the GCN (Kipf & Welling, 2017), the GAT (Velickovic
et al., 2017), and the GraphSAGE (Hamilton et al., 2017) as layers of the GNN.

We use the $10$-fold cross-validation adapted for OCL. We divided only the
interest instances into 10 folds in the procedure since we used only the
interest class to train. We separate $1$ fold for the test and $9$ folds for
the training. Furthermore, all folders are used once as a test set. We add 50%
of non-interest instances to the test set. For the validation set of each
iteration, we use 10% of the training set and the other 50% of the non-
interest instances. We use the $f_{1}$-macro to compare all models, as the
$f_{1}$-macro is not biased by imbalance which is natural in OCL.

###  4.3 Results and Discussion

Tables 1 and 2 present the results for the methods in the eight datasets.
Table 1 presents the compared methods with embeddings of dimensions 128 (high)
and OLGA with 2 or 3 (low). Table 2 presents OLGA and the other methods with 2
and 3 dimensions (low). Bold values indicate the best results. Underline
values indicate second-best results. Our OLGA approach outperforms the other
methods. OLGA obtained higher values of $f_{1}$-macro in four out of eight
datasets considering Table 1 and five out of eight for Table 2. The most
competitive method with OLGA was DeepWalk, which obtained better results in
two datasets in the high embeddings scenario and two for low dimensions. OCGAT
and GAE outperform the other methods in the other three datasets for high and
low dimensions. OCGCN, OCSAGE, and GAE obtained the worst results in most
cases.

Table 1: $10$-fold average for $f_{1}$-macro in the test set for high dimension results. Each line represents a dataset, and each column a method. Datasets | GAE | Deepwalk | Node2Vec | OCGCN | OCGAT | OCSAGE | OLGA
---|---|---|---|---|---|---|---
Fakenews | 0.942 | 0.879 | 0.829 | 0.746 | 0.884 | 0.824 | 0.940
Terrorism | 0.727 | 0.981 | 0.800 | 0.845 | 0.900 | 0.785 | 0.921
Relevant R. | 0.728 | 0.598 | 0.644 | 0.592 | 0.691 | 0.623 | 0.750
Food | 0.982 | 0.752 | 0.960 | 0.901 | 0.994 | 0.985 | 0.997
Pneumonia | 0.639 | 0.878 | 0.645 | 0.505 | 0.752 | 0.655 | 0.914
Strawberry | 0.516 | 0.961 | 0.584 | 0.435 | 0.629 | 0.635 | 0.647
Musk | 0.623 | 0.774 | 0.640 | 0.666 | 0.620 | 0.603 | 0.785
Tuandromd | 0.904 | 0.689 | 0.905 | 0.898 | 0.978 | 0.974 | 0.974
Table 2: $10$-fold average for $f_{1}$-macro in the test set for low dimension results. Each line represents a dataset, and each column a method. Datasets | GAE | Deepwalk | Node2Vec | OCGCN | OCGAT | OCSAGE | OLGA
---|---|---|---|---|---|---|---
Fakenews | 0.950 | 0.868 | 0.892 | 0.635 | 0.506 | 0.486 | 0.940
Terrorism | 0.624 | 0.978 | 0.671 | 0.824 | 0.744 | 0.640 | 0.921
Relevant R. | 0.742 | 0.614 | 0.707 | 0.546 | 0.541 | 0.521 | 0.750
food | 0.995 | 0.768 | 0.980 | 0.787 | 0.635 | 0.526 | 0.997
Pneumonia | 0.394 | 0.878 | 0.787 | 0.588 | 0.664 | 0.661 | 0.914
Strawberry | 0.531 | 0.942 | 0.544 | 0.594 | 0.598 | 0.557 | 0.647
Musk | 0.478 | 0.726 | 0.526 | 0.505 | 0.437 | 0.477 | 0.785
Tuandromd | 0.863 | 0.689 | 0.823 | 0.756 | 0.598 | 0.854 | 0.974

OLGA outperforms the other methods in the two tabular datasets in Table 2 and
in the Musk dataset in Table 1. Furthermore, OLGA also outperforms the other
methods in datasets with textual content, obtaining higher $f_{1}$-macro in
the Relevant Reviews dataset in Table 1 and 2. OLGA was able to detect
relevant reviews satisfactorily. In the dataset with texts on terrorism and
non-terrorism, Deepwalk + OCSVM outperformed OLGA. In the dataset with texts
on fake and real news, GAE + OCSVM outperformed OLGA. Still, in the image
datasets, OLGA obtains higher $f_{1}$-macro in two out of three datasets. We
detected images of food and pneumonia satisfactorily. In the dataset with
strawberry images, Deepwalk + OCSVM outperformed OLGA.

In the two datasets in which we were competitive with Deepwalk + OCSVM, we
observed a not natural imbalance, i.e., we have more interest instances in the
test than non-interest (see Table 3). This imbalance is not natural in the
real world since the interest class is usually a small sample of a larger
universe. Also, one difference between OLGA and OCSVM is that OCSVM is
inductive while OLGA is transductive. Therefore, our transductive method was
not robust enough for this unbalanced scenario, while the OCSVM was not harmed
because it is inductive.

Table 1 presents OLGA’s performance considering low dimensions compared with
the results from other methods with high dimensions. OLGA obtains better
performances in four datasets (food, musk, pneumonia, and relevant reviews)
with the advantage of being interpretable, explainable, and with the power of
visualization. OLGA has competitive results in the other four datasets since
OLGA obtains the second-best results when other methods outperform OLGA. We
observe in table 2 that when another method obtains the highest $f_{1}$, OLGA
obtains the second best result, i.e., compared to methods with high or low
dimensionality, OLGA is competitive.

We performed Friedman’s statistical test with Nemenyi’s post-test (Trawinski
et al., 2012) to compare the methods considering the low and high scenarios.
Figure 3 presents a critical difference diagram generated through the Friedman
test with Nemenyi’s post-test result. The diagram presents the methods’
average rankings. Methods connected by a line do not present statistically
significant differences between them with 95% of confidence. The OLGA has the
best average ranking with a statistically significant difference from all
methods except Deepwalk. The Deepwalk obtained the second-best average ranking
without statistically significant differences from the other methods. OCGCN
and OCSAGE obtained the worst average rankings.

Figure 3: Critical difference diagram of Friedman’s statistical test with
Nemenyi post-test considering $f_{1}$-macro for low and high dimensional
scenarios.

We chose a dataset of each data type, textual, image, and tabular, to present
the representations generated by the OLGA learning. In this sense, Figure 4
presents the OLGA representations in the experimental evaluation on the
relevant review, food, and tuandromd datasets. We present OLGA’s learning
process, i.e., four learning epochs at different stages. The first stage is
epoch $0$. The second is epoch $150$. We chose $150$ because the patience for
this analysis was $300$. The fourth stage is the epoch when the model
converges, and the third stage is the average epoch between the second and
fourth stages. Blue points represent the interest class, and green points the
non-interest class.

Figure 4: Two-dimensional representations of OLGA last layer consider the
learned representations in three datasets. The colors indicate the interest
class (blue) and the non-interest class (green).

We observe the entire learning process through representations of our method,
shown in Figure 4. During stages one and two, we observe the method ignoring
the hypersphere and focusing only on graph reconstruction (loss
$\mathcal{L}_{2}$), which really is the OLGA goal. In the third stage, we
observe multi-task learning since the interest instances get closer to the
hypersphere while non-interest instances are outside the hypersphere (losses
$\mathcal{L}_{1}$ and $\mathcal{L}_{3}$). Finally, in the fourth stage, we
observed that the one-class loss $\mathcal{L}_{1}$ encouraged the instances to
continue coming to the hypersphere center, as proposed.

OLGA learns non-agnostic and customized representations for OCL, as shown in
Figure 4. OLGA obtains the best performances, as shown above, with a
representation with less overlap between classes, a one-class visualization
more promising, and an interpretable representation learning model. We
emphasize that combining OLGA loss functions in a multi-task way and using low
dimensions allowed our proposal to be interpretable, explicable, and used for
visualization.

##  5 Conclusions and Future Work

We propose OLGA, an end-to-end graph neural network for OCL. OLGA combines a
hypersphere and reconstruction loss functions. We also introduce a novel
hypersphere loss function that encapsulates the interest instances and
encourages these instances to approach the center even within the hypersphere.
The learning process of OLGA allows us to explore low-dimensional
representations during the classification process without harming the
classification performance, providing interpretability and visualization. Our
approach outperforms other state-of-the-art methods, showing statistically
significant differences from five out of six compared methods, and generates
visually meaningful representations suitable for OCL.

## References

  * Alam et al. (2020) Shamshe Alam, Sanjay Kumar Sonbhadra, Sonali Agarwal, and P Nagabhushan.  One-class support vector classifiers: A survey.  _Knowledge-Based Systems_ , 2020.
  * Antonov (2019) Aleksandr Antonov.  Food-5k image dataset.  Kaggle Repository, 2019.  URL: https://www.kaggle.com/datasets/trolukovich/food5k-image-dataset.
  * Borah et al. (2020) Parthajit Borah, DK Bhattacharyya, and JK Kalita.  Malware dataset generation and evaluation.  In _CICT_ , 2020.
  * Chapman & Jain (1994) David Chapman and Ajay Jain.  Musk (Version 2).  UCI Machine Learning Repository, 1994.  DOI: https://doi.org/10.24432/C51608.
  * Choi et al. (2022) Taeyeong Choi, Owen Would, Adrian Salazar-Gomez, and Grzegorz Cielniak.  Self-supervised representation learning for reliable robotic monitoring of fruit anomalies.  In _ICRA_. IEEE, 2022.
  * da Silva et al. (2022) Angelo da Silva, Marcos Gôlo, and Ricardo Marcacini.  Unsupervised heterogeneous graph neural network for hit song prediction through one class learning.  In _KDMiLe_. SBC, 2022.
  * de Souza et al. (2022) Mariana Caravanti de Souza, Bruno Magalhães Nogueira, Rafael Geraldeli Rossi, Ricardo Marcondes Marcacini, Brucce Neves Dos Santos, and Solange Oliveira Rezende.  A network-based positive and unlabeled learning approach for fake news detection.  _Machine Learning_ , 2022.
  * Deldar et al. (2022) Fatemeh Deldar, Mahdi Abadi, and Mohammad Ebrahimifard.  Android malware detection using one-class graph neural networks.  _ISeCure_ , 2022.
  * Deng et al. (2022) Songgaojun Deng, Huzefa Rangwala, and Yue Ning.  Causality enhanced societal event forecasting with heterogeneous graph learning.  In _ICDM_. IEEE, 2022.
  * Dietterich et al. (1993) Thomas Dietterich, Ajay Jain, Richard Lathrop, and Tomas Lozano-Perez.  A comparison of dynamic reposing and tangent distance for drug activity prediction.  _NeurIPS_ , 1993.
  * Dou et al. (2020) Yingtong Dou, Zhiwei Liu, Li Sun, Yutong Deng, Hao Peng, and Philip S Yu.  Enhancing graph neural network-based fraud detectors against camouflaged fraudsters.  In _Int. Conf. on Information & Knowledge Management_, 2020.
  * Emmert-Streib & Dehmer (2022) Frank Emmert-Streib and Matthias Dehmer.  Taxonomy of machine learning paradigms: A data-centric perspective.  _WIREs Data Mining and Knowledge Discovery_ , 2022.
  * Feng et al. (2022) Yong Feng, Jinglong Chen, Zijun Liu, Haixin Lv, and Jun Wang.  Full graph autoencoder for one-class group anomaly detection of iiot system.  _IoT Journal_ , 2022.
  * Fernández et al. (2018) Alberto Fernández, Salvador García, Mikel Galar, Ronaldo C Prati, Bartosz Krawczyk, and Francisco Herrera.  _Learning from imbalanced data sets_.  Springer, 2018.
  * Ganz et al. (2023) Tom Ganz, Inaam Ashraf, Martin Härterich, and Konrad Rieck.  Detecting backdoors in collaboration graphs of software repositories.  In _CODASPY_ , 2023.
  * Gôlo et al. (2021a) Marcos Gôlo, Mariana Caravanti, Rafael Rossi, Solange Rezende, Bruno Nogueira, and Ricardo Marcacini.  Learning textual representations from multiple modalities to detect fake news through one-class learning.  In _Webmedia_ , 2021a.
  * Gôlo et al. (2021b) Marcos PS Gôlo, Rafael G Rossi, and Ricardo M Marcacini.  Triple-vae: A triple variational autoencoder to represent events in one-class event detection.  In _ENIAC_. SBC, 2021b.
  * Gôlo et al. (2022) Marcos PS Gôlo, Adailton F Araújo, Rafael G Rossi, and Ricardo M Marcacini.  Detecting relevant app reviews for software evolution and maintenance through multimodal one-class learning.  _IST_ , 2022.
  * Grover & Leskovec (2016) Aditya Grover and Jure Leskovec.  node2vec: Scalable feature learning for networks.  In _SIGKDD_ , 2016.
  * Gôlo et al. (2022) Marcos Gôlo, Leonardo Moraes, Rudinei Goularte, and Ricardo Marcacini.  One-class recommendation through unsupervised graph neural networks for link prediction.  In _KDMiLe_. SBC, 2022.
  * Hamilton et al. (2017) Will Hamilton, Zhitao Ying, and Jure Leskovec.  Inductive representation learning on large graphs.  In _NeurIPS_ , 2017.
  * Huang et al. (2021) Bin Huang, Xuhong Wang, Ping Cui, Wenjian Jiang, Yupu Yang, and Qifu Fan.  One-class temporal graph attention neural network for dynamic graph anomaly detection.  In _CECIT_. IEEE, 2021.
  * Huang et al. (2022a) Yen-Hao Huang, Yi-Hsin Chen, and Yi-Shin Chen.  Contexting: Granting document-wise contextual embeddings to graph neural networks for inductive text classification.  In _Int. Conf. Computational Linguistics_ , 2022a.
  * Huang et al. (2022b) Zeqi Huang, Yonghao Gu, and Qing Zhao.  One-class directed heterogeneous graph neural network for intrusion detection.  In _ICIAI_ , 2022b.
  * Kermany et al. (2018) Daniel S Kermany, Michael Goldbaum, Wenjia Cai, Carolina CS Valentim, Huiying Liang, Sally L Baxter, Alex McKeown, Ge Yang, Xiaokang Wu, Fangbing Yan, et al.  Identifying medical diagnoses and treatable diseases by image-based deep learning.  _cell_ , 2018.
  * Khan & Madden (2014) Shehroz S Khan and Michael G Madden.  One-class classification: taxonomy of study and review of techniques.  _Knowledge Engineering Review_ , 2014.
  * Kipf & Welling (2016) Thomas N Kipf and Max Welling.  Variational graph auto-encoders.  _stat_ , 2016.
  * Kipf & Welling (2017) Thomas N Kipf and Max Welling.  Semi-supervised classification with graph convolutional networks.  In _ICLR_ , 2017.
  * Liu et al. (2021) Bo Liu, Haoxin Xie, and Yanshan Xiao.  Multi-task analysis discriminative dictionary learning for one-class learning.  _Knowledge-Based Systems_ , 2021.
  * Liu et al. (2022a) Fanzhen Liu, Xiaoxiao Ma, Jia Wu, Jian Yang, Shan Xue, Amin Beheshti, Chuan Zhou, Hao Peng, Quan Z Sheng, and Charu C Aggarwal.  Dagad: Data augmentation for graph anomaly detection.  In _ICDM_. IEEE, 2022a.
  * Liu et al. (2022b) Zipeng Liu, Yang Wang, Jürgen Bernard, and Tamara Munzner.  Visualizing graph neural networks with corgie: Corresponding a graph to its embedding.  _IEEE Transactions on Visualization and Computer Graphics_ , 2022b.
  * Ma & Mei (2019) Jiaqi Ma and Qiaozhu Mei.  Graph representation learning via multi-task knowledge distillation.  In _NeurIPS_ , 2019.
  * Nguyen & Grishman (2018) Thien Nguyen and Ralph Grishman.  Graph convolutional networks with argument-aware pooling for event detection.  In _AAAI_ , 2018.
  * Perozzi et al. (2014) Bryan Perozzi, Rami Al-Rfou, and Steven Skiena.  Deepwalk: Online learning of social representations.  In _SIGKDD_ , 2014.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _ICML_ , 2021.
  * Reimers & Gurevych (2019) Nils Reimers and Iryna Gurevych.  Sentence-bert: Sentence embeddings using siamese bert-networks.  In _EMNLP_. ACL, 2019.
  * Ruff et al. (2018) Lukas Ruff, Robert Vandermeulen, Nico Goernitz, Lucas Deecke, Shoaib Ahmed Siddiqui, Alexander Binder, Emmanuel Müller, and Marius Kloft.  Deep one-class classification.  In _ICML_ , 2018.
  * Sami et al. (2022) Mirza Tanzim Sami, Da Yan, Bhadhan Roy Joy, Jalal Khalil, Ricardo Cevallos, Md Emon Hossain, Kejin Hu, and Yang Zhou.  Center-based ipsc colony counting with multi-task learning.  In _ICDM_. IEEE, 2022.
  * Schölkopf et al. (2001) Bernhard Schölkopf, John C Platt, John Shawe-Taylor, Alex J Smola, and Robert C Williamson.  Estimating the support of a high-dimensional distribution.  _Neural computation_ , 2001.
  * Smith & Vamanamurthy (1989) David J Smith and Mavina K Vamanamurthy.  How small is a unit ball?  _Mathematics Magazine_ , 62(2):101–107, 1989\.
  * Stanik et al. (2019) Christoph Stanik, Marlo Haering, and Walid Maalej.  Classifying multilingual user feedback using traditional machine learning and deep learning.  In _REW_. IEEE, 2019.
  * Tax (2001) David Martinus Johannes Tax.  _One-class classification: Concept learning in the absence of counter-examples_.  PhD thesis, Universiteit Delft, 2001.
  * Tax & Duin (2004) David MJ Tax and Robert PW Duin.  Support vector data description.  _Machine learning_ , 2004.
  * Trawinski et al. (2012) Bogdan Trawinski, Magdalena Smetek, Zbigniew Telec, and Tadeusz Lasota.  Nonparametric statistical analysis for multiple comparison of machine learning regression algorithms.  _Applied Mathematics and Computer Science_ , 2012.
  * Velickovic et al. (2017) Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio.  Graph attention networks.  _stat_ , 2017.
  * Wang et al. (2021) Xuhong Wang, Baihong Jin, Ying Du, Ping Cui, Yingshui Tan, and Yupu Yang.  One-class graph neural networks for anomaly detection in attributed networks.  _Neural computing and applications_ , 2021.
  * Wu & Mooney (2019) Jialin Wu and Raymond Mooney.  Faithful multimodal explanation for visual question answering.  In _Workshop BlackboxNLP_ , 2019.
  * Xia et al. (2021) Feng Xia, Ke Sun, Shuo Yu, Abdul Aziz, Liangtian Wan, Shirui Pan, and Huan Liu.  Graph learning: A survey.  _IEEE Transactions on Artificial Intelligence_ , 2021.
  * Xie et al. (2020) Yu Xie, Maoguo Gong, Yuan Gao, AK Qin, and Xiaolong Fan.  A multi-task representation learning architecture for enhanced graph classification.  _Frontiers in neuroscience_ , 2020.
  * Xue & Beauseroy (2016) Yongjian Xue and Pierre Beauseroy.  Multi-task learning for one-class svm with additional new features.  In _ICPR_. IEEE, 2016.
  * Zhang & Yang (2018) Yu Zhang and Qiang Yang.  An overview of multi-task learning.  _National Science Review_ , 2018.
  * Zhang & Yang (2021) Yu Zhang and Qiang Yang.  A survey on multi-task learning.  _IEEE Transactions on Knowledge and Data Engineering_ , 2021.
  * Zhou et al. (2020) Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun.  Graph neural networks: A review of methods and applications.  _AI Open_ , 2020.

##  Appendix A Appendix

###  A.1 Experimental Evaluation Details

We use the following parameters for the methods:

  * •

DeepWalk and Node2Vec: window size = $10$, embedding size = $\\{2,3,128\\}$,
walks per node = $15$, walk length = $10$, q=$1$ and p=$4$;

  * •

GAE: epochs=$100$, learning rate=$0.01$, optimizer=$Adam$, embedding size =
$\\{2,3,128\\}$;

  * •

OCSVM: degree for polynomial kernel =$\\{2\\}$, $\gamma=\\{scale,auto\\}$,
$\nu=\\{0.05*\iota,0.005*\iota;\iota\in[1..17]\\}$, and
$kernels=\\{rbf,linear,sigmoid,polynomial\\}$;

  * •

OCGCN, OCGAT, OCSAGE and OLGA: epoch = $5000$, patience = $\\{300,500\\}$,
learning rate= $\\{0.0001,0.0005\\}$, and optimizer=$Adam$;

  * •

OCGCN, OCGAT, and OCSAGE: $\nu=\\{0.001,0.01,0.1,0.2,0.3,0.4,0.5\\}$, weight
decay = $0.0005$, dropout = $0.5$, and activation = {ReLU}, embedding size =
$\\{2,3,128\\}$;

  * •

OLGA: embedding size = $\\{2,3\\}$, center = $0$, radius =
$\\{0.3,0.35,0.4\\}$, and activation = {TanH}. Until the patience/2 epoch, we
use $\alpha$ = 0, $\beta$ = 1, and $\delta$ = 1. After the patience/2 epoch,
we use $\alpha$ = 1, $\beta$ = 0, and $\delta$ = 1.

Table 3 presents the details of the datasets such as dataset type, number of
interest instances (#I), number of non-interest instances (#NI), number of
interest instances for training (#Tr), number of interest instances for the
test (#TeI), and number of non-interest instances for the test (#TeNI). We
obtain these values according to the 10-fold cross-validation.

Table 3: dataset properties. Datasets | Type | #I | #NI | #Tr | #TeI | #TeNI
---|---|---|---|---|---|---
FakeNews | Text | 1044 | 1020 | 940 | 104 | 510
Terrorism | Text | 5926 | 721 | 5333 | 593 | 360
Relevant R. | Text | 2537 | 3855 | 2283 | 254 | 1927
Food | Image | 2500 | 2500 | 2250 | 250 | 1025
Pneumonia | Image | 4273 | 1583 | 3846 | 427 | 791
Strawberry | Image | 3367 | 153 | 3030 | 337 | 76
Musk | Tabular | 1224 | 5850 | 1102 | 122 | 2925
Tuandromd | Tabular | 3565 | 899 | 3209 | 356 | 449

###  A.2 Dimension analysis for our hypersphere loss function

We show the hypersphere volume for the OLGA loss function considering the
values of the radius used (0.3, 0.45, 0.5). We show the volume considering
dimensions from 1 to 40. Figure 5 shows the analysis result. The green line
represents the 0.5 radius, the orange represents the 0.45, and the blue
represents the 0.3. The axis y represents the volume, and the axis x
represents the dimension. We use a red line to indicate the volume obtained
with our dimensions numbers. We note that with dimensions higher than 15, the
volume tends to be 0 regardless of radius. With smaller dimensions, we observe
that the volume does not tend to be 0, allowing OLGA to better represent
learning.

The volume was generated by:

$V_{n}(r)=\frac{\pi^{n/2}}{\Gamma(\frac{n}{2}+1)}r^{n},$ (11)

in which, $r$ is the radius, $n$ is number of dimension and $\Gamma$ is the
Euler’s gamma function.

Figure 5: Relations between the hypersphere volume and the dimension of the
representation used.

###  A.3 Results with standard deviation and complete 2D plot

Table 4: Averages and standard deviation for $f_{1}$-macro in the test set for high dimension results. Each line represents a dataset, and each column a method. Bold values indicate the best results. Datasets | GAE | Deepwalk | Node2Vec | OCGCN | OCGAT | OCSAGE | OLGA
---|---|---|---|---|---|---|---
Fakenews | 0.942 (0.011) | 0.879 (0.016) | 0.829 (0.022) | 0.746 (0.045) | 0.884 (0.017) | 0.824 (0.015 | 0.940 (0.016)
Terrorism | 0.727 (0.038) | 0.981(0.003) | 0.800 (0.015) | 0.845 (0.021) | 0.900 (0.007) | 0.785 (0.020) | 0.921 (0.007)
Relevant R. | 0.728 (0.009) | 0.598 (0.023) | 0.644 (0.009) | 0.592 (0.010) | 0.691 (0.006) | 0.623 (0.010) | 0.750 (0.007)
Food | 0.982 (0.004) | 0.752 (0.012) | 0.960 (0.007) | 0.901 (0.009) | 0.994 (0.003) | 0.985 (0.003) | 0.997 (0.002)
Pneumonia | 0.639 (0.012) | 0.878 (0.010) | 0.645 (0.012) | 0.505 (0.012) | 0.752 (0.011) | 0.655 (0.014) | 0.914 (0.010)
Strawberry | 0.516 (0.021) | 0.961 (0.007) | 0.584 (0.031) | 0.435 (0.093) | 0.629 (0.020) | 0.635 (0.025) | 0.647 (0.101)
Musk | 0.623 (0.063) | 0.774 (0.006) | 0.640 (0.019) | 0.666 (0.018) | 0.620 (0.008) | 0.603 (0.013) | 0.785 (0.029)
Tuandromd | 0.904 (0.011) | 0.689 (0.007) | 0.905 (0.007) | 0.898 (0.008) | 0.978 (0.004) | 0.974 (0.004) | 0.974 (0.006)
Table 5: Averages and standard deviation for $f_{1}$-macro in the test set for low dimension results. Each line represents a dataset, and each column a method. Bold values indicate the best results. Datasets | GAE | Deepwalk | Node2Vec | OCGCN | OCGAT | OCSAGE | OLGA
---|---|---|---|---|---|---|---
Fakenews | 0.950 (0.010) | 0.868 (0.018) | 0.892 (0.015) | 0.635 (0.014) | 0.506 (0.013) | 0.486 (0.022) | 0.940 (0.016)
Terrorism | 0.624 (0.011) | 0.978 (0.004) | 0.671 (0.014) | 0.824 (0.012) | 0.744 (0.012) | 0.640 (0.010) | 0.921 (0.007)
Relevant R. | 0.742 (0.009) | 0.614 (0.017) | 0.707 (0.008) | 0.546 (0.007) | 0.541 (0.005) | 0.521 (0.008) | 0.750 (0.007)
food | 0.995 (0.003) | 0.768 (0.012) | 0.980 (0.003) | 0.787 (0.016) | 0.635 (0.015) | 0.526 (0.012) | 0.997 (0.002)
Pneumonia | 0.394 (0.000) | 0.878 (0.009) | 0.787 (0.010) | 0.588 (0.010) | 0.664 (0.013) | 0.661 (0.006) | 0.914 (0.010)
Strawberry | 0.531 (0.017) | 0.942 (0.013) | 0.544 (0.022) | 0.594 (0.023) | 0.598 (0.015) | 0.557 (0.013) | 0.647 (0.101)
Musk | 0.478 (0.068) | 0.726 (0.015) | 0.526 (0.013) | 0.505 (0.008) | 0.437 (0.007) | 0.477 (0.011) | 0.785 (0.029)
Tuandromd | 0.863 (0.020) | 0.689 (0.009) | 0.823 (0.012) | 0.756 (0.011) | 0.598 (0.016) | 0.854 (0.012) | 0.974 (0.006)
Figure 6: Two-dimensional representations of each method consider the learned
representations in three datasets. The colors indicate the interest class
(blue) and the non-interest class (green). Representation methods that show
less overlap between classes and a one-class visualization are more promising.

Figure 6 shows that the two-step methods generated non-customized
representations for one-class learning. Even GAE generates some good
representations, which partially separate the interest and non-interest
classes in most cases, these representations are not customized because they
are agnostic to the classification model. End-to-end methods based on OCGNN
generate representations non-agnostic to the classification model. However,
OCGNN does not customize the representations enough for a hypersphere to
classify interest and non-interest instances. OLGA also learns non-agnostic
representations such as OCGNNs. On the other hand, OLGA generates customized
representations for one-class learning, obtaining the best performances, as
shown above.


# Simultaneous Data Communication and Channel Estimation in Multi-User Full
Duplex MIMO Systems

Md Atiqul Islam2, George C. Alexandropoulos4, and Besma Smida2  2Department of
Electrical and Computer Engineering, University of Illinois at Chicago, USA
4Department of Informatics and Telecommunications, National and Kapodistrian
University of Athens, Greece
emails<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

In this paper, we study Simultaneous Communication of Data and Control (SCDC)
information signals in Full Duplex (FD) Multiple-Input Multiple-Output (MIMO)
wireless systems. In particular, considering a FD MIMO base station serving
multiple single-antenna FD users, a novel multi-user communication scheme for
simultaneous DownLink (DL) beamformed data transmission and UpLink (UL) pilot-
assisted channel estimation is presented. Capitalizing on a recent FD MIMO
hardware architecture with reduced complexity self-interference analog
cancellation, we jointly design the base station’s transmit and receive
beamforming matrices as well as the settings for the multiple analog taps and
the digital SI canceller with the objective to maximize the DL sum rate. Our
simulation results showcase that the proposed approach outperforms its
conventional half duplex counterpart with $50\%$ reduction in hardware
complexity compared to the latest FD-based SCDC schemes.

###### Index Terms:

Full duplex, multi-user MIMO, joint communication and control, channel
estimation, optimization.

##  I Introduction

Multi-user Full Duplex (FD) Multiple-Input Multiple-Output (MIMO)
communication technology has the potential of substantial spectral efficiency
improvement and simplification of the control information exchange over
conventional frequency- and time-division duplexing systems through concurrent
UpLink (UL) and DownLink (DL) communication in the same frequency and time
resources [1, 2, 3, 4, 5, 6, 7, 8, 9].

The main bottleneck of FD MIMO systems is the in-band Self Interference (SI)
signal at the reception side resulting from simultaneous transmission and
reception, as well as limited Transmitter (TX) and Receiver (RX) isolation.
Therefore, a combination of propagation domain isolation, analog domain
suppression, and digital SI cancellation techniques are adopted in practice to
suppress the strong SI below the noise floor. In FD MIMO systems, the
suppression techniques are particularly challenging due to higher SI
components, as a consequence of the increased number of transceiver antennas.
Analog SI cancellation in FD MIMO systems can be implemented by subtracting a
processed copy of every TX signal from every RX input to avoid saturation of
the RXs’ Radio Frequency (RF) chain [1]. However, the hardware requirements
for such an approach scale with the number of TX/RX antennas, rendering the
implementation of analog SI a core design bottleneck. In [8, 9], authors
presented spatial suppression techniques that alleviate the need of analog SI
cancellation, relying solely on digital TX/RX beamforming. In [10], a joint
design of multi-tap analog cancellation and TX/RX beamforming, where the
number of taps does not scale with the product of TX and RX antenna elements,
was proposed. According to this work, every analog tap refers to a line of
fixed delay, variable phase shifter, and attenuator. The authors in [7]
presented a unified beamforming approach including both Analog and Digital
(A/D) SI cancellation that further improves the achievable rate performance
under practical transceiver imperfections.

Recently, in [11, 12, 13], simultaneous DL data transmission and UL Channel
State Information (CSI) reception has been considered at an FD MIMO Base
Station (BS) serving multiple Half Duplex (HD) User Equipment (UE) nodes. In
these studies, the UEs transmit training symbols through the UL channel in a
Time Division Multiple Access (TDMA) manner, which are utilized by the BS to
estimate the DL channels leveraging channel reciprocity, while at the same
time transmitting the DL payload to UEs for whom the DL CSI is already
available. However, the adopted multi-user models consider perfect analog
cancellation that is based on conventional FD MIMO architectures with fully
connected analog cancellation, interconnecting all TX antenna elements in the
FD node with all its RX antennas. In [14], a low complexity solution of
simultaneous DL data transmission and UL CSI acquisition was proposed for a
single user FD MIMO system.

Figure 1: The considered multi-user FD MIMO system model for simultaneous DL
data transmission and UL pilot-assisted channel estimation. BS node $b$ and
all UE nodes deploy A/D SI cancellation with the former node $b$ realizing
[7]’s reduced complexity multi-tap analog cancellation.

In this paper, we present a multi-user MIMO communication system for
Simultaneous Communication of Data and Control (SCDC) information signals,
capitalizing on the FD MIMO architecture of [7] combining TX beamforming with
A/D SI cancellation. Exploiting channel reciprocity and relying on FD
operation, the proposed system performs joint digital TX beamforming for DL
communication and UL pilot-assisted CSI estimation. Considering realistic
modeling for imperfect channel estimation and CSI delay error, we present a
joint optimization framework for the DL rate optimization and the accurate CSI
estimation. Our simulation results showcase superior achievable DL rate
performance for the proposed SCDC scheme with more than $50\%$ reduction in
the analog cancellation hardware complexity compared to the conventional FD
MIMO architectures with fully connected analog cancellation.

Notation: Vectors and matrices are denoted by boldface lowercase and boldface
capital letters, respectively. The transpose, Hermitian transpose, and
conjugate of $\mathbf{A}$ are denoted by $\mathbf{A}^{\rm T}$,
$\mathbf{A}^{\rm H}$, and $\mathbf{A}^{*}$, respectively, and
$\det(\mathbf{A})$ is $\mathbf{A}$’s determinant, while $\mathbf{0}_{m\times
n}$ ($m\geq 2$ and $n\geq 1$) represents the $m\times n$ matrix with all
zeros. $\|\mathbf{a}\|$ stands for the Euclidean norm of $\mathbf{a}$.
$[\mathbf{A}]_{i,j}$, $[\mathbf{A}]_{(i,:)}$, and $[\mathbf{A}]_{(:,j)}$
represent $\mathbf{A}$’s $(i,j)$-th element, $i$-th row, and $j$-th column,
respectively, while $[\mathbf{a}]_{i}$ denotes the $i$-th element of
$\mathbf{a}$. $\text{sort}(\mathbf{A})$ represents a rearranged row formation
of $\mathbf{A}$, where the subsequent rows have decreasing Euclidean norm.
$\mathbb{C}$ represents the complex number set, $\mathbb{E}\\{\cdot\\}$ is the
expectation operator, and $|\cdot|$ denotes the amplitude of a complex number.
$x\sim\mathcal{CN}(0,\sigma^{2})$ represents a circularly symmetric complex
Gaussian random variable with zero mean and variance $\sigma^{2}$.

##  II System and Signal Models

A multi-user MIMO communication is considered with a BS node $b$ containing
$N_{b}$ TX and RX antennas, and $K$ single-antenna FD UE nodes, as shown in
Fig. 1. Each antenna is attached to a dedicated TX/RX RF chain at all the
nodes. A multi-tap analog SI canceller is applied in the FD BS node $b$,
whereas each UE node deploys a single-tap SI canceller. The FD BS node is
capable of performing digital TX beamforming realized, for simplicity, with
linear filters. We consider UL/DL channel reciprocity and focus on
simultaneous data communication and channel estimation, where the DL is
intended for information data communication while the UL is used for
transmitting training signals from UEs to the BS. The UL CSI estimation is
used for designing the DL precoder to digitally process the data signals
before transmission.

We assume that, for every channel use, the BS node $b$ transmits the complex-
valued information data symbols $\mathbf{s}_{b}\in\mathbb{C}^{m_{b}\times 1}$
(chosen from a discrete modulation set) using the unit norm digital precoding
vector $\mathbf{V}_{b}\in\mathbb{C}^{N_{b}\times m_{b}}$, where the $m_{b}$ is
the number of data streams. In the UL direction, the $K$ users simultaneously
send orthogonal training symbols to the BS node $b$, which can be denoted as
$\mathbf{s}_{K}\in\mathbb{C}^{K\times 1}$. The signal transmissions at both BS
and UE nodes are power limited to ${P}_{b}$ and ${P}_{k}$, respectively.
Specifically, the DL signal is such that
$\mathbb{E}\\{\|\mathbf{V}_{b}\mathbf{s}_{b}\|^{2}\\}\leq{P}_{b}$, whereas the
UL signal from all $K$ UEs is constrained as
$\mathbb{E}\\{\|\mathbf{s}_{K}\|^{2}\\}\leq{P}_{K}$.

###  II-A Channel Modeling

The Rayleigh faded DL channel from BS node $b$ to UE node $k$, denoted by
$\mathbf{h}_{k}^{\rm T}\in\mathbb{C}^{1\times N_{b}}$, is modeled as
Independent and Identically Distributed (IID)
$\mathcal{C}\mathcal{N}(0,l_{K})$, where $l_{K}$ is the UL pathloss. Stacking
all UE channel vectors, the DL channel for $K$ users can be expressed as
$\mathbf{H}=[\mathbf{h}_{1}\,\mathbf{h}_{2}\,\cdots\,\mathbf{h}_{K}]^{\rm
T}\in\mathbb{C}^{K\times N_{b}}$. Based on the UL/DL channel reciprocity, the
Rayleigh faded UL channel is $\mathbf{H}^{\rm T}\in\mathbb{C}^{N_{b}\times
K}$. As all nodes are capable of FD operation, the simultaneous DL data and UL
training transmission induce SI in the RXs of the BS and UEs, respectively. We
consider the Rician fading model for the SI channels denoted by
$\mathbf{H}_{b,b}\in\mathbb{C}^{N_{b}\times N_{b}}$ for the BS and
${h}_{k,k}\in\mathbb{C},\,\forall k=1,2,\ldots,K$, for the UEs with Rician
factor $\kappa$ and pathlosses $l_{b,b}$ and $l_{k,k}$ at nodes $b$ and $k$,
respectively [2]. Using matrix notation, the SI channel for $K$ users can be
expressed as
$\mathbf{H}_{K,K}=\text{diag}\\{{h}_{1,1},{h}_{2,2},\cdots,{h}_{K,K}\\}\in\mathbb{C}^{K\times
K}$. Similarly, the inter-node interference channel between $K$ users
containing only off-diagonal elements is denoted as
$\mathbf{H}_{\text{IN}}\in\mathbb{C}^{K\times
K}~{}\mathcal{C}\mathcal{N}(0,l_{\text{IN}})$, where $l_{\text{IN}}$ is the
inter-node interference channel pathloss.

###  II-B Signal Modeling at the BS and UE Nodes

Because of FD operation, BS node $b$ RXs receive the training symbols
transmitted from $K$ users as well as the SI signal induced by simultaneous
data transmission. As shown in Fig. 1, the SI is suppressed using low
complexity analog SI cancellation, which is followed by a digital canceller in
the baseband of BS node $b$. Denoting analog and digital SI cancellers as
$\mathbf{C}_{b}\in\mathbb{C}^{N_{b}\times N_{b}}$ and
$\mathbf{D}_{b}\in\mathbb{C}^{N_{b}\times N_{b}}$, respectively, the received
baseband signal at BS node $b$, $\mathbf{y}_{b}\in\mathbb{C}^{N_{b}\times 1}$,
can be expressed as

$\begin{split}\mathbf{y}_{b}&\triangleq\mathbf{H}^{\rm
T}\mathbf{s}_{K}+\left(\mathbf{H}_{b,b}+\mathbf{C}_{b}+\mathbf{D}_{b}\right)\mathbf{V}_{b}\mathbf{s}_{b}+\mathbf{n}_{b},\\\
&=\mathbf{H}^{\rm
T}\mathbf{s}_{K}+\widetilde{\mathbf{H}}_{b,b}\mathbf{V}_{b}\mathbf{s}_{b}+\mathbf{n}_{b},\end{split}$
(1)

where $\mathbf{n}_{b}\in\mathbb{C}^{N_{b}\times 1}$ is the zero-mean Additive
White Gaussian Noise (AWGN) with variance $\sigma^{2}_{b}\mathbf{I}_{N_{b}}$.
Here,
$\widetilde{\mathbf{H}}_{b,b}\triangleq\left(\mathbf{H}_{b,b}+\mathbf{C}_{b}+\mathbf{D}_{b}\right)$
represents the residual SI channel after A/D SI cancellation.

Similarly, after A/D SI cancellation at each UE node, the precoded DL signal
received at $K$ user nodes, $\mathbf{y}_{K}\in\mathbb{C}^{K\times 1}$ is
written as

$\displaystyle\mathbf{y}_{K}$
$\displaystyle\triangleq\mathbf{H}\mathbf{V}_{b}\mathbf{s}_{b}+\left(\mathbf{H}_{K,K}+\mathbf{C}_{K}+\mathbf{D}_{K}\right)\mathbf{s}_{K}+\mathbf{H}_{\text{IN}}\mathbf{s}_{K}+\mathbf{n}_{K},$
$\displaystyle=\mathbf{H}\mathbf{V}_{b}\mathbf{s}_{b}+\widetilde{\mathbf{H}}_{K,K}\mathbf{s}_{K}+\mathbf{H}_{\text{IN}}\mathbf{s}_{K}+\mathbf{n}_{K},$
(2)

where $\mathbf{n}_{K}\in\mathbb{C}^{K\times 1}$ is the zero-mean AWGN with
variance $\sigma^{2}_{K}\mathbf{I}_{K}$. Here,
$\mathbf{C}_{K}\triangleq\text{diag}\\{{c}_{1,1},{c}_{2,2},\cdots,{c}_{K,K}\\}\in\mathbb{C}^{K\times
K}$,
$\mathbf{D}_{K}\triangleq\text{diag}\\{{d}_{1,1},{d}_{2,2},\cdots,{d}_{K,K}\\}\in\mathbb{C}^{K\times
K}$, where the diagonal elements of $\mathbf{C}_{K}$ and $\mathbf{D}_{K}$
represent the analog and digital SI canceller of each UE node, and
$\widetilde{\mathbf{H}}_{K,K}\triangleq\left(\mathbf{H}_{K,K}+\mathbf{C}_{K}+\mathbf{D}_{K}\right)$
is the residual SI channel at all UE nodes.

##  III Proposed SCDC Scheme

In this section, we present a multi-user FD MIMO based data communication and
channel estimation scheme. First, we describe the CSI acquisition and DL
beamforming scheme in Section III-A. Then, in Section III-B, we characterize
the realistic channel estimation error, CSI delay error, and average
achievable DL rate.

###  III-A Proposed CSI Acquisition and Beamforming

The proposed SCDC scheme is illustrated in Fig. 2(a), where DL is dedicated
for data transmission and UL is accessed to transmit training symbols. We
assume a time division duplexing based data transmission, where the considered
channels remains constant for all the channel uses in a time slot and the
channels of successive time slots are temporally correlated[15, 16]. Each time
slot contains $T$ symbols (i.e. $T$ channel uses). The proposed data
transmission scheme and CSI acquisition scheme is described as follows:

  1. 1.

For data transmission at any $i$th time slot, first, we obtain the UL channel
estimate of $(i\\!-\\!1)$th time slot using $T$ orthogonal training symbols
sent from each UE, while maintaining simultaneous data transmission in the DL.

  2. 2.

Using UL/DL reciprocity, we obtain the DL channel estimate at $(i\\!-\\!1)$th
time slot from the UL channel estimate.

  3. 3.

Based on the DL channel estimate, we derive the DL precoder
$\mathbf{V}_{b}[i]$ to process all the data streams at the $i$th time slot.

In Fig. 2(b), the FD sequential beamforming approach of [12] is depicted,
where the UEs access the UL channel in TDMA manner, and the BS initiates DL
data transmission to the UEs with available CSI, instead of waiting for all
the UEs to finish training. Figure 2(c) represents the conventional HD MIMO
scheme, where a fraction of the total channel uses is dedicated for UL
training and the rest of each time slot is utilized for data transmission in
DL direction.

Figure 2: (a) Proposed SCDC Scheme, (b) Sequential Beamforming with DL data
transmission and UL CSI, and (c) Conventional HD Beamforming with UL training
over two consecutive time slots.

###  III-B Channel Estimation Error and Average DL Rate

To estimate the UL channel at $(i\\!-\\!1)$th time slot, the $K$ users
transmit orthogonal training symbols
$\mathbf{S}_{K}[i-1]\in\mathbb{C}^{K\times T}$, such that
$\mathbb{E}[\mathbf{S}_{K}[i-1]\mathbf{S}_{K}^{\rm H}[i-1]]=TP_{K}$. Here, we
utilize $T$ training symbols in the UL direction for estimation, while
maintaining DL data transmission enabled by FD. Therefore, the received
training symbols at node $b$ after A/D cancellation,
$\mathbf{Y}_{b}[i-1]\in\mathbb{C}^{N_{b}\times T}$, can be expressed as

$\displaystyle\mathbf{Y}_{b}[i\\!-\\!1]$
$\displaystyle\triangleq\mathbf{H}^{\rm
T}[i\\!-\\!1]\mathbf{S}_{K}[i\\!-\\!1]+\widetilde{\mathbf{H}}_{b,b}[i\\!-\\!1]\mathbf{V}_{b}[i\\!-\\!1]\mathbf{S}_{b}[i\\!-\\!1]$
$\displaystyle\qquad+\mathbf{N}_{b}[i\\!-\\!1],$ (3)

where $\mathbf{H}^{\rm
T}[i-1],\widetilde{\mathbf{H}}_{b,b}[i-1],\mathbf{V}_{b}[i-1],\mathbf{S}_{b}[i-1]$,
and $\mathbf{N}_{b}[i-1]$ represent the UL channel, residual SI channel, DL
precoder, transmit signal matrix, and AWGN noise matrix of node $b$ RXs at
$(i\\!-\\!1)$th time slot.

The considered MIMO channels $\mathbf{H}[i\\!-\\!1]$ and $\mathbf{H}[i]$ of
the $(i\\!-\\!1)$ and $i$th time slots, respectively, are assumed temporally
correlated. Using the classical Jakes model, the correlation coefficient
between DL channels $\mathbf{H}[i\\!-\\!1]$ and $\mathbf{H}[i]$ is defined as
[16, Eq. (2.58)]

$\begin{split}\rho\triangleq\mathbb{E}[\mathbf{H}^{\rm
H}[i]\mathbf{H}[i\\!-\\!1]]=J_{0}(2\pi f_{d}T_{c}),\end{split}$ (4)

where $f_{d}$ is the Doppler frequency, $T_{c}$ is the time difference between
two consecutive time slots, $J_{0}(\cdot)$ is the zero-th order Bessel
function of the first kind. Using the Gauss-Markov error model, the time
correlated DL channels in successive time slots can be expressed as [15]

$\begin{split}\mathbf{H}[i\\!-\\!1]\triangleq\rho\mathbf{H}[i]+\sqrt{1-\rho^{2}}\mathbf{E}[i],\end{split}$
(5)

where $\mathbf{E}[i]\in\mathbb{C}^{K\times N_{b}}$ is the error matrix, whose
entries are modeled as IID $\mathcal{C}\mathcal{N}(0,l_{K})$, and is
independent of $\mathbf{H}[i]$. Due to reciprocity, similar time correlation,
as in (5), hold for all UL channels. Under this model, using UL/DL
reciprocity, the received training signals at BS node $b$ at the
$(i\\!-\\!1)$th time slot can be written using (III-B) as

$\begin{split}\mathbf{Y}_{b}[i\\!-\\!1]\triangleq&\rho\mathbf{H}^{\rm
T}[i]\mathbf{S}_{K}[i\\!-\\!1]+\sqrt{(1-\rho^{2})P_{K}}\mathbf{E}_{\text{UL}}[i]\\\
&+\widetilde{\mathbf{H}}_{b,b}[i\\!-\\!1]\mathbf{V}_{b}[i\\!-\\!1]\mathbf{S}_{b}[i\\!-\\!1]+\mathbf{N}_{b}[i\\!-\\!1],\end{split}$
(6)

where $\mathbf{E}_{\text{UL}}[i]\in\mathbb{C}^{N_{b}\times T}$ is the UL CSI
delay error matrix modeled as IID $\mathcal{C}\mathcal{N}(0,l_{K})$ and
independent of $\mathbf{N}_{b}[i\\!-\\!1]$. Based on (6), the UL channel
$\mathbf{H}^{\rm T}[i]$ can be estimated using the Minimum Mean Squared Error
(MMSE) estimator as [15]

$\begin{split}\widehat{\mathbf{H}}^{\rm
T}[i]\triangleq&\frac{\rho\mathbf{Y}_{b}[i\\!-\\!1]\mathbf{S}^{\rm
H}_{K}[i\\!-\\!1]}{\sigma_{b}^{2}+\sigma_{r,b}^{2}+(1-\rho^{2})P_{K}l_{K}+\rho^{2}TP_{k}},\end{split}$
(7)

where
$\sigma_{r,b}^{2}=\|\widetilde{\mathbf{H}}_{b,b}[i\\!-\\!1]\mathbf{V}_{b}[i\\!-\\!1]\mathbf{S}_{b}[i\\!-\\!1]\|^{2}$
is the residual SI power after A/D cancellation at node $b$. From the UL
estimate at the $i$th time slot, the relationship between the actual and
estimated DL channels at the $i$th time slot, using the Gauss-Markov error
model, can be written as[17]

$\begin{split}\mathbf{H}[i]\triangleq\sqrt{1-\tau^{2}_{\text{DL}}}\widehat{\mathbf{H}}[i]+\tau_{\text{DL}}\mathbf{E}_{\text{DL}}[i],\end{split}$
(8)

where $\mathbf{E}_{\text{DL}}[i]$ is the DL estimation error matrix having IID
elements each modeled as $\mathcal{C}\mathcal{N}(0,l_{K})$, and
$\tau_{\text{DL}}\in[0,1]$ is the Gauss-Markov error parameter that depends on
the effective DL Signal-to-Noise Ratio (SNR). The case $\tau_{\rm DL}=0$
implies ideal DL CSI, whereas $\tau_{\rm DL}=1$ signifies unavailable channel
estimation. The Mean Squared Error (MSE) of the DL channel from (7) at the BS
node $b$ is given by [15]

$\begin{split}{\rm
MSE_{FD}}\triangleq\tau_{\text{DL}}^{2}=\frac{\sigma_{b}^{2}+\sigma_{r,b}^{2}+(1-\rho^{2})P_{K}l_{K}}{\sigma_{b}^{2}+\sigma_{r,b}^{2}+(1-\rho^{2})P_{K}l_{K}+\rho^{2}TP_{k}},\end{split}$
(9)

Using the DL estimate $\widehat{\mathbf{H}}[i]$, the achievable DL rate per
channel use for the proposed SCDC scheme at the $i$th time slot can be derived
as

$\displaystyle\mathcal{R}_{\text{DL}}[i]\\!\triangleq$
$\displaystyle\log_{2}\Big{(}\text{det}\Big{(}\mathbf{I}_{K}+(1-\tau_{\text{DL}}^{2}){P}_{b}\widehat{\mathbf{H}}[i]\mathbf{V}_{b}[i]$
$\displaystyle\qquad\qquad\times\mathbf{V}_{b}^{\rm
H}[i]\widehat{\mathbf{H}}^{\rm
H}[i]\boldsymbol{\Sigma}_{k}[i]^{-1}\Big{)}\Big{)},$ (10)

where the interference-plus-noise covariance matrix:

$\displaystyle\boldsymbol{\Sigma}_{k}[i]\triangleq$
$\displaystyle\sigma^{2}_{K}\mathbf{I}_{K}\\!+\\!\widetilde{\mathbf{H}}_{K,K}[i]\mathbf{s}_{K}[i]\mathbf{s}_{K}^{\rm
H}[i]\widetilde{\mathbf{H}}_{K,K}^{\rm
H}[i]\\!+\\!\mathbf{H}_{\text{IN}}[i]\mathbf{s}_{K}[i]$
$\displaystyle\times\mathbf{s}_{K}^{\rm H}[i]\mathbf{H}_{\text{IN}}^{\rm
H}[i]+\tau_{\text{DL}}^{2}{P}_{b}l_{K}\mathbf{V}_{b}[i]\mathbf{V}_{b}^{\rm
H}[i].$ (11)

##  IV Proposed Joint Optimization Framework

Algorithm 1 Proposed FD MIMO Design

0: $\widehat{\mathbf{H}}_{b,b}$, $\widehat{\mathbf{H}}$,
$\widehat{\mathbf{H}}_{K,K}$, ${P}_{b}$, ${P}_{K}$, $N$, and $N_{b}$.

0: $\mathbf{V}_{b}$, $\mathbf{C}_{b}$, $\mathbf{C}_{K}$, $\mathbf{D}_{b}$, and
$\mathbf{D}_{K}$.

1: Obtain the $N$-tap analog canceller $\mathbf{C}_{b}$ using [10, Eq. 1].

2: Obtain $\mathbf{Q}_{b}$ including the $N_{b}$ right-singular vectors of
$(\widehat{\mathbf{H}}_{b,b}+\mathbf{C}_{b})$ corresponding to the singular
values in descending order.

3: Set $\mathbf{C}_{K}=-\widehat{\mathbf{H}}_{K,K}$.

4: for $\alpha=N_{b},N_{b}-1,\dots,2$ do

5: Set $\mathbf{F}_{b}=[\mathbf{Q}_{b}]_{(:,N_{b}-\alpha+1:N_{b})}$.

6: Set $\mathbf{W}_{b}=\text{sort}(\widehat{\mathbf{H}}\mathbf{F}_{b})$,
rearranging the rows of $\widehat{\mathbf{H}}\mathbf{F}_{b}$ based on
descending row vector norm.

7: Set $\mathbf{Z}_{b}=[\mathbf{W}_{b}]_{(1:m_{b},:)}$.

8: Set $\mathbf{G}_{b}=\beta_{\text{ZF}}\mathbf{Z}_{b}^{\rm
H}(\mathbf{Z}_{b}\mathbf{Z}_{b}^{\rm H})^{-1}$, where $\beta_{\text{ZF}}$ is
the normalization constant that ensures
$\mathbb{E}[\text{tr}(\mathbf{G}_{b}\mathbf{G}_{b}^{\rm H})]=1$.

9: Set the DL precoder as $\mathbf{V}_{b}=\mathbf{F}_{b}\mathbf{G}_{b}$.

10: if
${P}_{b}\|[(\widehat{\mathbf{H}}_{b,b}+\mathbf{C}_{b})\mathbf{V}_{b}]_{(j,:)}\|^{2}\\!\leq\\!\lambda_{b},\forall{j\\!=\\!1,2,\ldots,N_{b}}$,
and
$P_{K}\|[(\widehat{\mathbf{H}}_{k,k}+\mathbf{C}_{k})]_{(j,:)}\|^{2}\leq\lambda_{k},\,\forall
j\\!=\\!1,2,\ldots,K$ then

11: Output $\mathbf{V}_{b}$, $\mathbf{C}_{b}$, $\mathbf{C}_{K}$,
$\mathbf{D}_{b}=-(\widehat{\mathbf{H}}_{b,b}+\mathbf{C}_{b})$,
$\mathbf{D}_{K}=-(\widehat{\mathbf{H}}_{K,K}+\mathbf{C}_{K})$, and stop the
algorithm.

12: end if

13: end for

14: Set $\mathbf{V}_{b}=[\mathbf{Q}_{b}]_{(:,N_{b})}$.

15: if
${P}_{b}\|[(\widehat{\mathbf{H}}_{b,b}+\mathbf{C}_{b})\mathbf{V}_{b}]_{(j,:)}\|^{2}\leq\lambda_{b},\forall{j}\\!=\\!1,2,\ldots,N_{b}$,
and
$P_{K}\|[(\widehat{\mathbf{H}}_{k,k}+\mathbf{C}_{k})]_{(j,:)}\|^{2}\leq\lambda_{k},\,\forall
j\\!=\\!1,2,\ldots,K$ then

16: Output $\mathbf{V}_{b}$, $\mathbf{C}_{b}$, $\mathbf{C}_{K}$,
$\mathbf{D}_{b}=-(\widehat{\mathbf{H}}_{b,b}+\mathbf{C}_{b})$,
$\mathbf{D}_{K}=-(\widehat{\mathbf{H}}_{K,K}+\mathbf{C}_{K})$, and stop the
algorithm.

17: else

18: Output that the $\mathbf{C}_{b}$ realizations or $\mathbf{C}_{K}$ do not
meet the receive RF saturation constraints.

19: end if

In this section, we focus on the joint design of the digital TX precoder
$\mathbf{V}_{b}$, the analog SI cancellers $\mathbf{C}_{b}$ and
${c}_{k},\forall k=1,\cdots,K$, as well as the digital cancellers
$\mathbf{D}_{b}$ and ${d}_{k},\forall k$ at BS node $b$ and $K$ UE nodes,
respectively, maximizing the estimated achievable DL rate of $i$th time slot.
It is to be noted that, for simplicity, we omit the time slot index in the
subsequent optimization problem, as we are only dealing with variables at
$i$th time slot, unless mentioned otherwise. Based on the estimated DL channel
$\widehat{\mathbf{H}}$, SI channels $\widehat{\mathbf{H}}_{b,b}$ and
$\widehat{\mathbf{H}}_{K,K}$ at BS and UE nodes, respectively, the considered
optimization problem is expressed as:

$\displaystyle\underset{\begin{subarray}{c}\mathbf{V}_{b},\mathbf{C}_{b},\mathbf{D}_{b}\\\
\mathbf{C}_{K},\mathbf{D}_{K}\end{subarray}}{\text{max}}$
$\displaystyle\log_{2}\\!\left(\\!\text{det}\left(\mathbf{I}_{K}\\!+\\!(1\\!-\\!\tau_{\text{DL}}^{2}){P}_{b}\widehat{\mathbf{H}}\mathbf{V}_{b}\mathbf{V}_{b}^{\rm
H}\widehat{\mathbf{H}}^{\rm H}\boldsymbol{\Sigma}_{k}^{-1}\right)\right)$ s.t.
$\displaystyle
P_{b}\|[(\widehat{\mathbf{H}}_{b,b}+\mathbf{C}_{b})\mathbf{V}_{b}]_{(j,:)}\|^{2}\leq\lambda_{b}\,\forall
j=1,2,\ldots,N_{b},$ $\displaystyle
P_{K}\|[(\widehat{\mathbf{H}}_{k,k}+\mathbf{C}_{k})]_{(j,:)}\|^{2}\leq\lambda_{k},\,\forall
j=1,2,\ldots,K,$
$\displaystyle\mathbb{E}\\{\|\mathbf{V}_{b}\mathbf{s}_{b}\|^{2}\\}\leq{P}_{b},\,\,\text{and}\,\,\,\mathbb{E}\\{\|\mathbf{s}_{K}\|^{2}\\}\leq{P}_{K}.$
(12)

In this formulation, the first constraint imposes the RX RF chain saturation
threshold $\lambda_{b}$ after analog cancellation at BS node $b$. As
previously discussed, this threshold ensures proper reception of the training
symbols by all $N_{b}$ RX RF chains of the BS, which means that the UL channel
can be efficiently estimated using (7). The second constraint enforces the
saturation threshold $\lambda_{K}$ at the $K$ UEs assuring feasible decoding
of BS’s information data symbols. The final two constraints in (IV) refer to
the nodes’ average transmit powers.

The optimization problem in (IV) is quite difficult to tackle, since it is
non-convex including couplings among the optimization variables. In this
paper, we suboptimally solve it using an alternating optimization approach,
leaving other possibilities for future work. To this end, we start with an
allowable $\mathbf{C}_{b}$ realization given the available number of analog
canceller taps $N$, where the tap values are set to be the respective
amplitude elements of the estimated SI channel $\widehat{\mathbf{H}}_{b,b}$.
Based on the chosen $\mathbf{C}_{b}$, we seek for the precoding matrix
$\mathbf{V}_{b}$ maximizing the DL rate, while meeting the first constraint
for the BS analog SI cancellation threshold $\lambda_{b}$. This procedure is
repeated for all allowable realizations of $\mathbf{C}_{b}$ to find the best
pair of $\mathbf{C}_{b}$ and $\mathbf{V}_{b}$. Adopting the approach in [10],
the BS precoder for DL data communication is constructed as
$\mathbf{V}_{b}=\mathbf{F}_{b}\mathbf{G}_{b}$, where
$\mathbf{F}_{b}\in\mathbb{C}^{N_{b}\times\alpha}$ aims at reducing the
residual SI after analog cancellation and
$\mathbf{G}_{b}\in\mathbb{C}^{\alpha\times m_{b}}$ is the Zero-Forcing (ZF)
beamformer maximizing the rate of the effective DL channel
$\widehat{\mathbf{H}}\mathbf{F}_{b}$. The parameter $\alpha$ is a positive
integer taking the values $1\leq\alpha\leq N_{b}$. At each of the UE nodes, a
single-tap analog canceller is employed resulting in a diagonal analog SI
canceller matrix $\mathbf{C}_{K}=-\widehat{\mathbf{H}}_{K,K}$. To maximize the
signal-to-interference-plus-noise ratio, the residual SI is further reduced by
setting the digital cancellation signal at both FD nodes as their respective
complementary residual SI channels after analog SI cancellation. For each
allowable realization of $\mathbf{C}_{b}$, the proposed solution for the
considered optimization problem (IV) is summarized in Algorithm 1.

Figure 3: Achievable DL Rate w.r.t DL transmit power for $N_{b}=8$, $K=4$,
$f_{d}=50$Hz and different analog canceller taps $N$.

##  V Simulation Results and Discussion

In this section, we provide representative simulation results of the proposed
FD-based MU MIMO simultaneous data communication and channel estimation
approach. In Section V-A, we describe the existing FD and HD MIMO solutions to
be compared with our proposed approach. The simulations parameters and
assumptions are detailed in Section V-B, whereas the hardware complexity, and
achievable DL rate results are presented in Section V-C.

###  V-A Compared FD and HD MIMO Designs

We compare our novel FD SCDC scheme with the Sequential Beamforming (SB) FD
approach presented in [12], where the UEs send their training symbols using
TDMA. Contrary to the conventional HD MIMO approach, which is also illustrated
here for comparison, in SB FD approach, the BS node does not wait for all the
UEs to finish training. In SB FD, the BS starts transmitting to the UE with
available CSI at the BS node, while receiving training from other UEs. The SB
FD approach considers perfect analog SI cancellation, which includes full-tap
analog SI canceller. In addition, we also illustrate the proposed FD approach
with ideal CSI and full-tap analog SI canceller.

###  V-B Simulation Parameters

We perform an extensive simulation following the FD MIMO architecture
illustrated in Fig. 1. We have considered an $8\times 8$ (i.e., $N_{b}=8$) FD
MIMO BS node $b$ serving $K=4$ single-antenna FD UE nodes. DL, UL, and inter-
node interference channels are assumed as block Rayleigh fading channels with
a pathloss of $110$dB. The SI channels at the BS and all UE nodes are
simulated as Rician fading channels with a $\kappa$-factor of $30$dB and
pathloss of $40$dB [2]. We have considered a narrowband communication system
with a bandwidth and carrier frequency of $1.4$MHz and $2.4$GHz, respectively.
RX noise floors at all nodes were assumed to be $-100$dBm. To this end, the
RXs have effective dynamic range of $50$dB provided by the $12$-bit analog-to-
digital converters (ADC) for a Peak-to-Average-Power-Ratio (PAPR) of $10$ dB
[18]. Therefore, the residual SI power after analog SI cancellation at the
input of each RX chain has to be below $-50$dBm to avoid RX RF chain
saturation. Furthermore, non-ideal multi-tap analog canceller is considered
with steps of $0.02$dB for attenuation and $0.13^{\circ}$ for phase as in
[10]. All the user nodes employ single-tap SI canceller, where the MIMO BS
node requires an $N$-tap analog SI canceller. We have used $1000$ independent
Monte Carlo simulation runs to calculate the performance of all considered
designs. Every transmission time slot is considered to be $1$ms with $T=400$
symbols. For the compared HD and SB FD scheme, $10\%$ of the total symbols
were dedicated for UL channel sounding. The UL transmit power of each UE node
is limited to $10$dBm for all considered designs.

Figure 4: Achievable DL Rate w.r.t number of UEs $K$ for $N_{b}=8$,
$P_{b}=40$dBm, $f_{d}=50$Hz and different analog canceller taps $N$.

###  V-C Analog Canceller Complexity and Achievable DL Rate

We illustrate the DL rate performance of the proposed and other considered
designs in Fig. 3 with respect to DL transmit power for $K=4$, $f_{d}=50$Hz
and varying analog canceller taps $N$. The proposed SCDC scheme with full-tap
analog canceller with $N=64$ provides DL rate close to the ideal case for
transmit power below $30$dBm and outperforms all other considered designs for
all transmit powers. After $50\%$ reduction of analog canceller taps, the
proposed approach with $N=32$ still provides $1.25\times$ DL rate compared to
the SB FD [12] and conventional HD approach for transmit power of $40$dBm,
while achieving identical DL rate performance for low transmit powers.
Furthermore, we consider a more extreme case with $N=8$, which results in an
analog SI canceller with only $12.5\%$ taps. Although, such low-complexity
SCDC approach provides less DL rate compared to the SB FD approach, it can
achieve higher performance for high transmit powers above $35$dBm.

In Fig. 4, we plotted the achievable DL rate of the considered transmission
schemes with respect to the the number of UEs $K$ being served by the FD BS
node for a fixed DL transmit power of $40$dBm. It is evident from the figure
that, the proposed SCDC scheme with $N=64$ provides substantial DL rate
increment compared to the SB FD [12] and the conventional FD design for any
number of UEs. However, the proposed approach with $N=32$ can efficiently
serve $6$ UEs providing better DL rate than the SB FD approach. For a further
reduction of canceller taps, the proposed SCDC approach with $N=8$ achieves
higher rate than SB FD and conventional HD scheme up to $5$ UEs. Therefore,
the proposed SCDC scheme provides a flexible design with an important trade-
off between number of UEs and analog SI canceller taps that achieves higher DL
rate performance compared to the full-tap SB FD approach.

We showcase the DL rate performance of the considered design with respect to
the Doppler frequency $f_{d}$ in Fig. 5 for $K=4$ UEs and fixed DL transmit
power of $40$dBm. As described in III-B, the successive time slots are
correlated and the correlation coefficient is measured using $f_{d}$ given
$T_{c}=1$ms. It is evident from Fig. 5 that the increment of $f_{d}$ only
affects the proposed scheme, as it estimates the DL channel based on the
training symbols acquired in the preceding time slot. However, it is shown
that the proposed schemes with $N=64$ and $N=32$ taps provide superior rate
compared to the SB FD[12] and the conventional HD approach for a high Doppler
frequency of $220$Hz, which for our considered communication parameter
represents a relative velocity $100$km/h between BS and UE nodes. The proposed
SCDC scheme with $N=8$ can achieve higher rate for $f_{d}$ up to $140$Hz.
Therefore, we conclude that, compared to the SB FD and conventional HD scheme,
the proposed SCDC approach can achieve higher rates with $50\%$ less taps.

Figure 5: Achievable DL Rate w.r.t Doppler frequency $f_{d}$ for $N_{b}=8$,
$P_{b}=40$dBm, $K=4$ and different analog canceller taps $N$.

##  VI Conclusion

In this paper, we proposed a multi-user FD MIMO communication system for
simultaneous DL information data transmission and UL CSI estimation with
reduced complexity multi-tap analog SI cancellation. Considering an MMSE-based
channel estimation error model, we presented a unified optimization framework
for the joint design of digital TX precoding and A/D SI cancellation. Our
performance evaluation results demonstrated that the proposed SCDC protocol is
capable of achieving improved achievable DL rates compared to the existing FD
and conventional HD systems, requiring reduced complexity analog cancellation
compared to conventional FD MIMO architectures.

## Acknowledgments

This work was partially funded by the National Science Foundation CAREER award
#1620902.

## References

  * [1] A. Sabharwal, P. Schniter, D. Guo, D. W. Bliss, S. Rangarajan, and R. Wichman, “In-band full-duplex wireless: Challenges and opportunities.” _IEEE J. Sel. Areas Commun._ , vol. 32, no. 9, pp. 1637–1652, Sep. 2014.
  * [2] M. Duarte, C. Dick, and A. Sabharwal, “Experiment-driven characterization of full-duplex wireless systems,” _IEEE Trans. Wireless Commun._ , vol. 11, no. 12, pp. 4296–4307, Dec. 2012.
  * [3] D. Bharadia, E. McMilin, and S. Katti, “Full duplex radios,” in _Proc. ACM SIGCOMM_ , Hong Kong, China, 12-16 Aug. 2013, pp. 375–386.
  * [4] B. Smida and S. Khaledian, “ReflectFX: In-band full-duplex wireless communication by means of reflected power,” _IEEE Trans. Commun._ , vol. 65, no. 5, pp. 2207–2219, May 2017.
  * [5] M. A. Islam and B. Smida, “A comprehensive self-interference model for single-antenna full-duplex communication systems,” in _Proc. IEEE ICC_ , Shanghai, China, May 2019, pp. 1–7.
  * [6] G. C. Alexandropoulos, M. A. Islam, and B. Smida, “Full duplex hybrid A/D beamforming with reduced complexity multi-tap analog cancellation,” in _Proc. IEEE SPAWC_ , Atlanta, USA, May 2020, pp. 1–6.
  * [7] M. A. Islam, G. C. Alexandropoulos, and B. Smida, “A unified beamforming and A/D self-interference cancellation design for full duplex MIMO radios,” in _Proc. IEEE PIMRC_ , Istanbul, Turkey, Sep. 2019, pp. 1–7.
  * [8] T. Riihonen, S. Werner, and R. Wichman, “Mitigation of loopback self-interference in full-duplex MIMO relays,” _IEEE Trans. Signal Process._ , vol. 59, no. 12, pp. 5983–5993, Dec. 2011.
  * [9] E. Everett, C. Shepard, L. Zhong, and A. Sabharwal, “Softnull: Many-antenna full-duplex wireless via digital beamforming,” _IEEE Trans. Wireless Commun._ , vol. 15, no. 12, pp. 8077–8092, Dec. 2016.
  * [10] G. C. Alexandropoulos and M. Duarte, “Joint design of multi-tap analog cancellation and digital beamforming for reduced complexity full duplex MIMO systems,” in _Proc. IEEE ICC_ , Paris, France, May 2017, pp. 1–7.
  * [11] X. Du, J. Tadrous, C. Dick, and A. Sabharwal, “MU-MIMO beamforming with full-duplex open-loop training,” in _Proc. IEEE SPAWC_ , Stockholm, Sweden, Jun. 2015, pp. 301–305.
  * [12] X. Du, J. Tadrous, and A. Sabharwal, “Sequential beamforming for multi-user MIMO with full-duplex training,” _IEEE Trans. Wireless Commun._ , vol. 15, no. 12, pp. 8551–8564, Dec. 2016.
  * [13] J. Mirza, G. Zheng, K.-K. Wong, S. Lambotharan, and L. Hanzo, “On the performance of multi-user MIMO systems relying on full-duplex CSI acquisition,” _IEEE Trans. Wireless Commun._ , vol. 66, no. 10, pp. 4563–4577, Oct. 2018.
  * [14] M. A. Islam, G. C. Alexandropoulos, and B. Smida, “Simultaneous downlink data transmission and uplink channel estimation with reduced complexity full duplex MIMO radios,” in _Proc. IEEE ICC_ , Dublin, Ireland, Jun. 2020, pp. 1–6.
  * [15] M. Kobayashi, N. Jindal, and G. Caire, “Training and feedback optimization for multi-user MIMO downlink,” _IEEE Tran. on Commun._ , vol. 59, no. 8, pp. 2228–2240, May 2011.
  * [16] D. Tse and P. Viswanath, _Fundamentals of wireless communication_. Cambridge university press, 2005.
  * [17] H. Iimori, G. T. F. de Abreu, and G. C. Alexandropoulos, “MIMO beamforming schemes for hybrid SIC FD radios with imperfect hardware and CSI,” _IEEE Trans. Wireless Commun._ , vol. 18, no. 10, pp. 4816–4830, Oct. 2019\.
  * [18] _Analog-to-Digital Converters_ , ADC3241, Texas Instruments, 2016.