# Towards an Abolitionist AI: the role of Historically Black Colleges and
Universities

Charles C. Earl
Automattic.com
<EMAIL_ADDRESS>All opinions and analysis in this work are the
author?s alone.

###### Abstract

_Abolition_ is the process of destroying and then rebuilding the structures
that impede liberation. This paper addresses the particular case of Black folk
in the United States, but is relevant to the global decolonization movement.
Using notions of abolition and infrastructures of feeling developed by Ruth
Wilson Gilmore, I view Historically Black Colleges and Universities ( _HBCUs_
) as a particular kind of abolitionist project, created for the explicit
purpose of nurturing and sustaining Black excellence particularly within the
sciences. I then examine how artificial intelligence (AI) in particular and
computing in general have contributed to racial oppression and the further
confinement and diminishing of Black existence. I conclude by examining how
the space held by HBCUs in computing might contribute to a reimagining of AI
as a technology that enhances the possibility and actualization of Black life.

##  1 Introduction

As a child, I attended the nursery school of Spelman College, a historically
Black women’s college located in Atlanta. One of my earliest memories is of
witnessing a protest taking place on the campus. The young women of Spelman,
in their proud defiant afros, were protesting the visit of Nelson Rockefeller.
They were demanding that the Rockefeller family – a major funder of the
college – divest from all assets invested in the South African apartheid
regime. Rockefeller’s grandfather, John D Rockefeller, had given the college
its first major endowment, and what had been the Atlanta Baptist Female
Seminary had been renamed in honor of John Rockefeller’s wife, Laura Spelman.

In an age that demands resistance and the abolition of structures that
continue to oppress Black people, what is the role of historically Black
institutions like Spelman College? And in particular what should be the role
of those institutions in defining how Black people interact with and define
the development of AI? Given the outsize role these institutions play in the
development of Black computing professionals – 9% of Black undergrads attend
HBCUs, yet HBCUs produce 37% of the Black STEM graduates in the United States
– it is important to assess how these graduates are prepared to address the
inequities in computing that exacerbate these harms. Further, these
institutions continue to receive significant funding from sources whose aims
are at odds with liberation. In this paper, my hope is to raise questions and
initiate dialog about how these institutions might contribute to liberatory
computing.

##  2 Abolition and the geographies of freedom

The geographer Ruth Wilson Gilmore has been one of the most eloquent and
incisive voices articulating what abolition means in 2020. Her book The Golden
Gulag Gilmore (2007) is both a meditation on the campaign to slow the growth
of state and private prisons in California as well as an exploration of the
harm done by removing scores of people from society for the sake of
retribution. In the specific case of prisons, Gilmore is asking us to confront
what we think prisons are for – punishment? rehabilitation? justice? – and the
reality of what they actually accomplish.

Though Gilmore’s work focuses primarily on prison abolition, she articulates a
broad, encompassing vision of liberation, developed out of her geographer’s
view of the world. In her essay Abolition Geography and the Problem of
Innocence Gilmore (2017) she writes Abolition geography starts from the homely
premise that freedom is a place (Gilmore, 2017, pg. 227). Abolition then is
overall the affirming of life out of places intended for domination,
exploitation, and ultimately spiritual, intellectual and physical death:

> If unfinished liberation is the still-to-be-achieved work of abolition, then
> at the bottom what is to be abolished isn’t the past or its present ghost,
> but rather the process of hierarchy, dispossession, and exclusion that
> congeal in and as group-differentiated vulnerability to premature death.
> (Gilmore, 2017, pg. 228)

It is then this process of dispossession of life itself – epitomized by
prisons – that is the enforcement of the inequality necessitated by
capitalism. The central tenet of Gilmore’s work is the examination of these
qualities of existence. The carceral landscape of capitalism that demands an
extractive hierarchy on the one hand, and the affirming alternate reality that
the exploited imagine and create in real time and in real spaces on the other.

But Wilson’s work sees liberation as a dual of abolition. Referencing W.E.B.
DuBois’s classic Black Reconstruction 1998 she makes clear that the formerly
enslaved were always creating free spaces in which they could exist as whole
human beings. For Wilson, this continuous building creates an infrastructure
of feeling (Gilmore, 2017, pg. 236-7) – an emotional, spiritual practices that
connect the present generation to the love and aspirations of ancestors that
struggled and dream of independence.

HBCUs exemplify a place of freedom in Gilmore’s conception – most having been
birthed in during Reconstruction. HBCUs provide havens in which Black scholars
can exist without fear of thinking/coding while Black. In his study of the
affirming environment at Florida A&M University (one of the largest producers
of Black CS grads) Allen concludes that HBCUs "provide a relief from the
burdens of oppressive articulations and experiences of society and space"
Allen (2020).

The impact that this space cannot be overstated. There are 102 HBCUs in the
U.S. Despite the fact that 9% of Black undergrads students attend these
institutions, they account for 37 percent of the degrees awarded to Black
people in STEM disciplines. 33 percent of the Black recipients of doctoral
degrees in computing earned their first degree from a HBCU, and 12 % of the
Black PhDs in computing receive their doctorate from an HBCU. Over the nine
year period 2007 - 2017, the HBCUs Spelman College, Howard University, and
North Carolina A&T State University had the highest yield of doctorates (Black
CS or Math doctorates per 100 Bachelors degrees in STEM) of any institutions
in the United States Gasman and Nguyen (2014). Beyond the CS discipline, two
of the most impactful thinkers on abolition – Dr. Ruha Benjamin and Dr.
Tressie McMillan Cottam – themselves received undergraduate degrees from HBCUs
– respectively Spelman College and North Carolina Central University.

##  3 The need for abolition in AI

If capitalism requires inequity, and if racism enshrines it Gilmore (2017),
then the hyper profit motive of AI applied to platform capitalism has been a
successful effort to optimize racism. The uses of AI – particularly in
platforms that derive value from manipulative practices – are necessarily not
in the interest of Black people. Benjamin (2019), Buolamwini and Gebru (2018)
and others have documented how AI has been used to enforce racial hierarchies
of power. Cottom (2020) further discuses how the logic of racial capitalism
has been ensconced into the logic of all web scale platforms – it is at this
point core to Internet economy.

From above work we can distill a few kinds of technologies and practices that
reinforce racial oppression:

  * •

_Carceral technologies_ : for example predictive policing and algorithmic
sentencing are used explicitly for the purposes of capturing, isolating, and
destroying Black life.

  * •

_Discriminatory resource allocation_ existing technologies are used to limit
the access of Black people to resources from health care to credit, often
optimizing for profit over well-being.

  * •

_Surveillance technologies_ are used explicitly for expropriation of
behavioral surplus and circumscribing autonomy by predictively tracking and
controlling the behavior of the surveilled.

  * •

_Racialized hiring and software engineering practices_ practices of
recruiting, advancement, and project management within software organizations
that enable the reproduction of racial harms.

  * •

_Militarization of AI_ particularly in the secretive use of technology to
further colonialist agendas on the global south – the use of this technology
upon the civilian Black and Indigenous population of the U.S. is particularly
concerning.

On the other hand, there are tools that Benjamin identifies as being able to
dismantle these technologies, mitigate their harm, or even better create the
spaces that allow free spaces to come into being.

  * •

_Creating space for and voice to the marginalized_ , the practice of centering
marginalized voices by enabling full participation, contribution, and
equitable control of the development process.

  * •

_Algorithmic and process audits_ that identify the harms caused by algorithmic
decision making and also identify the systemic issues in the software
development process that allow them to go unchecked.

  * •

_Racial abuse protection and mitigation_ technologies that intercept racial
attacks and mitigate harm (e.g. micro-aggression classifiers)

  * •

_Liberatory design_ practices which enable the development of software
products center the aspirations and requirements of marginalized community
members.

  * •

_Equitable work practices_ that recognize and reward the contribution of data
annotators, content moderators, and other "gig" workers whose value is
extracted and erased by racialized platform capitalism.

##  4 Abolition and the reproduction of oppression at HBCUs

We can get some understanding of the degree to which abolitionist computing
within HBCUs occurs by looking at the research funding. The major U.S. federal
agencies – among them Department of Defense (DoD), National Institutes of
Health (NIH), and National Science Foundation (NSF) – allocate large grants
specifically targeted to improve the research and teaching capacity of HBCUs.
The NSF annually funds the HBCU Excellence in Research (EiR) program ($10
million per year), and the Historically Black Colleges and Universities
Undergraduate Program (HBCU-UP) ($55 million per year).

A close look at research activity at Howard University raises the other
question of to what degree Black computing scholars are being trained to
participate in the reproduction of racist technology. At Howard, approximately
$15 million awarded in computing research for 2020 (excluding on-going
grants), 50% were from department of defense grants in bio-metrics and cyber
defense (e.g. Howard was awarded $7.5 million for a DoD Center for Excellence
in AI/ML CoE-AIML (2020) ), 4% from educational awards for development of
retention and STEM curricula, while 1% was awarded to research specifically
aimed at addressing online abuse directed toward Black people (e.g. "Deep
Learning of Sentiment Analysis and Codeswitching for Identification of Harmful
Technologies" and other initiatives from Dr. Gloria Washington’s Affective
Biometrics Lab ). Similar patterns emerge when looking at other HBCUs like
North Carolina A & T State University.

That is, with heavy DoD and platform capitalism support Black students and
researchers must navigate being participants in the reproduction of what
Mohamed et al. (2020) refers to as colonizing AI especially with respect to
surveillance and military technology. This is concerning especially given the
potential harms for Black folk in the U.S. and global south Garcia (2019).

##  5 Abolitionists agendas for HBCUs

To begin in earnest the development of abolitionist approaches to AI within
the HBCU landscape requires a return to the source. Tapping into the
infrastructure of feelings that already permeates these institutions and
enables their success in nurturing so many impactful people. Within available
resources already devoted to capacity development, there could be informal
meetups, conferences, gatherings across schools, across disciplines to explore
the implications of abolitionist approaches to computing. The significant
number of students involved in BLM actions combined with resources already
devoted to making computing accessible could be a powerful accelerator. The
development of conferences and centers to promote tools for community audits
and interrogation of software platforms is another approach that could have
wide adoption and immediate impact given the millions of Black people are
impacted daily by racialized ad targeting and unfair pricing experiments by
software platforms. Lastly, the Association for Computing Machinery (ACM) has
long acknowledged the dearth of adequate ethics education in computing. For AI
in particular, this is especially necessary given the potential for harm.
Perspectives on ethics that would explore methodologies for assessing what
should and should not be built would be an important preparation for students.
Engagement with scholars of the social sciences, law, and healthcare – present
at so many HBCUs – that could collectively work on the development of archives
and models capable of addressing community needs is also important.

##  6 Conclusion

I have attempted to consider the current state of affairs in AI viewed from
the perspective of abolition as articulated by Ruth Wilson Gilmore. Viewing AI
as a carceral space that replicates and accelerates inequity I have attempted
to look at the way in which HBCUs might chart a new course for it. There
remains much to be done.

## References

  * (1)
  * Allen (2020) Douglas L. Allen. 2020\.  Black Geographies of Respite: Relief, Recuperation, and Resonance at Florida A & M University.  _Antipode_ (2020).  https://doi.org/10.1111/anti.12658
  * Benjamin (2019) Ruha Benjamin. 2019\.  _Race after Technology: Abolitionist Tools for the New Jim Code._ Polity.
  * Buolamwini and Gebru (2018) Joy Buolamwini and Timnit Gebru. 2018.  Gender shades: Intersectional accuracy disparities in commercial gender classification.. In _Conference on fairness, accountability and transparency_.
  * CoE-AIML (2020) DoD CoE-AIML. 2020 (accessed November 17, 2020).  _Howard University DoD Center of Excellence in Artificial Intelligence & Machine Learning_.  https://www.dod-coe4ai-ml.org/
  * Cottom (2020) Tressie McMillan Cottom. 2020\.  Where Platform Capitalism and Racial Capitalism Meet: The Sociology of Race and Racism in the Digital Society.  _The Sociology of Race and Ethnicity_ (2020).
  * Du Bois (1998) W. E. B. Du Bois. 1998\.  _Black reconstruction in America: 1860-1880_.  New York : The Free Press.
  * Garcia (2019) Eugenio V. Garcia. 2019\.  The militarization of artificial intelligence: a wake-up call for the Global South. (September 2019).  https://dx.doi.org/10.2139/ssrn.3452323
  * Gasman and Nguyen (2014) M. Gasman and T. Nguyen. 2014.  Historically black colleges and universities (HBCUs): Leading our nation’s effort to improve the science, technology, engineering, and mathematics (STEM) pipeline.  _Texas Education Review_ 2, 1 (2014), 75–79.
  * Gilmore (2007) Ruth Wilson Gilmore. 2007\.  _Golden gulag: Prisons, surplus, crisis, and opposition in globalizing California_.  University of California Press.
  * Gilmore (2017) Ruth Wilson Gilmore. 2017\.  Abolition Geography and the Problem of Innocence.  In _Futures of Black Radicalism_ , G. T.. Johnson and A. Loubin (Eds.). New York: Verso, 225–240.
  * Lab (2020) Howard University Affective Biometrics Lab. 2020 (accessed November 17, 2020).  _Affective Biometrics Lab Research_.  https://sites.google.com/view/affective-biometrics-lab-resea/contact?authuser=0
  * Mohamed et al. (2020) Shakir Mohamed, Marie-Therese Png, and William Isaac. 2020\.  Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence.  arXiv:2007.04068 [cs.CY]


11institutetext: CMI Lab, School of Engineering and Informatics, University of
Wolverhampton, Wolverhampton, United Kingdom 22institutetext: Department of
Computing, São Paulo State University, Bauru, Brazil 33institutetext: School
of Computing, Edinburgh Napier University, Edinburgh, Scotland, United
Kingdom.
44institutetext: deepCI.org 20/1 Parkside Terrace, Edinburgh, United Kingdom
44email<EMAIL_ADDRESS>
# Canonical Cortical Graph Neural Networks and its Application for Speech
Enhancement in Audio-Visual Hearing Aids††thanks: The authors are grateful to
FAPESP grants #2013/07375-0, #2014/12236-1, #2017/02286-0, #2018/21934-5,
#2019/07665-4, and #2019/18287-0, CNPq grants #307066/2017-7, and
#427968/2018-6, as well as the Engineering and Physical Sciences Research
Council (EPSRC) grant EP/T021063/1.

Leandro A. Passos 11 João Paulo Papa 22 Amir Hussain 33 Ahsan Adeel 1144

###### Abstract

Despite the recent success of machine learning algorithms, most models face
drawbacks when considering more complex tasks requiring interaction between
different sources, such as multimodal input data and logical time sequences.
On the other hand, the biological brain is highly sharpened in this sense,
empowered to automatically manage and integrate such streams of information.
In this context, this work draws inspiration from recent discoveries in brain
cortical circuits to propose a more biologically plausible self-supervised
machine learning approach. This combines multimodal information using intra-
layer modulations together with Canonical Correlation Analysis, and a memory
mechanism to keep track of temporal data, the overall approach termed
Canonical Cortical Graph Neural networks. This is shown to outperform recent
state-of-the-art models in terms of clean audio reconstruction and energy
efficiency for a benchmark audio-visual speech dataset. The enhanced
performance is demonstrated through a reduced and smother neuron firing rate
distribution. suggesting that the proposed model is amenable for speech
enhancement in future audio-visual hearing aid devices.

###### Keywords:

Cortical Circuits Canonical Correlation Analysis Multimodal Learning Graph
Neural Network Prior Frames Neighborhood Positional Encoding.

##  1 Introduction

According to the World Health Organization (WHO), the number of people
requiring hearing rehabilitation in the world is estimated to rise from $430$
million nowadays up to $700$ million until 2050, with nearly $2.5$ billion
people presenting at least some degree of hearing impairment [23]. Despite the
impairment itself, deafness also impacts on social relationships [19, 16] and
environment perception [22], leading to other psychological and health
conditions [15]. Over such circumstances, employing high-end energy-efficient
technological approaches to build cross-modal sensory devices, i.e., combining
audio and visual inputs to enhance hearing aid devices, seems a plausible way
to improve individuals’ life quality.

In the last decades, machine learning-based techniques have shown themselves
as a suitable approach to tackle issues related to virtually any field of
science, industry, or even daily life, ranging from computer vision [36] to
medicine [29], and satellite imagery processing [12, 28]. Machine learning
also has been successfully employed in the context of speech enhancement [34,
35], whose aim is to enhance speech quality and intelligibility when noise
degrades them significantly [5]. Nevertheless, such methods may suffer massive
performance degradation in the presence of overwhelming noise [7]. Many works
address this problem using multimodal audio-visual (AV) information fusion.
Combining AV information usually demands more sophisticated approaches, which
intrinsically comprises several challenges, like data alignment, finding
semantic gaps between low-level features and high-level information [8], and
learning coherent and correlated latent patterns on different input
modalities.

Combining noisy audio and clean images for clean signal reconstruction is
analogous to reading the lips and body movements of a speaker in a boisterous
environment, e.g., a pub with loud music, to obtain some additional
information and create a context to enhance the information quality of a
speech suppressed by the loud sound. Ngiam et at. [21], for instance, proposed
a multimodal method capable of improving a target modality feature
representation. Further, Adeel et al. [2] provided several improvements to the
field, presenting a chaotic model for lip-reading integrating Internet of
Things (IoT) and 5G Cloud-Radio Access Network, further improving AV
information transmission for real-time speech reconstruction. Further work
employ deep learning-based approach to exploits AV cues to estimate clean
audio [4, 20], also considering distinct language speakers [13].

Recently, Passos et al. [25] proposed a multimodal self-supervised Graph
Neural Network (GNN) that combines AV data through using Canonical Correlation
Analysis Graph Neural Networks (CCA-GNN) [37], also modeling the temporal
information in the graph using the so-called prior-frame positional encoding.
The method obtained outstanding results considering audio reconstruction and
energy efficiency, analyzed in terms of neuronal activation rate.

Despite the advantages presented [25], the model lacks some points in the
context of a biologically plausible approach. In this context, Passos et al.
[24] proposed a multimodal approach using burst-dependent learning [26], a
method inspired by more recent studies on the physiological mechanism of
pyramidal neurons that regulates the learning by the frequency of bursts,
where the credit assignment problem is addressed by the primary principles of
pyramidal neurons suggested by Körding and König [18]. In parallel, the study
of canonical cortical circuits [9, 14] provides some interesting insights
regarding the brain procedure toward multimodal information processing.
Canonical cortical circuits model pyramidal neurons to receive different
modalities of information, modulated in an excitatory or inhibitory fashion on
deeper layers. Moreover, biologically plausible models should not
underestimate the importance of memory in the learning process, which performs
a fundamental role in information inference, acting as an intrinsic context
for novelty comprehension.

The attributes mentioned above motivated the development of the Canonical
Cortical Graph Neural Network, a novel self-supervised architecture that
remodels and improves the ideas developed in [25] by introducing a more
biologically plausible approach to modulating and filtering the multimodal
information inside the so-called cortical graph layers. It also introduces a
memory concept inside each cortical graph block, composed of a mechanism to
“forget” irrelevant facts and update itself considering every consecutive
node, which is presented in a logical time-step sequence using prior-frame-
based positional encoding. Experiments conducted over the AV ChiME3 [5]
dataset show the Canonical Cortical Graph Neural Network obtained state-of-
the-art results, outperforming CCA-GNN in the contexts of faster convergence,
audio reconstruction, and firing neuron activation rates. Regarding the
latter, the model not only obtained lower rates of activation, but also
produced smothier firing-rate distributions.

The main contributions of this paper are presented as follows:

  1. 1.

To propose the Canonical Cortical Graph Neural Network, a biologically
plausible model for multimodal information feature extraction.

  2. 2.

To introduce a novel paradigm for intra-layer information fusion and memory
modeling.

  3. 3.

To provide a self-supervised energy-efficient model for correlated feature
extraction for AV-based clean audio data reconstruction.

  4. 4.

To foster the literature regarding speech enhancement and AV hearing aids.

The remainder of this paper is presented as follows. Section 2 provides a
theoretical background regarding Graph Neural Networks with Canonical
Correlation Analysis and the prior frame-based graph positional encoding,
while Section 3 introduces the novel Canonical Cortical Graph Neural Network.
Further, Sections 4 and 5 present the methodology and the experimental
results, respectively. Finally, Section 6 states conclusions and future works.

##  2 Theoretical Background

This section provides a brief theoretical background regarding Graph Neural
Networks with Canonical Correlation Analysis and the prior frame-based graph
positional enconding.

###  2.1 Graph Neural Networks with Canonical Correlation Analysis

Let $G=(X,A)$ be a graph where $A\in\mathbb{R}^{N\times N}$ represents the
adjacency matrix and $X\in\mathbb{R}^{N\times F}$ the input data represented
by graph nodes. Additionally, $F$ represents the feature space dimension, and
$N$ denotes the number of nodes. The CCA-GNN [37] comprises three main steps,
i.e., a random graph generator $T$, a graph neural network encoder
$f_{\theta}$, where $\theta$ denotes the network’s learnable weights, and an
objective function based on Canonical Correlation Analysis. The graph
generator produces two augmented versions of the same graph, which are
presented to the graph neural network encoder for further computing and
maximizing the canonical correlation between their outputs.

The idea behind such an approach is discarding decorrelated components while
preserving correlated ones. In a nutshell, the model tries to keep the more
significant information present in both augmented versions and to avoid
individual behaviors, such as anomalies and noise. Figure 1 depicts the
Canonical Correlation Analysis Graph Neural Network.

Figure 1: Canonical Correlation Analysis Graph Neural Network. Each sample
represents a node in a graph whose edges describe the relationship between
pairs of samples. The random graph generator $T$ produces two augmented
versions of this graph, which are employed to feed the GNN model. The output
of both versions is compared using the canonical correlation analysis, and the
network parameters are adjusted to maximize this metric.

The graph augmentation process comprises the same approach presented in [38,
30], which conducts a random feature masking and edge dropping. In this
context, each $t_{i}\sim T$ comprises a distinct view, i.e., a transformed
version of $G$, sampled at each iteration $i$.

The encoder is composed of a two-layered GNN but can be easily replaceable by
any fancier model. The target function is designed to model the learning
process as a canonical correlation maximization problem [10] using a self-
supervised approach that considers two normalized views, $\bm{Z}_{A}$ and
$\bm{Z}_{B}$, produced over randomly augmented versions of the original graph.
The objective is to maximize the correlation between these views, formalized
as follows:

${\cal
L}(\bm{Z}_{A},\bm{Z}_{B})=||\bm{Z}_{A}-\bm{Z}_{B}||_{F}^{2}+\lambda\left(||\bm{Z}_{A}^{T}\bm{Z}_{A}-\bm{I}||_{F}^{2}+||\bm{Z}_{B}^{T}\bm{Z}_{B}-\bm{I}||_{F}^{2}\right),$
(1)

where $\lambda$ is a non-negative trading-off hyperparameter, and $I$ is the
identity matrix. The left term indicates the invariance term, which is
responsible for minimizing the invariance between the two views. In contrast,
the term on the right side describes the decorrelation term, which facilitates
distinct features to capture different semantics through a regularization
procedure.

The terms in Equation 1 can be decomposed using a variance-covariance
perspective [31]. Let $\bm{s}$ be an augmented version of the graph sampled
from an input $\bm{x}$, and $\bm{Z_{s}}$ is a view of $\bm{s}$ obtained from a
decoder output. The invariance term is minimized using expectation, described
as follows:

${\cal
L}_{inv}=||\bm{Z}_{A}-\bm{Z}_{B}||_{F}^{2}=\sum_{i=1}^{N}\sum_{k=1}^{D}(z_{i,j}^{A}-z_{i,j}^{B})^{2}\cong\mathbb{E}_{\bm{x}}\left[\sum_{k=1}^{D}\mathbb{V}_{\bm{s}|\bm{x}}[\bm{Z_{s}},k]\right]*2N,$
(2)

where $\mathbb{V}$ denotes the variance. Similarly, one can formalize the
decorrelation term as follows:

${\cal
L}_{dec}=||\bm{Z}_{S}^{T}\bm{Z}_{S}-\bm{I}||_{F}^{2}=||\bm{Cov}_{\bm{s}}[\bm{z}]-I||_{F}^{2}\cong\sum_{i\neq
j}(\rho_{i,j}^{\bm{z_{s}}})^{2},\forall\bm{Z}_{S}\in\\{\bm{Z}_{A},\bm{Z}_{B}\\},$
(3)

where $\rho$ denotes the Pearson correlation coefficient, and $\bm{Cov}$
stands for the covariance matrix.

###  2.2 Prior Frame-based Graph Positional Enconding

A time-sequence based approach for graph positional encoding was recently
proposed by Passos et al. [25]. The method computes the node neighborhood by
connecting them to their $k$ prior frame nodes in time sequence and
attributing connection weights to the edges according to their distances in
this time-based space. Figure 2 depicts this idea.

Figure 2: Prior frame-based node neighborhood modeling considering $2$ prior
frames.

The calculation of the edge weight $w_{ij}$ that connects a node $i$ to a node
$j$ is performed as follows:

$w_{ij}=k+1-d_{ij},$ (4)

where $d_{ij}$ denotes the distance from node $i$ to node $j$ in a frame-step
space. Those values are stored in a distance matrix used to compute the
positional encoding of the nodes.

##  3 Canonical Cortical Graph Neural Network

This paper presents a novel self-supervised approach for training multi-modal
graph neural networks in a more biologically plausible way. In this context,
the architecture combines several concepts inspired on cortical circuits
observed in the brain [14, 9] to model memory and multimodal information
fusion with canonical correlation analysis [10], and to maximize the
correlation of the information extracted from different inputs. Figure 3
presents a general overview of the proposed model, which suggests the more
interesting procedures of the model are implemented at layer level. Therefore,
Figure 4 illustrates this in-layer process, also depicting the behavior of
each operation.

Figure 3: Canonical Cortical Graph Neural Network.

Figure 4: Cortical graph layer. The top and middle frames describe the symbols
and operations employed in the cortical graph block, respectively. The bottom
block depicts the layer’s pipeline.

A formal description of the procedures depicted in Figure 4 and performed
inside each cortical layer is provided bellow. Firstly, one should compute the
audio $f_{a}$, visual $f_{v}$, memory $f_{m}$, and modulation $f_{w}$ filters
as follow:

$f_{a}=\sigma\left({\bm{W}_{a}h_{a}+b_{a}}\right),$ (5)

$f_{v}=\sigma\left({\bm{W}_{v}h_{v}+b_{v}}\right),$ (6)

$f_{m}=\sigma\left({\bm{W}_{m}[h_{a},h_{v}]+b_{m}}\right),$ (7)

and

$f_{w}=\sigma\left({\bm{W}_{w}[h_{a},h_{v}]+b_{w}}\right),$ (8)

where $\sigma$ stands for the Sigmoid function, $\bm{W}_{a}$, $\bm{W}_{v}$,
$\bm{W}_{m}$, and $\bm{W}_{w}$, are the weight matrices for the audio, visual,
memory, and modulation filters, respectively, and $b_{a}$, $b_{v}$, $b_{m}$,
and $b_{w}$ are the biases for the audio, visual, memory, and modulation
filters, respectively, while $[h_{a},h_{v}]$ denotes the concatenation of the
audio $h_{a}$ and the visual $h_{v}$ graph convolution outputs. Further, the
pre-modulation $\rho$ is computed as follows:

$\rho=tanh\left({\bm{W}_{\rho}[h_{a},h_{v}]+b_{\rho}}\right),$ (9)

where $\bm{W}_{\rho}$ and $b_{\rho}$ are the pre-modulation weight matrix and
bias, respectively. Finally, the modulation $\omega$ is computed as follows:

$\omega=f_{w}\otimes\rho,$ (10)

where $\otimes$ stands for the dot product.

The memory $\mu$ is the more tricky updating since it considers both
“forgetting” irrelevant memories using the memory filter $f_{m}$ and
introducing new experiences presented in the modulation $\omega$. Moreover,
the operation is performed individually for each node $n\in\\{0,\dots,N\\}$
since they are connected in a logical temporal sequence established by the
prior frame positional encoding [25], described as follows:

$\mu^{n}=\omega^{n}+\left(f_{m}^{n}\otimes\mu^{n-1}\right),$ (11)

where $\mu^{n}$, $\omega^{n}$, and $f_{m}^{n}$ are the node’s $n$ memory,
modulation, and memory filter. In the sequence, the memory is updated as
follows:

$\mu=tanh\left({\bm{W}_{\mu}\mu+b_{\mu}}\right),$ (12)

where $\bm{W}_{\mu}$ and $b_{\mu}$ are the memory weight matrix and bias,
respectively. Finally, the layer output, i.e., the new node representation of
the audio and visual graphs $h_{a}^{\prime}$ and $h_{v}^{\prime}$,
respectively, are computed as follows:

$h_{a}^{\prime}=\mu\otimes f_{a}$ (13)

and

$h_{v}^{\prime}=\mu\otimes f_{v}.$ (14)

Finally, $h_{a}^{\prime}$ and $h_{v}^{\prime}$ become the the node
representations of the subsequent layer audio and visual graphs, respectively,
in case of an intermediate layer. Regarding the output layer, the model
performs the canonical correlation analysis between $h_{a}^{\prime}$ and
$h_{v}^{\prime}$ using Equation 1 and backpropagates this value to optimize
the network parameters.

##  4 Methodology

This section describes the dataset and configuration employed during the
experiments.

###  4.1 AV ChiME3 Dataset

The dataset used in this paper aims to combine environmental information,
i.e., audio and visuals, to train an efficient model for enhancing and
amplifying clean audio signals considering multimodal data. The dataset
comprises triples composed of image, clean audio, and noisy audio signals,
from which the image and noisy audio denote the model’s input while the clean
signal stands for the desired output, i.e., the instance target in the context
of supervised learning. The videos are extracted from Grid [11] dataset, in
which different classes of noises (public transport, pedestrian area, street
junction, cafe) with signal to noise ratios (SNR) ranging from -12 to 12dB
extracted from ChiME3 [6] are introduced, composing the AV ChiME3 [5] dataset.
Further, the samples are preprocessed to improve the sentence alignment and
incorporate multiple visual frames to include temporal data. In total, the
dataset contains $989$ sequences from $5$ different speakers, described as one
black male, two white males, and two white females. Each sequence comprises
$48$ frames, summing up to a total of $47,472$ synchronized triples of
samples.

####  4.1.1 Audio feature extraction

Log-FB vectors were employed to extract both clean and noisy audio features.
The technique samples the audio signal at $22,050$kHz for further segmenting
it into $M$ $16$ms frames with $800$ samples per frame and $62.5\%$ increment
rate. Furthermore, it uses a hamming window and Fourier transformations to
produce a 2048-bin power spectrum. Moreover, it employs a logarithmic
compression to obtain the $22$-dimensional log-FB signals.

####  4.1.2 Visual feature extraction

The samples extracted to generate the visual set of features were obtained
using an encoder-decoder architecture over the Grid Corpus dataset. Lip-
regions were detected using Viola-Jones [32] algorithm, for further tracking
the frame sequence using the method proposed in [27]. Additionally, a manual
effort was employed to randomly inspect the sentences, ensuring good lip
tracking [1]. The encoder-decoder approach is then used to create vectors of
pixel intensities, in which the $50$ first components are vectorized in a
zigzag order and then interpolated to match the equivalent audio sequence.

###  4.2 Experimental Setup

The experiments conducted in this paper compare the proposed Canonical
Cortical Graph Neural Networks against the recent state-of-the-art model CCA-
GNN for the task of multimodal clean audio reconstruction based on noisy audio
and visual features, extracted using logFB and an encoder-decoder approach,
respectively, as described in Section 4.1. Both models comprise a similar
architecture composed of two hidden layers, the first comprising $512$ and the
second $256$ neurons. The parameters were selected empirically. The learning
is conducted to maximize the canonical correlation analysis for coherent
feature extraction during $200$ epochs using the Adam optimizer with a
learning rate of $10^{-3}$ and a trading-off parameter of $\lambda=0.0001$
(Equation 1). At the same time, the hyperparameters of the CCA-GNN follow the
configuration employed in [25]. Finally, the graphs are generated considering
eight distinct prior-frame scenarios, i.e., $k\in[3,5,7,10,15,20,25,30]$.
Notice that the plots provided in experimental results comprise only
$k\in[3,10,30]$ to illustrate the difference between a low, medium, and a high
number of neighbors/prior frames.

After maximizing the canonical correlation analysis between the two channels,
the features extracted by the networks are employed to feed a dense layer
responsible for reconstructing the clean signal by minimizing the mean squared
error as the cost function. The model is optimized during $2,000$ epochs using
the Adam optimizer with a learning rate of $0.005$ and a weight decay of
$0.0004$.

The dataset was divided into $20$ folds to provide an in-depth statistical
analysis. Each fold comprises $50$ sequences of $48$ frames each, summing up
to $2,400$ samples per fold. As stated in Section 4.1, the dataset is formed
by three subsets: (i) clean audio, (ii) noisy audio, and (iii) clean visual.
The noisy audio and the clean visual input are used to feed the multimodal
GNNs, while the clean audio is considered the reconstruction target. Finally,
each fold is split into train, validation, and test sets, following the
proportions of $60\%$, $20\%$, and $20\%$, respectively. The Wilcoxon signed-
rank test [33] with $5\%$ of significance was considered for statistical
evaluation.

##  5 Experiments

This section exploits the superiority of the proposed Canonical Cortical Graph
Neural Networks over the state-of-the-art CCA-GNN. The experimental results
consider the contexts of feature extraction analysis, clean audio signal
reconstruction, as well as neuronal activation and energy efficiency.

###  5.1 Feature Extraction Analysis

This section explores the task of self-supervised feature extraction in terms
of canonical correlation analysis. The idea consists of extracting correlated
features considering both the audio and visual channels, contributing to
better features for clean audio reconstruction. In this context, the Canonical
Cortical Graph Neural Networks, presented in Figure 5 as Cortical, showed a
performance $75\%$ higher than the CCA-GNN, on average, considering a small,
medium, and high neighborhood, i.e., $k\in[3,10,30]$.

Figure 5: Canonical correlation analisys maximization.

###  5.2 Clean Signal Reconstruction

Figure 7 provides some insights regarding the convergence of the model
regarding the Mean Squared Error (MSE) over the testing set reconstruction. In
this context, one can observe that (i) the proposed model (Cortical) converges
a bit faster than CCA-GNN (the blue line when $k=3$) and remain the best
result until completing the $2,000$ epochs; (ii) the convergence is faster for
small values of $k$, i.e., $k=3$, for both Cortical and CCA-GNN; (iii) the
proposed model presents an overfitting behavior for larger values of $k$,
described by the orange ($k=10$) and green ($k=30$) lines.

30

Figure 6: Clean audio reconstruction error convergence considering the
proposed model (Cortical) and CCA-GNN architectures based on the noisy audio
and clean visuals.

Further, Table 1 provides the final MSE values over the testing set
considering eight distinct $k$ scenarios, i.e., $k\in[3,5,7,10,15,20,25,30]$.
Notice bold values stand for the best values considering the Wilcoxon signed-
rank test [33] with a significance of $5\%$. Such results show that the
proposed model presents a better behavior when exposed to a reduced number of
neighbors, i.e., the historical information is reduced, obtaining the lowest
MSE overall over this scenario. This result can be explained by the memory
implemented in the architecture, i.e., since the memory tries to model the
predictions based on past frames, a longer temporal sequence makes this
information ambiguous, leading the model to an extra exposure to past
instances, thus overfitting. The idea is reinforced by the CCA-GNN approach,
where an opposite behavior is observed in most cases, i.e., since CCA-GNN does
not implement a memory, higher numbers of neighbors usually lead to lower
prediction errors.

Table 1: Average Mean Squared Error and standard deviation over Canonical
Cortical GNN and CCA-GNN considering clean audio reconstruction given noisy
audio and visual inputs over two block layers network architecture.

Neighbors | CCA GNN | Cortical
---|---|---
3 | $0.0204\pm 0.0048$ | $\bm{0.0188\pm 0.0041}$
5 | $0.0208\pm 0.0049$ | $0.0200\pm 0.0057$
7 | $0.0214\pm 0.0052$ | $0.0243\pm 0.0067$
10 | $0.0216\pm 0.0052$ | $0.0282\pm 0.0059$
15 | $0.0216\pm 0.0048$ | $0.0278\pm 0.0049$
20 | $0.0205\pm 0.0045$ | $0.0266\pm 0.0048$
25 | $0.0200\pm 0.0045$ | $0.0262\pm 0.0056$
30 | $0.0199\pm 0.0045$ | $0.0246\pm 0.0058$

Finally, Figure 7 depicts some examples of clean audio reconstruction for both
models. Notice that the Canonical Cortical Graph Neural Networks with a
reduced number of past frames, i.e., $k=3$, obtained almost perfect
reconstructions of the clean signal in both cases, which reinforces the idea
that the memory implementation replaces the necessity of longer temporal
information, described by a higher number of neighbors. Figure 7(b) shows that
CCA-GNN is also capable of producing good representations when then the number
of past frames is big enough, i.e., $k=30$, even though the reconstruction is
not as good as the proposed method.

(a) (b)

Figure 7: Two examples of clean audio signal reconstruction considering the
proposed approach (Cortical) and the baseline, i.e., CCA-GNN.

###  5.3 State-of-the-art Comparison

Table 2 provides a comparison of the Canonical Cortical Graph Neural Network
against the state-of-the-art results reported in recent works using the AV
ChiME3 dataset and multimodal approaches for audio-visual speech enhancement,
i.e., CCA-GNN and CCA Multilayer Perceptron (CCA-MLP) [25], a Long-Short Term
Memory (LSTM) [3] and a Multilayer Perceptron (MLP) [5] based approaches, as
well as a canonical correlation-based short-time objective intelligibility
deep learning (CC-STOI DL) [17] method. Notice the proposed approach provided
the most accurate results, outperformed all the compared tecniques.

Table 2: Comparison of the Canonical Cortical Graph Neural Network against
state-of-the-art results reported in recent approaches for audio-visual speech
reconstruction in terms of ean squared error.

Technique | Best value
---|---
Cortical (ours) | $0.0188$
CCA-GNN | $0.0199$
CCA-MLP [25] | $0.0189$
MLP [5] | $0.0200$
LSTM [3] | $0.0780$
CC-STOI DL [17] | $1.2770$

###  5.4 Neuronal Activation Analysis

The neuronal activation rate analysis is fundamental for future audio-visual
hearing aids since the metric reflects the model’s energy efficiency, a
critical feature considering energy-constrained environments like hearing aids
and embedded devices. In this context, reducing the firing rate directly
implies a reduction in energy consumption. Figure 8 depicts the neuron
activation rate over the intermediate block layers concerning the audio and
visual channels. Notice that both audio and video channels obtained pretty
similar values concerning the Cortical model, suggesting that the two tracks
contribute almost equally to the results.

(a) (b)

Figure 8: Neuron activation rate considers the multimodal architecture’s
hidden block over (a) noisy audio and (b) visual channels.

A numerical representation is presented in Table 3, which comprises a more
complete set of neighbors (past frames), i.e., $k=[3,5,7,10,15,20,25]$. The
results reinforce the idea that the proposed model relies almost equally upon
both noisy audio and visual modalities for the final output since their areas
under the curve are practically the same for all cases, except for $k=10$,
with an irrelevant difference, i.e., $256$ and $255$ for noisy audio and
visual, respectively. The results also show that the proposed model
outperforms the baseline in the context of neuronal activation and,
consequently, energy consumption in every possible scenario, showing itself as
a suitable approach for speech enhancement in future audio-visual hearing aid
devices considering both accuracy and energy performance.

Table 3: Area under the curve considering the multimodal architecture’s hidden
block.

| Audio | Visual
---|---|---
Neighbors | Cortical | CCA-GNN | Cortical | CCA-GNN
$3$ | $255$ | $276$ | $255$ | $291$
$5$ | $253$ | $265$ | $253$ | $288$
$7$ | $256$ | $266$ | $256$ | $288$
$10$ | $256$ | $268$ | $255$ | $290$
$15$ | $255$ | $268$ | $255$ | $288$
$20$ | $258$ | $267$ | $258$ | $285$
$25$ | $251$ | $267$ | $251$ | $288$
$30$ | $255$ | $260$ | $255$ | $280$

##  6 Conclusion

This paper proposed a novel self-supervised method for multimodal correlated
feature extraction through canonical correlation analysis maximization. The
proposed model comprises a block-based neural network, where each block
comprises two Graph Neural Networks layers, i.e., one for noisy audio and the
other for the visual features, a memory, and a set of operations to filter,
insert, delete, and modulate the input signals. Such operations are inspired
in recent discoveries related to cortical cells and their interactions.

Experiments were conducted over the AV ChiME3 dataset, designed for the task
of multimodal clean audio reconstruction considering noisy audio and clean
visual instances, compared the proposed approach against the CCA-GNN, a
similar state-of-the-art method proposed recently for the task. Results show
that the proposed Canonical Cortical GNN provides more coherent and better-
quality features, reaching higher values of canonical correlation analysis.
The proposed approach also obtained more accurate reconstructions, generating
cleaner reconstructions. Moreover, the model delivers higher efficiency in
terms of energy, evaluated by neurons’ firing rate. Finally, it also showed
itself to be less dependent on a more extended prior-frame sequence, i.e.,
high values of $k$, since the memory can store and track temporal information.

Regarding future work, we aim to extend a similar cortical-based architecture
to Convolutional Neural Networks and applications to two-dimensional data. We
also aim to implement the model on chips for training and inference for
possible future implementation in hearing aid devices for AV speech
enhancement.

## References

  * [1] Abel, A., Marxer, R., Barker, J., Watt, R., Whitmer, B., Derleth, P., Hussain, A.: A data driven approach to audiovisual speech mapping. In: International Conference on Brain Inspired Cognitive Systems. pp. 331–342. Springer (2016)
  * [2] Adeel, A., Ahmad, J., Hussain, A.: Real-time lightweight chaotic encryption for 5g iot enabled lip-reading driven secure hearing-aid. arXiv preprint arXiv:1809.04966 (2018)
  * [3] Adeel, A., Ahmad, J., Larijani, H., Hussain, A.: A novel real-time, lightweight chaotic-encryption scheme for next-generation audio-visual hearing aids. Cognitive Computation 12(3), 589–601 (2020)
  * [4] Adeel, A., Gogate, M., Hussain, A.: Contextual deep learning-based audio-visual switching for speech enhancement in real-world environments. Information Fusion 59, 163–170 (2020)
  * [5] Adeel, A., Gogate, M., Hussain, A., Whitmer, W.M.: Lip-reading driven deep learning approach for speech enhancement. IEEE Transactions on Emerging Topics in Computational Intelligence (2019)
  * [6] Barker, J., Marxer, R., Vincent, E., Watanabe, S.: The third ‘chime’speech separation and recognition challenge: Dataset, task and baselines. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU). pp. 504–511. IEEE (2015)
  * [7] Benesty, J., Chen, J., Habets, E.A.: Speech enhancement in the STFT domain. Springer Science & Business Media (2011)
  * [8] Bokhari, M.U., Hasan, F.: Multimodal information retrieval: Challenges and future trends. International Journal of Computer Applications 74(14) (2013)
  * [9] Capone, F., Paolucci, M., Assenza, F., Brunelli, N., Ricci, L., Florio, L., Di Lazzaro, V.: Canonical cortical circuits: current evidence and theoretical implications. Neuroscience and Neuroeconomics 5, 1–8 (2016)
  * [10] Chang, X., Xiang, T., Hospedales, T.M.: Scalable and effective deep cca via soft decorrelation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1488–1497 (2018)
  * [11] Cooke, M., Barker, J., Cunningham, S., Shao, X.: An audio-visual corpus for speech perception and automatic speech recognition. The Journal of the Acoustical Society of America 120(5), 2421–2424 (2006)
  * [12] da Costa, K.A., Papa, J.P., Passos, L.A., Colombo, D., Del Ser, J., Muhammad, K., de Albuquerque, V.H.C.: A critical literature survey and prospects on tampering and anomaly detection in image data. Applied Soft Computing p. 106727 (2020)
  * [13] Gogate, M., Dashtipour, K., Adeel, A., Hussain, A.: Cochleanet: A robust language-independent audio-visual model for real-time speech enhancement. Information Fusion 63, 273–285 (2020)
  * [14] Grossberg, S.: A canonical laminar neocortical circuit whose bottom-up, horizontal, and top-down pathways control attention, learning, and prediction. Frontiers in Systems Neuroscience 15 (2021)
  * [15] Helvik, A.S., Jacobsen, G., Hallberg, L.R.: Psychological well-being of adults with acquired hearing impairment. Disability and rehabilitation 28(9), 535–545 (2006)
  * [16] Huang, A.R., Deal, J.A., Rebok, G.W., Pinto, J.M., Waite, L., Lin, F.R.: Hearing impairment and loneliness in older adults in the united states. Journal of Applied Gerontology 40(10), 1366–1371 (2021)
  * [17] Hussain, T., Diyan, M., Gogate, M., Dashtipour, K., Adeel, A., Tsao, Y., Hussain, A.: A novel speech intelligibility enhancement model based on canonicalcorrelation and deep learning. arXiv preprint arXiv:2202.05756 (2022)
  * [18] Körding, K.P., König, P.: Supervised and unsupervised learning with two sites of synaptic integration. Journal of computational neuroscience 11(3), 207–215 (2001)
  * [19] Kramer, S.E., Kapteyn, T.S., Festen, J.M., Tobi, H.: Factors in subjective hearing disability. Audiology 34(6), 311–320 (1995)
  * [20] Kumar, L.A., Renuka, D.K., Rose, S.L., Wartana, I.M., et al.: Deep learning based assistive technology on audio visual speech recognition for hearing impaired. International Journal of Cognitive Computing in Engineering 3, 24–30 (2022)
  * [21] Ngiam, J., Khosla, A., Kim, M., Nam, J., Lee, H., Ng, A.Y.: Multimodal deep learning. In: ICML (2011)
  * [22] Noble, W.: Self-assessment of hearing and related function. Wiley-Blackwell (1998)
  * [23] Organization, W.H., et al.: Hearing screening: considerations for implementation (2021)
  * [24] Passos, L.A., Khubaib, A., Raza, M., Adeel, A.: Multimodal speech enhancement using burst propagation. arXiv preprint arXiv:2209.03275 (2022)
  * [25] Passos, L.A., Papa, J.P., Del Ser, J., Hussain, A., Adeel, A.: Multimodal audio-visual information fusion using canonical-correlated graph neural network for energy-efficient speech enhancement. Information Fusion (2022)
  * [26] Payeur, A., Guerguiev, J., Zenke, F., Richards, B.A., Naud, R.: Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature neuroscience 24(7), 1010–1019 (2021)
  * [27] Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. International journal of computer vision 77(1), 125–141 (2008)
  * [28] Santos, D.F., Pires, R.G., Passos, L.A., Papa, J.P.: Ddipnet and ddipnet+: Discriminant deep image prior networks for remote sensing image classification. In: 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. pp. 2843–2846. IEEE (2021)
  * [29] de Souza, R.W., Silva, D.S., Passos, L.A., Roder, M., Santana, M.C., Pinheiro, P.R., de Albuquerque, V.H.C.: Computer-assisted parkinson’s disease diagnosis using fuzzy optimum-path forest and restricted boltzmann machines. Computers in Biology and Medicine 131, 104260 (2021)
  * [30] Thakoor, S., Tallec, C., Azar, M.G., Munos, R., Veličković, P., Valko, M.: Bootstrapped representation learning on graphs. arXiv preprint arXiv:2102.06514 (2021)
  * [31] Tian, Y., Chen, X., Ganguli, S.: Understanding self-supervised learning dynamics without contrastive pairs. arXiv preprint arXiv:2102.06810 (2021)
  * [32] Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001. vol. 1, pp. I–I. Ieee (2001)
  * [33] Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bulletin 1(6), 80–83 (1945). https://doi.org/https://doi.org/10.2307/3001968
  * [34] Xu, Y., Du, J., Dai, L.R., Lee, C.H.: A regression approach to speech enhancement based on deep neural networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23(1), 7–19 (2014)
  * [35] Yu, W., Zhou, J., Wang, H., Tao, L.: Setransformer: speech enhancement transformer. Cognitive Computation 14(3), 1152–1158 (2022)
  * [36] Zeng, N., Wu, P., Wang, Z., Li, H., Liu, W., Liu, X.: A small-sized object detection oriented multi-scale feature fusion approach with application to defect detection. IEEE Transactions on Instrumentation and Measurement 71, 1–14 (2022)
  * [37] Zhang, H., Wu, Q., Yan, J., Wipf, D., Philip, S.Y.: From canonical correlation analysis to self-supervised graph neural networks. In: Thirty-Fifth Conference on Neural Information Processing Systems (2021)
  * [38] Zhu, Y., Xu, Y., Yu, F., Liu, Q., Wu, S., Wang, L.: Deep graph contrastive representation learning. arXiv preprint arXiv:2006.04131 (2020)