Predicting the Silent Majority on Graphs: Knowledge Transferable Graph
  Neural Network
Graphs consisting of vocal nodes ("the vocal minority") and silent nodes
("the silent majority"), namely VS-Graph, are ubiquitous in the real world. The
vocal nodes tend to have abundant features and labels. In contrast, silent
nodes only have incomplete features and rare labels, e.g., the description and
political tendency of politicians (vocal) are abundant while not for ordinary
people (silent) on the twitter's social network. Predicting the silent majority
remains a crucial yet challenging problem. However, most existing
message-passing based GNNs assume that all nodes belong to the same domain,
without considering the missing features and distribution-shift between
domains, leading to poor ability to deal with VS-Graph. To combat the above
challenges, we propose Knowledge Transferable Graph Neural Network (KT-GNN),
which models distribution shifts during message passing and representation
learning by transferring knowledge from vocal nodes to silent nodes.
Specifically, we design the domain-adapted "feature completion and message
passing mechanism" for node representation learning while preserving domain
difference. And a knowledge transferable classifier based on KL-divergence is
followed. Comprehensive experiments on real-world scenarios (i.e., company
financial risk assessment and political elections) demonstrate the superior
performance of our method. Our source code has been open sourced.
Wendong Bi, Bingbing Xu, Xiaoqian Sun, Li Xu, Huawei Shen, Xueqi Cheng
# Predicting the Silent Majority on Graphs:

###### Abstract.

Graphs consisting of vocal nodes ("the vocal minority") and silent nodes ("the silent majority"), namely VS-Graph, are ubiquitous in the real world. The vocal nodes tend to have abundant features and labels. In contrast, silent nodes only have incomplete features and rare labels, e.g., the description and political tendency of politicians (vocal) are abundant while not for ordinary civilians (silent) on the twitter's social network. Predicting the silent majority remains a crucial yet challenging problem. However, most existing Graph Neural Networks (GNNs) assume that all nodes belong to the same domain, without considering the missing features and distribution-shift between domains, leading to poor ability to deal with VS-Graph. To combat the above challenges, we propose Knowledge Transferable Graph Neural Network (KTGNN), which models distribution-shifts during message passing and learns representation by transferring knowledge from vocal nodes to silent nodes. Specifically, we design the domain-adapted "feature completion and message passing mechanism" for node representation learning while preserving domain difference. And a knowledge transferable classifier based on KL-divergence is followed. Comprehensive experiments on real-world scenarios (i.e., company financial risk assessment and political elections) demonstrate the superior performance of our method. Our source code has been open-sourced1.

Footnote 1: The source code is available at [https://github.com/wendongbi/KT-GNN](https://github.com/wendongbi/KT-GNN) +
Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*]Footnote †: [leftmargin=*] †: [leftmargin=*]Footnote †: [leftmargin=*] †: [leftmargin=*
While the silent nodes (**"the silent majority"**) have rare labels and incomplete features. VS-Graph is also common in other real-world scenarios, such as celebrities and ordinary netizens in social networks. Meanwhile, predicting the silent majority on VS-Graph is important yet challenging.

Recently, Graph neural networks (GNNs) have achieved state-of-the-art performance on graph-related tasks. Generally, GNNs follow the message-passing mechanism, which aggregates neighboring nodes' representations iteratively to update the central node representation. However, the following three problems lead to the failure of GNNs to predict the silent majority: 1) the data distribution-shift problem exists between vocal nodes and silent nodes, e.g., listed companies have more assets and cash flows and therefore show different distribution from unlisted companies in terms of attributes, which is rarely considered by the previous GNNs; 2) the feature missing problem exists and can not be solved by traditional heuristic feature completion strategies due to the above data distribution-shift; 3) the lack of labels for silent majority hinders an ideal model, and directly training models on vocal nodes and silent nodes concurrently leads to poor performance in predicting the silent majority due to the distribution-shift.

To solve the aforementioned challenges, we propose **K**nowledge **T**ransferable **G**raph **N**eural **N**etwork (KTGNN), which targets at learning representations for the silent majority on graphs via transferring knowledge adaptively from vocal nodes. KTGNN takes domain transfer into consideration for the whole learning process, including feature completion, message passing, and the final classifier. Specifically, Domain-Adapted Feature Complementor (DAFC) and Domain-Adapted Message Passing (DAMP) are designed to complement features for silent nodes and conduct message passing between vocal nodes and silent nodes while modeling and preserving the distribution-shift. With the learned node representations from different domains, we propose the Domain Transferable Classifier (DTC) based on KL-divergence minimization to predict the labels of silent nodes. Different from existing transfer learning methods that aim to learn domain-invariant representations, we propose to transfer the parameters of classifiers from vocal domain to silent domain rather than forcing a single classifier to capture domain-invariant representations. Comprehensive experiments on two real-world scenarios (i.e., company financial risk assessment and political elections) show that our method gains significant improvements on silent node prediction over SOTA GNNs.

The contributions of this paper are summarized as follows:

* We define a practical and widespread VS-Graph, and solve the new problem of predicting silent majority, one important and realistic AI application.
* To predict silent majority, we design a novel Knowledge Transferable Graph Neural Network (KTGNN) model, which is enhanced by the Domain-Adapted Feature Complementor and Messaging Passing modules that preserve domain-difference as well as the KL-divergence minimization based Domain Transferable Classifier.
* Comprehensive experiments on two real-world scenarios (i.e., company financial risk assessment and political elections) show the superiority of our method, e.g., AUC achieves an improvement of nearly 6% on company financial risk assessment compared to the current SOTA methods.

## 2. Preliminary

In this section, we give the definitions of important terminologies and concepts appearing in this paper.

### Graph Neural Network

Graph Neural Networks (GNNs) aim at learning representations for nodes on the graph. Given a graph \(\mathbf{G}(V,E,X,Y)\) as input, where \(V=\{v_{i}|v_{i}\in V\}\) denotes the node set and \(E=\{e_{ij}=(v_{i},v_{j})|v_{i}\text{ and }\ v_{j}\}\) is connected. \(N=|V|\) is the number of nodes, \(X\in\mathbb{R}^{N\times D_{in}}\) is the feature matrix and \(Y\in\mathbb{R}^{N\times 1}\) is the labels of all nodes. Then GNNs update node representations by stacking multiple graph convolution layers and current mainstream GNNs follow the messaging passing mechanism, each layer of GNNs updates node representations with the following function:

\[h_{i}^{(I)}=\mathbf{U}\left(h_{i}^{(I-1)},\mathbf{M}(\{h_{i}^{(I-1)},h_{j}^{( I-1)}|v_{j}\in\mathcal{N}(v_{i})\})\right)\]

where \(h_{i}^{(I)}\) is the node representation vector at \(I\)-th layer of GNN, \(\mathbf{M}\) denotes the message function of aggregating neighbor's features, and \(\mathbf{U}\) denotes the update functions with the neighborhood messages and central node feature as inputs. By stacking multiple layers, GNNs can aggregate information from higher-order neighbors.

### Problem Definition: Silent Node Classification on the VS-Graph

In this paper, we propose the problem of silent node prediction on VS-Graph and we mainly focus on the node classification problem. First, we give the definition of VS-Graph:

Definition 2.1 (VS-Graph).: Given a VS-Graph \(\mathbf{G}^{\text{BS}}(V,E,X,Y,\Psi)\), where \(V=V^{\text{ococal}}\cup V^{silent}\) is the node-set, including all vocal nodes

Figure 1. Examples of silent node classification, where (a) and (b) show the difference between silent node classification vs. traditional node classification. (c) and (d) show two real-world VS-Graphs.
and silent nodes. \(E=\{e_{ij}=(v_{i},e_{j})|v_{i},e_{j}\in V^{local}\cup V^{silent}\}\) is the edge set. \(X=[X^{local};\ X^{silent}]\) is the attribute matrix for all nodes and \(Y=[Y^{local};\ Y^{silent}]\) is the class label of all nodes (\(y_{i}=-1\) denotes the label of \(v_{i}\) is unavailable). \(\Psi:V\rightarrow\{vocal,silent\}\) is the population indicator that maps a node to its node population (i.e., vocal nodes, silent nodes).

\[\Psi(v_{i})=\begin{cases}1&\text{if $v_{i}$ is a silent node}\\ 0&\text{if $v_{i}$ is a vocal node}\end{cases} \tag{1}\]

For a VS-Graph, the attributes and labels of vocal nodes (i.e., \(X^{local}\), \(Y^{local}\)) are complete. However, \(Y^{silent}\) is rare and \(X^{silent}\) is incomplete. Specifically, for vocal node \(v_{i}\in V^{local}\), its attribute vector \(x_{i}=[x_{i}^{o}|x_{i}^{u}]\), where \(x_{i}^{o}\in\mathbb{R}^{Do}\) is part of attributes observable for all nodes and \(x_{i}^{u}\in\mathbb{R}^{Do}\) is part of attributes observable for silent nodes while observable for vocal nodes. For silent node \(v_{i}\in V^{silent}\), its attribute vector \(x_{i}=[x_{i}^{o}||\mathbf{0}]\), where \(x_{i}^{o}\in\mathbb{R}^{Do}\) is the observable part and \(\mathbf{0}\in\mathbb{R}^{Do}\) means the unobservable part which is complemented by \(0\) per-dimension initially. Then the problem of Silent Node Classification on VS-Graph is defined as follows:

Definition 2 (Silent Node Classification on the VS-Graph).: Given a VS-Graph \(\mathbf{G}^{\text{BS}}(V,E,X,Y,\Psi)\), where \(|V^{local}|<<|V^{silent}|\) and the attributes of nodes from the two populations (i.e., \(X^{local}\) and \(X^{silent}\)) belong to different domains with distinct distributions \(\mathcal{P}^{local}\neq\mathcal{P}^{silent}\). Under these settings, the target is to predict the labels (\(Y^{silent}\)) of silent nodes (\(V^{silent}\)) with the support of a small set of vocal nodes (\(V^{local}\)) with complete but out-of-distribution information.

## 3. Exploratory Analysis

**"The Silent Majority" in Real-world Graphs:** To demonstrate that the distribution-shift problem between vocal nodes and silent nodes does exist in the real-world, we choose two representative real-world scenarios (i.e., political social network, company equity network) as examples and conduct comprehensive analysis on the two real-world VS-Graphs. And the observations show that there exists a significant distribution-shift between vocal nodes and silent nodes on their shared feature dimensions. The detailed information of the datasets is summarized in Sec. 5.1.

### Single-Variable Analysis

We analyze the distribution of single-variable on the real-world Company VS-Graph, including the listed company nodes and unlisted company nodes (more details in Sec. 5.1). All node attributes of this dataset are from real-world companies and have practical physical meanings (e.g., register capital, actual capital, staff number), which are appropriate for statistical analysis of a single dimension.

Instead of directly calculating the empirical distribution, we visualize the distribution with box-plots in a more intuitive manner. We select three important attributes (i.e., register capital, actual capital, staff number) and present the box plot of all nodes distinguished by their labels (i.e., risky company or normal company) and populations (i.e., listed company or unlisted company) in the Fig. 2. We observe that there exists **significant distribution-shift** between listed companies (vocal nodes) and unlisted companies (silent nodes), which confirms our motivations. Considering that there are hundreds of attributes, we only present three typical attributes correlated to the company assets here due to the space limitation.

### Multi-Variable Visualization

The above single-variable analysis demonstrates that there exists significant domain difference on the certain dimension of attributes between vocal nodes and silent nodes. To further demonstrate the domain difference of all attributes, we also visualize by scatter plot on all attributes with t-SNE Algorithm (Zhu et al., 2017). Specifically, we first project the \(D_{in}\)-dimensional attributes to 2-dimensional vectors and then present the scatter plot on the 2D plane.

As shown in Fig. 3, we respectively visualize the two VS-Graph Datasets: Twitter (a political social network) and Company (a company equity network). Specifically, Fig. 3 (a) shows the nodes of

Figure 3. T-SNE visualization of the raw node features distinguished by their populations (vocal\(\rightarrow\)orange, silent\(\rightarrow\)cyan) or labels (binary classes: positive\(\rightarrow\)red, negative\(\rightarrow\)blue). Note that we only visualize the observable attributes \(X^{o}\) (dimensions shared by vocal and silent nodes).

Figure 2. Box-plot of single-variable distribution for company equity graph dataset. Each box-plot actually represents the conditional distribution \(P(X|Y,\mathcal{O})\), where the Y-axis is the value of attributes after the log function.
different populations in the Twitter dataset (two colors denote vocal nodes and silent nodes); Fig. 3 (b) shows the vocal nodes (politicians) of different classes (i.e., political parties, including democrat and republican) and Fig. 3 (c) shows the silent nodes (civilians) of different classes. For the company dataset, Fig. 3 (d) shows the nodes of different populations (two colors denote vocal nodes and silent nodes); Fig. 3 (e) shows the vocal nodes (listed companies) of different classes (i.e., risky/normal company) and Fig. 3 (c) shows the silent nodes (unlisted companies) of different classes. Fig. 3 (a) and (d) demonstrate that nodes from different populations have distinct distributions, which are reflected as different clusters with distinct shapes and positions. From Fig. 3 (b) and (e), we observe that the vocal nodes of different classes have similar distributions, which are reflected as indistinguishable clusters mixed together, and the silent nodes of different classes Fig. 3 (c) and (f) have similar phenomena as vocal nodes. All these findings demonstrate that there exists a significant distribution-shift between vocal nodes and silent nodes on these real-world VS-Graphs.

## 4. Methodology

We propose the Knowledge Transferable Graph Neural Network (KTGNN) to learn effective representations for silent nodes on VS-Graphs.

### Model Overview

We first present an overview of our KTGNN model in Fig. 4. KTGNN can be trained through an end-to-end manner on the VS-Graph and is composed of three main components: (1) Domain Adapted Feature Complementary; (2) Domain Adapted Message Passing module; (3) Domain Transferable Classifier.

### Domain Adapted Feature Complementary (DAFC)

Considering that part of the attributes are unobservable for silent nodes, we first complement the unobservable features for silent nodes with the cross-domain knowledge of vocal nodes. An intuitive idea is to complete the unobservable features of silent nodes by their vocal neighboring nodes which have complete features. However, not all silent nodes on the VS-Graph have vocal neighbors. To complete the features of all silent nodes, we propose a Domain Adapted Feature Complementary (DAFC) module which can complement the features of silent nodes by adaptively transferring knowledge from vocal nodes with several iterations.

Before completing features, we first partition the nodes on the VS-Graph into two sets: \(\mathcal{V}^{+}\) and \(\mathcal{V}^{-}\), where nodes in \(\mathcal{V}^{+}\) have complete features while nodes in \(\mathcal{V}^{-}\) have incomplete features. Initially, \(\mathcal{V}^{+}=V^{local}\) and \(\mathcal{V}^{-}=V^{silent}\). Then the DAMP module completes the features of nodes in \(\mathcal{V}^{-}\) by transferring the knowledge of nodes in \(\mathcal{V}^{+}\) to \(\mathcal{V}^{-}\) iteratively. After each iteration, we add the nodes in \(\mathcal{V}^{-}\) whose features are completed to \(\mathcal{V}^{+}\) and remove these nodes from \(\mathcal{V}^{-}\). And we set a hyper-parameter \(K\) as the max-iteration of feature completion. By setting different \(K\), the set of nodes with complete features can cover all nodes on the Graph, and we have \(V^{noal}\subset\mathcal{V}^{+}\subset V\). Fig. 4-(a) present the illustration of our DAFC module. At the first iteration, features of vocal nodes are used to complete features of incomplete silent nodes that directly connect to the vocal nodes. Given incomplete silent node \(v_{i}\in\{V^{silent}\cap\mathcal{V}^{-}\}\), its complemented unobservable feature \(\widetilde{x_{i}^{u}}\) can be calculated:

\[\widetilde{x_{i}^{u}}=\sum_{v_{j}\in\{\mathcal{N}(v_{2})\cap V^{noal}\}} \widetilde{x_{j}^{u}}\cdot f(x_{i}^{\alpha}W^{s},x_{j}^{\alpha}W^{\alpha}) \tag{2}\]

where \(\widetilde{x_{j}^{u}}\) is the calibrated variable for \(x_{j}^{u}\) by eliminating the effects of domain differences:

\[\begin{cases}\widetilde{x_{j}^{u}}=x_{j}^{u}-\Delta\widetilde{X}^{u}\cdot \sigma([x_{j}^{u}||\Delta\widetilde{X}^{u}]\cdot W^{\beta})\\ \Delta\widetilde{X}^{u}=[\widetilde{X}_{o}^{\alpha}-\widetilde{X}_{o}^{\alpha }]\cdot W^{\alpha-u}\end{cases} \tag{3}\]

where \(W^{\alpha-u}\in\mathbb{R}^{D^{\alpha}\times D^{u}}\) and \(W^{\beta}\in\mathbb{R}^{2D^{\alpha}\times D^{\mu}}\) are learnable parametric matrix, and \(\sigma\) is a \(Tanh\) activation function; \(\Delta\widetilde{X}^{u}\) is the transformed domain difference; \(\widetilde{X}_{o}^{\alpha}=\mathbb{E}_{v_{2}-V^{noal}}(x_{i}^{\alpha})\) and \(\widetilde{X}_{o}^{s}=\mathbb{E}_{v_{2}-V^{silent}}(x_{i}^{\alpha})\) are respectively the expectation of the observable features for vocal nodes and silent nodes, and \(\widetilde{X}_{o}^{\alpha}-\widetilde{X}_{o}^{s}\) represents the domain difference between vocal and silent nodes according to their observable attributes. In this part, we aim at learning the unobservable domain difference based on the observable domain difference \(\widetilde{X}_{o}^{\alpha}-\widetilde{X}_{o}^{s}\). Then for other incomplete silent nodes that do not directly connect to any vocal nodes, they will be complemented from the second iteration until the algorithm converges (reaching the max-iteration \(K\) or all silent nodes have been complemented). After iteration 1, DAFC uses complete silent nodes to further complement features for the remaining incomplete silent nodes at each iteration. During this process, the set of complete nodes \(\mathcal{V}^{+}\) expand layer-by-layer like breadth-first search (BFS), as shown in Eq.4:

\[\widetilde{x_{i}^{u}}=\sum_{v_{j}\in\{\mathcal{N}(v_{i})\cap\mathcal{V}^{+}\}} x_{j}^{u}\cdot f(x_{j}^{\alpha}W^{\alpha},x_{i}^{\alpha}W^{s}) \tag{4}\]

where \(f(\cdot)\) is the neighbor importance factor:

\[f(x_{j}^{\alpha}W^{\alpha},x_{i}^{\alpha}W^{s})=\sigma\left([x_{j}^{\alpha}W^ {\alpha}||x_{i}^{\alpha}W^{s}]\cdot a^{\alpha s}\right) \tag{5}\]

where \(\sigma\) is a \(LeakyReLU\) activation function. Note that the node set \(\{\mathcal{N}(v_{i})\cap\mathcal{V}^{+}\}\in V^{silent}\), because all silent nodes that have vocal neighbors have been complemented and merged into \(\mathcal{V}^{+}\) after iteration 1 (see Eq. 2), therefore the unobservable features \(x_{j}^{u}\) in Eq. 4 do not need to be calibrated by the domain difference factor. Finally, we get the complemented features for silent nodes \(\widetilde{x_{i}}=[x_{i}^{\alpha}||\widetilde{x_{i}^{u}}]\).

To guarantee that the learned unobservable domain difference (i.e., \(\Delta\widetilde{X}^{u}\) in Eq. 3) is actually applied in the process of the domain-adapted feature completion, we add a **Distribution-Consistency Loss**\(\mathcal{L}^{dist}\) when optimizing the DAFC module:

\[\mathcal{L}^{dist}=\left|\Delta\widetilde{X}^{u}-\left(\mathbb{E}_{v_{i}-V^{noal} }(x_{i}^{u})-\mathbb{E}_{v_{i}-V^{sitent}}(\widetilde{x_{i}^{u}})\right) \right|^{2} \tag{6}\]

### Domain Adapted Message Passing (DAMP)

According to the directions of edges based on the population types from source nodes to target nodes, we divide the message passing on a VS-Graph into four parts: (1) Messages from vocal nodes to silent nodes; (2) Messages from silent nodes to vocal nodes; (3) Messages from vocal nodes to vocal nodes; (4) Messages from silent nodes to silent nodes. Among the four directions of message passing, directions (1) and (2) are cross-domain (out-of-distribution)
message passing, while directions (3) and (4) are within-domain (in-distribution) message passing. However, out-of-distribution messages from cross-domain neighbors should not be passed to central nodes directly, otherwise, the domain difference will become noises that degrade the model performance. To solve the aforementioned problem, we design a Domain Adapted Message Passing (DAMP) module. For messages from cross-domain neighbors, we first calculate the domain-difference scaling factor and then project the OOD features of source nodes into the domain of target nodes. e.g., for edges from vocal nodes to silent nodes, we project the features of vocal nodes to the domain of silent nodes and then pass the projected features to the target silent nodes.

Specifically, the DAMP module considers two factors: **the bi-directional domain difference** and **the neighbor importance**. Given an edge \(e_{i,j}=(v_{i},v_{j})\), the message function is given as follows:

\[\mathcal{M}_{v_{i}\to v_{j}}=\widetilde{h_{i}}\cdot f^{\Psi(v_{j})}( \widetilde{h_{i}},h_{j}) \tag{7}\]

where \(\widetilde{h_{i}}\) is the source node feature calibrated by the domain difference factor and \(f^{\Psi(v_{j})}(\cdot)\) is the neighbor importance function:

\[\widetilde{h_{i}}=\mathcal{P}_{\Psi(v_{i})\rightarrow\Psi(v_{j})}(h_{i})=h_{ i}+\Delta_{i,j} \tag{8}\]

where \(\sigma\) is \(LeakyReLU\) and \(\Delta_{i,j}\) is the distribution-shift variable:

\[\Delta_{i,j}=\left\{\begin{array}{cc}0&if\ \Psi(v_{i})==\Psi(v_{j})\\ (-1)^{\Psi(v_{j})}\cdot\sigma(\{h_{i}[h_{i}][\tilde{X}^{\sigma}-\tilde{X}^{ \sigma}]\cdot\mathbf{a}^{\Psi(v_{i})}\}\cdot(\tilde{X}^{\sigma}-\tilde{X}^{ \sigma})&else\end{array}\right. \tag{9}\]

where \(\sigma\) is \(Tanh\); \(\Psi(v_{i})\) is the population indicator (see Eq. 1); \(\tilde{X}^{\sigma}=\mathbb{E}_{v_{i}\sim V^{local}}(x_{i})\) and \(\tilde{X}^{s}=\mathbb{E}_{v_{i}\sim V^{silent}}(\widetilde{x_{i}})\) are respectively the expectation of the observable features for vocal nodes and silent nodes, and \(\tilde{X}^{\sigma}-\tilde{X}^{s}\) represents the domain difference between vocal and silent nodes according to their complete attributes (the attributes of silent nodes have been complemented with DAFC). With the DAMP module, we calibrate the source nodes to the domain of target nodes while message passing, which eliminates the noises caused by the OOD features. It should be noticed that the distribution-shift variable \(\Delta_{i,j}\) only works for cross-domain message passing (\(V^{local}\rightarrow\)\(V^{silent}\) or \(V^{silent}\rightarrow\)\(V^{local}\)), and \(\Delta_{i,j}=1\) for within-domain message passing (\(V^{local}\rightarrow\)\(V^{local}\) or \(V^{silent}\rightarrow\)\(V^{silent}\)). With DAMP, we finally obtain the representations of silent nodes and vocal nodes while preserving their domain difference. Different from mainstream domain adaption methods that project the samples from source and target domains into the common space and only preserve the domain-invariance features, our DAMP method conducts message passing while preserving domain difference.

### Domain Transferable Classifier (DTC)

With DAFC and DAMP, we solve the feature incompleteness problem and obtain the representations of silent nodes and vocal nodes that preserve the domain difference. Considering that the node representations come from two distinct distributions and the data-hunry problem of silent nodes (see Sec. 3 and Sec. 2.2), we cannot directly train a good classifier for silent nodes. To solve the cross-domain problem and label scarcity problem of silent nodes, we design a novel Domain Transferable Classifier (DTC) for the silent node classification by transferring knowledge from vocal nodes. Traditional domain adaption methods usually transfer cross-domain knowledge by constraining the learned representations to retain domain-invariance information only, under the assumption that the distribution of two domains is similar, which may not be satisfied. Rather than constraining representations, DTC targets at transferring model parameters so that the knowledge of the optimized source classifier can be transferred to the target domain.

The solid rectangular box in Fig. 4-(c) shows our motivation to design DTC. Specifically, the classifier trained on source domain

Figure 4. Architecture overview of KTGNN, which includes three main components: Domain Adapted Feature Complementary (DAFC), Domain Adapted Message Passing (DAMP), and Domain Transferable Classifier (DTC).
only performs under-fitting in target domain due to domain shift (blue dotted line). Meanwhile, the classifier trained on target domain tends to overfit due to label scarcity (orange dotted line). An ideal classifier is between these two classifiers (green line). Based on this motivation (verified via experimental results in Fig.5), we transfer knowledge from both source classifier and target classifier to introduce an ideal classifier by minimizing the KL divergence between them. Specifically, as shown in Fig. 4-(c), DTC has three components, including the source classifier \(CLF^{s}\), the target classifier \(CLF^{t}\) and the cross-domain transformation module \(Trans^{s\to t}\).

To predict the silent majority, the source classifier is trained with the vocal nodes and the target domain classifier is trained with the silent nodes. the cross-domain transformation module is used to transfer the original source domain classifier to a newly generated target domain classifier (\(\overline{CLF^{t}}=Trans^{s\to t}\) (\(CLF^{s}(\cdot)\))), which takes the parameters of source domain classifier as input and then generate the parameters of a new target domain classifier. Specifically, both \(CLF^{s}=\sigma(W^{s}H)\) and \(CLF^{t}=\sigma(W^{t}H)\) are implemented by one fully-connected layer with _Sigmoid_ activation. And \(Trans^{s\to t}(W^{s})=MLP(W^{s})\) is implemented by a multi-layer perception with nonlinear transformation to give it the ability to change the shape of the discriminant boundary of the classifier.

The loss function to optimize DTC and the whole model can be divided into two parts: the KL loss \(\mathcal{L}^{kl}\) and the classification loss \(\mathcal{L}^{clf}\).

The KL loss function \(\mathcal{L}^{kl}\) is proposed to realize the knowledge transfer between the source domain and the target domain by constraining the discriminative bound of the generated classifier \(\overline{CLF^{t}}\) to locate between that of \(CLF^{s}\) and \(CLF^{t}\):

\[\mathcal{L}^{kl}=KL(\mathcal{P}^{s},\widetilde{\mathcal{P}^{s}})+KL( \mathcal{P}^{t},\widetilde{\mathcal{P}^{t}}) \tag{11}\]

where \(\mathcal{P}^{s}\in\mathbb{R}^{|V^{wood}|\times|\mathcal{C}|}\) is the output probability of \(CLF^{s}\), \(\mathcal{P}^{t}\in\mathbb{R}^{|V^{silent}|\times|\mathcal{C}|}\) is the output probability of \(CLF^{t}\), \(\widetilde{\mathcal{P}^{s}}\in\mathbb{R}^{|V^{wood}|\times|\mathcal{C}|}\) and \(\widetilde{\mathcal{P}^{t}}\in\mathbb{R}^{|V^{silent}|\times|\mathcal{C}|}\) are the output probability of \(\overline{CLF^{t}}\), \(|\mathcal{C}|\) is the number of classes and \(|\mathcal{C}|=2\) for binary classification tasks.

The classification loss \(\mathcal{L}^{clf}\) is defined as:

\[\begin{split}\mathcal{L}^{clf}=BCE(Y^{wood},\widetilde{Y}^{ wood}_{CLF^{s}})&+BCE(Y^{silent},\widetilde{Y}^{silent}_{CLF^{t}})\\ &+BCE(Y^{silent},\widetilde{Y}^{silent}_{CLF^{t}})\end{split} \tag{12}\]

where \(BCE\) is the binary cross-entropy loss function:

\[BCE(Y,\widetilde{Y},N)=\frac{1}{N}\sum_{i=1}^{N}Y_{i}\cdot\log\widetilde{Y}_{ i}+(1-Y_{i})\cdot\log(1-\widetilde{Y}_{i}) \tag{13}\]

Combined with the Distribution-Consistency Loss \(\mathcal{L}^{dict}\), our final loss function \(\mathcal{L}\) is:

\[\mathcal{L}=\mathcal{L}^{clf}+\lambda\cdot\mathcal{L}^{kl}+\gamma\cdot \mathcal{L}dist \tag{14}\]

where \(\lambda\) is a hyper-parameter to control the weight of \(\mathcal{L}^{kl}\), and we use \(\gamma=1\) in all our experiments.

## 5. Experiments

In this section, we compare KTGNN with other state-of-the-art methods on two real-world datasets in different scenarios.

### Datasets

Basic information of the real-world VS-Graphs are shown in Table 1. More details about the datasets are shown in Appendix A.

**Company:** Company dataset is a VS-Graph based on 10641 real-world companies in China (provided by TianYanCha), and the target is to classify each company into binary classes (risky/normal). On this VS-Graph, vocal nodes denote listed companies and silent nodes denote unlisted companies. The business information and equity graphs of all companies are available, while only the financial statements of listed companies can be obtained (missing for unlisted companies). Edges of this dataset indicate investment relations between companies.

**Twitter:** Following the dataset proposed by Xiao et al. (Xiao et al., 2017), we construct the Twitter VS-Graph based on data crawled from Twitter, and the target is to predict the political tendency (binary classes: democrat or republican) of civilians. On this VS-Graph, vocal nodes denote famous politicians and silent nodes denote ordinal civilians. The tweet information is available for all Twitter users, while the personal descriptions from the homepage are available only for politicians (missing for civilians). Edges of this dataset represent follow relations between Twitter users.

### Baselines and Experimental Settings

We implement nine representative baseline models, including _MLP_ as well as \(Spectral-based\), \(Spatial-based\) and _Deeper_ GNN models, which are GCN (Zhou et al., 2017), GAT (Zhou et al., 2018), GraphSAGE (Zhou et al., 2018), JKNet (Zhou et al., 2018), APPNP (Zhou et al., 2018), GATv2 (Chen et al., 2018), DAGNN (Zhou et al., 2018), GCNII (Chen et al., 2018). Besides, we choose a recent state-of-the-art GNN model (OODGAT (Zhou et al., 2018)) designed for the OOD node classification task. We implement all baseline models and our KTGNN models in \(Pytorch\) and \(Torch-Geometric\).

For each dataset (Company and Twitter), we randomly divide the annotated silent nodes into train/valid/test sets with a fixed ratio of 60%/20%/20%. And we add all annotated vocal nodes into the training set because our target is to classify the silent nodes. The detailed hyper-parameter settings of KTGNN and other baselines are presented in Appendix D.

### Main Results

We focus on silent node classification on the VS-Graph in this paper and conduct comprehensive experiments on two critical real-world scenarios (i.e., political election and company financial assessment).

**Results of Silent Node Classification** We select two representative metrics (i.e., F1-Score and AUC) to evaluate the model performance on the silent node classification task. Considering that baseline models cannot directly handle graphs with partially-missing features (silent nodes in VS-Graphs), we combine these baseline GNNs with some heuristic feature completion strategies (e.g., completion by zero, completion by mean of neighbors) to keep fair comparison with our methods, and we finally choose the best

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline Dataset & \(|V^{wood}|\) & \(|V^{silent}|\) & \(|E|\) & \(|Y^{wood}|\) & \(|Y^{silent}|\) \\ \hline Company & 3987 & 6654 & 116785 & 3987 & 1923 \\ Twitter & 581 & 20230 & 915,438 & 581 & 625 \\ \hline \hline \end{tabular}
\end{table}
Table 1. Basic information of the dataset used in this paper.
completion strategy for each baseline GNN (results in Table 2). As shown in Table 2, our KTGNN model gains significant improvement in the performance of silent node classification on both Twitter and Company datasets compared with other state-of-the-art GNNs.

**Effects of Feature Completion Strategy** For baseline GNNs, we also analyze the effects of heuristic feature completion methods on the Company dataset at Table 6 (results of Twitter datasets are presented at Table 6 in Appendix). And the results in Table 3 indicate that the "Mean-of-Neighbors" strategy wins in most cases. However, all these heuristic completion strategies ignore the distribution-shift between the vocal nodes and the silent nodes, and thus are far less effective than our KTGNN model.

### Ablation Study

**Effects of Each Module:** To validate the effects of each component of KTGNN, we design some variants of KTGNN by removing one of modules (i.e., DAFC, DAMP, DTC, \(\mathcal{L}^{dist}\), \(\mathcal{L}^{kl}\)) and the results are shown in the Table 4. We observe that the performance of all KTGNN variants deteriorates to some degree. And our full KTGNN module gains the best performance, which demonstrates the important effects of each component in KTGNN.

**Effects of KL Loss:** We also validate the role of KL loss \(\mathcal{L}^{kl}\) (see Eq. 11) in our method. As shown in the Fig. 5, the final target classifier generated from the source classifier gains the highest scores and lowest loss among the three sub-classifiers in the DTC module (see Sec. 4.4). And the result of KL Loss (components of \(\mathcal{L}^{kl}\)) indicates that the discriminant bound of the generated target classifier is located between that of the source classifier and target classifier, which further confirms our motivations.

**Effects of Cross-Domain Knowledge Transfer:** We further validate our approach with borderline cases (i.e., topological traits that may worsen the correct knowledge transfer from vocal nodes to silent nodes). We validate this by cutting part of such edges. The results in Table 5 indicate that our approach is robust over these borderline cases.

\begin{table}
\begin{tabular}{c c c c} \hline \hline Dataset & \multicolumn{2}{c}{Twitter} & \multicolumn{2}{c}{Company} \\ \cline{2-4} Evaluation Metric & F1 & AUC & F1 & AUC \\ \hline KTGNN\({}_{\text{DAC}}\) & 85.57\(\pm\)1.38 & 92.12\(\pm\)0.87 & 60.21\(\pm\)0.39 & 61.95\(\pm\)0.45 \\ KTGNN\({}_{\text{DAMP}}\) & 86.85\(\pm\)1.01 & 93.03\(\pm\)0.98 & 62.71\(\pm\)0.44 & 64.27\(\pm\)0.61 \\ KTGNN\({}_{\text{DTC}}\) & 88.80\(\pm\)0.93 & 94.13\(\pm\)1.08 & 63.05\(\pm\)0.52 & 64.35\(\pm\)0.55 \\ KTGNN\({}_{\mathcal{L}^{dist}}\) & 87.80\(\pm\)0.83 & 93.85\(\pm\)1.38 & 63.56\(\pm\)0.68 & 65.38\(\pm\)0.65 \\ KTGNN\({}_{\mathcal{L}^{kl}}\) & 89.20\(\pm\)1.20 & 94.06\(\pm\)1.03 & 62.71\(\pm\)0.63 & 65.02\(\pm\)0.55 \\ \hline KTGNN & **89.65\(\pm\)1.20** & **95.08\(\pm\)**0.93** & **64.96\(\pm\)**0.63** & **67.11\(\pm\)**0.52** \\ \hline \hline \end{tabular}
\end{table}
Table 4. Ablation studies of KTGNN compared with its variants by removing certain components.

\begin{table}
\begin{tabular}{c c c c c} \hline \hline Dataset & \multicolumn{2}{c}{Twitter} & \multicolumn{2}{c}{Company} \\ \cline{2-5} Evaluation Metric & F1 & AUC & F1 & AUC \\ \hline MLP & 70.85\(\pm\)1.31 & 80.12\(\pm\)1.38 & 57.01\(\pm\)0.42 & 56.35\(\pm\)0.55 \\ GCN & 80.19\(\pm\)0.87 & 86.88\(\pm\)1.23 & 57.60\(\pm\)0.49 & 57.83\(\pm\)0.93 \\ GAT & 82.31\(\pm\)1.56 & 87.88\(\pm\)1.21 & 57.98\(\pm\)0.43 & 58.40\(\pm\)0.43 \\ GraphSAGE & 85.40\(\pm\)1.03 & 92.08\(\pm\)1.61 & 58.93\(\pm\)0.67 & 60.63\(\pm\)0.57 \\ JKNet & 83.86\(\pm\)1.93 & 91.17\(\pm\)1.21 & 58.38\(\pm\)0.58 & 61.06\(\pm\)0.53 \\ APPNP & 80.60\(\pm\)1.60 & 87.01\(\pm\)1.08 & 57.53\(\pm\)0.63 & 57.99\(\pm\)0.80 \\ GATv2 & 82.63\(\pm\)1.09 & 89.83\(\pm\)1.27 & 57.78\(\pm\)0.52 & 58.93\(\pm\)0.93 \\ DAGNN & 84.37\(\pm\)0.93 & 91.98\(\pm\)1.05 & 59.62\(\pm\)0.43 & 59.07\(\pm\)0.39 \\ GCNII & 83.85\(\pm\)1.17 & 90.67\(\pm\)1.32 & 58.21\(\pm\)0.88 & 60.06\(\pm\)0.63 \\ OODGAT & 85.95\(\pm\)2.01 & 92.67\(\pm\)1.60 & 60.05\(\pm\)0.89 & 61.37\(\pm\)0.92 \\ \hline KTGNN & **89.65\(\pm\)1.20** & **95.08\(\pm\)**0.93** & **64.96\(\pm\)**0.63** & **67.11\(\pm\)**0.52** \\ \hline \hline \end{tabular}
\end{table}
Table 2. Results of silent node classification. All base models except KTGNN are combined with one optimal heuristic feature completion strategy.

\begin{table}
\begin{tabular}{c c c c} \hline \hline Dataset & Completion Method & \multicolumn{2}{c}{Company} \\ \cline{2-4} Evaluation Metric & \(\backslash\) & F1 & AUC \\ \hline \multirow{4}{*}{MLP} & None & 56.06\(\pm\)0.63 & 55.36\(\pm\)0.33 \\  & \(\theta\)-Completion & 56.60\(\pm\)0.67 & 55.07\(\pm\)0.58 \\  & Mean-of-Neighbors & 57.01\(\pm\)0.42 & 56.35\(\pm\)0.55 \\ \hline \multirow{4}{*}{GCN} & None & 56.61\(\pm\)0.53 & 56.09\(\pm\)0.44 \\  & \(\theta\)-Completion & 56.41\(\pm\)0.59 & 56.75\(\pm\)0.58 \\  & Mean-of-Neighbors & 57.60\(\pm\)0.49 & 57.83\(\pm\)0.93 \\ \hline \multirow{4}{*}{GraphSAGE} & None & 57.12\(\pm\)0.57 & 57.29\(\pm\)0.57 \\  & \(\theta\)-Completion & 58.63\(\pm\)0.71 & 59.78\(\pm\)0.65 \\  & Mean-of-Neighbors & 58.93\(\pm\)0.67 & 60.63\(\pm\)0.57 \\ \hline \multirow{4}{*}{JKNet} & None & 57.26\(\pm\)0.49 & 58.44\(\pm\)0.67 \\  & \(\theta\)-Completion & 57.81\(\pm\)0.65 & 60.95\(\pm\)0.63 \\  & Mean-of-Neighbors & 58.38\(\pm\)0.58 & 61.06\(\pm\)0.53 \\ \hline \multirow{4}{*}{GCNII} & None & 58.36\(\pm\)0.37 & 58.85\(\pm\)0.33 \\  & \(\theta\)-Completion & 58.07\(\pm\)0.47 & 59.91\(\pm\)0.56 \\  & Mean-of-Neighbors & 58.21\(\pm\)0.88 & 60.06\(\pm\)0.63 \\ \hline \multirow{4}{*}{OODGAT} & None & 56.73\(\pm\)1.33 & 57.83\(\pm\)1.20 \\  & \(\theta\)-Completion & 59.65\(\pm\)0.83 & 60.93\(\pm\)0.98 \\ \cline{1-1}  & Mean-of-Neighbors & 60.05\(\pm\)0.89 & 61.37\(\pm\)0.92 \\ \hline \hline KTGNN & \(\backslash\) & **64.96\(\pm\)0.63** & **67.11\(\pm\)0.52** \\ \hline \hline \end{tabular}
\end{table}
Table 3. Results of models with different completion strategies on Company dataset. “None” means we only use the observable attributes (i.e., \(X^{o}\)) for both vocal and silent nodes without completion; “0-Completion” and “Mean-of-Neighbors” use the zero vector and mean vector of vocal neighbors to complete the missing dimensions for silent nodes.

Figure 5. F1 Score and Loss Curve of KTGNN on Company dataset (results of Twitter dataset are shown in appendix).
### HyperParameter Sensitivity Analysis

We validate the sensitivity of main hyper-parameters of KTGNN: \(K\) **(max-iteration of DAFC )** and \(\lambda\) **(weight of \(\mathcal{L}^{kl}\))**. As shown in Fig. 6, we conduct experiments on the Company dataset with different \(K\) and \(\lambda\). The results show that KTGNN gains the best performance when \(K=2\), which is decided by the graph property that \(2\)-hop neighbors of vocal nodes cover most silent nodes, and gains the best performance when \(lambda=1.0\), which also indicates the effectiveness of the KL loss.

### Representation Visualization

We visualize the learned representations of KTGNN in Fig. 7. Subfigures (a) and (d) show that our KTGNN remains the domain difference between vocal nodes and silent nodes during message-passing. And Sub-figures (b) (c) and (d) (f) show that the representations learned by KTGNN have better distinguishability between different classes for both vocal nodes and silent nodes, compared with the T-SNE visualization of raw features in Fig. 3 (The representation visualization of baseline GNNs are shown in the appendix).

## 6. Related Work

Graph Neural Networks (GNNs), as powerful tools for modeling graph data, have been widely proposed and can be generally categorized into _spectral-based methods_(Krizhevsky et al., 2014; He et al., 2015; He et al., 2016; He et al., 2017; He et al., 2018) and _spatial-based methods_(Krizhevsky et al., 2014; He et al., 2015; He et al., 2016; He et al., 2017; He et al., 2018; He et al., 2019; He et al., 2018; He et al., 2019; He et al., 2018; He et al., 2019). Existing mainstream spatial-based GNNs (He et al., 2016; He et al., 2017; He et al., 2018; He et al., 2019; He et al., 2018; He et al., 2019; He et al., 2018; He et al., 2019) follow the message-passing framework, where node representations are updated based on features aggregated from neighbors. Recently, deeper GNNs (He et al., 2016; He et al., 2018; He et al., 2019) with larger receptive fields, have been proposed and gained better performance. However, most of the existing GNNs are built on the I.I.D. hypothesis, i.e., training and testing data are independent and identically distributed. And the I.I.D. hypothesis is hard to be satisfied in many real-world scenarios where the model performance degrades significantly when there exist distribution-shifts among the nodes on the graph.

O.O.D nodes commonly exist in real-world graphs (He et al., 2016; He et al., 2017; He et al., 2018; He et al., 2019; He et al., 2019), which bring obstacles to current message-passing based GNN models. Recently, GNNs for graph-level O.O.D prediction (He et al., 2018; He et al., 2019; He et al., 2019; He et al., 2019; He et al., 2019; He et al., 2019) have been proposed. However, GNNs for semi-supervised node classification with O.O.D nodes have been rarely studied. Some existing studies (He et al., 2016; He et al., 2019; He et al., 2019; He et al., 2019; He et al., 2019) combine GNN with the O.O.D detection task as an auxiliary to enhance the performance of GNNs on graphs with O.O.D nodes. Such kind of method usually views O.O.D nodes as noises, which aim at detecting potential O.O.D nodes first and then alleviating their effects. However, O.O.D nodes are not noises in all scenarios (e.g., VS-Graph introduced in this paper) and knowledge from other domains can be helpful to alleviate the data-hunpy problem of the target domain. Different from existing O.O.D detection methods, we propose to improve the model performance on the target domain by injecting out-of-distribution knowledge in this paper.

## 7. Conclusion

In this paper, we propose a novel and widespread problem: silent node classification on the VS-Graph ("predicting the silent majority on graphs"), where the silent nodes suffer from serious data-hungry problems (feature-missing and label scarcity). Correspondingly, we design a novel KTGNN model for silent node classification by transferring useful knowledge from vocal nodes to silent nodes adaptively. Comprehensive experiments on the two representative real-world VS-Graphs demonstrate the superiority of our approach.

###### Acknowledgements.

 This work was supported by the National Natural Science Foundation of China (Grant No.U21B2046, No.62202448, No.61902380, No.61802370, and No.U1911401), the Beijing Nova Program (No. Z201100006820061) and China Postdoctoral Science Foundation (2022M713208).

\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{Removing \#\% of} & \multicolumn{2}{c}{Twitter} & \multicolumn{2}{c}{Company} \\ \cline{2-5}  & F1 & AUC & F1 & AUC \\ \hline
70\% & 86.40\(\pm\)1.18 & 92.33\(\pm\)0.96 & 61.73\(\pm\)0.70 & 63.34\(\pm\)0.65 \\
50\% & 88.00\(\pm\)1.03 & 93.53\(\pm\)1.03 & 62.91\(\pm\)0.59 & 64.89\(\pm\)0.58 \\
30\% & 88.80\(\pm\)1.29 & 94.56\(\pm\)0.97 & 63.70\(\pm\)0.73 & 66.05\(\pm\)0.61 \\ \hline
0\% & **89.65\(\pm\)1.20** & **95.08\(\pm\)0.93** & **64.96\(\pm\)0.63** & **67.11\(\pm\)0.52** \\ \hline \hline \end{tabular}
\end{table}
Table 5. Boarder-line cases analysis on the effect of cross-domain edges by removing part (i.e., 0%, 30%, 50%, and 70%) of such edges randomly from the original VS-Graph.

Figure 6. Hyper-parameter analysis of \(K\) and \(\lambda\) on Company dataset (see results on Twitter dataset in Fig. 9 of Appendix).

Figure 7. T-SNE visualization of the node completed representations learned by KTGNN, distinguished by their populations (vocal\(\rightarrow\)orange, silent\(\rightarrow\)cyan) or node labels (binary classes: positive\(\rightarrow\)red, negative\(\rightarrow\)blue).