HAGNN: Hybrid Aggregation for Heterogeneous Graph Neural Networks
Heterogeneous graph neural networks (GNNs) have been successful in handling
heterogeneous graphs. In existing heterogeneous GNNs, meta-path plays an
essential role. However, recent work pointed out that simple homogeneous graph
model without meta-path can also achieve comparable results, which calls into
question the necessity of meta-path. In this paper, we first present the
intrinsic difference about meta-path-based and meta-path-free models, i.e., how
to select neighbors for node aggregation. Then, we propose a novel framework to
utilize the rich type semantic information in heterogeneous graphs
comprehensively, namely HAGNN (Hybrid Aggregation for Heterogeneous GNNs). The
core of HAGNN is to leverage the meta-path neighbors and the directly connected
neighbors simultaneously for node aggregations. HAGNN divides the overall
aggregation process into two phases: meta-path-based intra-type aggregation and
meta-path-free inter-type aggregation. During the intra-type aggregation phase,
we propose a new data structure called fused meta-path graph and perform
structural semantic aware aggregation on it. Finally, we combine the embeddings
generated by each phase. Compared with existing heterogeneous GNN models, HAGNN
can take full advantage of the heterogeneity in heterogeneous graphs. Extensive
experimental results on node classification, node clustering, and link
prediction tasks show that HAGNN outperforms the existing modes, demonstrating
the effectiveness of HAGNN.
Guanghui Zhu, Zhennan Zhu, Hongyang Chen, Chunfeng Yuan, Yihua Huang
# HAGNN: Hybrid Aggregation for Heterogeneous Graph Neural Networks

###### Abstract

Heterogeneous graph neural networks (GNNs) have been successful in handling heterogeneous graphs. In existing heterogeneous GNNs, meta-path plays an essential role. However, recent work pointed out that simple homogeneous graph model without meta-path can also achieve comparable results, which calls into question the necessity of meta-path. In this paper, we first present the intrinsic difference about meta-path-based and meta-path-free models, i.e., how to select neighbors for node aggregation. Then, we propose a novel framework to utilize the rich type semantic information in heterogeneous graphs comprehensively, namely HAGNN (Hybrid Aggregation for Heterogeneous GNNs). The core of HAGNN is to leverage the meta-path neighbors and the directly connected neighbors simultaneously for node aggregations. HAGNN divides the overall aggregation process into two phases: meta-path-based intra-type aggregation and meta-path-free inter-type aggregation. During the intra-type aggregation phase, we propose a new data structure called fused meta-path graph and perform structural semantic aware aggregation on it. Finally, we combine the embeddings generated by each phase. Compared with existing heterogeneous GNN models, HAGNN can take full advantage of the heterogeneity in heterogeneous graphs. Extensive experimental results on node classification, node clustering, and link prediction tasks show that HAGNN outperforms the existing modes, demonstrating the effectiveness of HAGNN.

 Heterogeneous graph, Graph neural network, Graph representation learning.

## I Introduction

Many real-world data can be naturally represented as graph structure. Meanwhile, in many practical scenarios, such as knowledge graphs [1], scholar networks [2, 3, 4], and biochemical networks [5], the graphs are heterogeneous. Compared with homogeneous graphs, heterogeneous graphs have more than one type of nodes or links, which encode more semantic information [6].

Graph neural networks (GNNs) [7, 8] have achieved remarkable success on graph-structured data by learning a low-dimensional representation for each node. Moreover, to tackle the challenge of heterogeneity, many representation learning models are proposed to utilize the rich semantic information in heterogeneous graphs. Among these methods, meta-path [6] is considered a natural way to decouple the diversified connection patterns among nodes. Specifically, meta-path is a composite relation consisting of multiple edge types. For example, in Figure 1, Paper-Author-Paper is a typical meta-path, which reflects that two papers are published by the same author. metapath2vec [9] formalizes meta-path-based random walks to compute node embeddings. HAN [10] and MAGNN [11] employ hierarchical attention to aggregate information from meta-path-based neighbors. GTN [12] implicitly learns meta-paths by fusing different node types based on the attention mechanism. Meta-path is also used for knowledge distillation [13], text summarization [14], and contrastive learning [15, 16] on heterogeneous graphs.

Meta-path plays an essential role in the existing heterogeneous GNNs. However, recent researchers [17] experimentally found that homogeneous GNNs such as GAT [18] actually perform pretty well on heterogeneous graph by revisiting the model design, data preprocessing, and experimental settings of the heterogeneous GNNs, which calls into question the necessity of meta-paths [17, 19]. To answer this question, we first give an in-depth analysis about the intrinsic difference about meta-path-based models (e.g., HAN, MAGNN, GTN) and meta-path-free models (e.g., RGCN [20], GAT, Simple-HGN [17]). For a given node, how to select neighbors for node aggregation is a fundamental step in heterogeneous GNNs, which is also the key difference between the two types of models. For meta-path-based models, they usually construct the meta-path-based graph for the target node type. In the meta-path-based graph, all nodes have the same type and any two adjacent nodes have at least one path instance following the specific symmetric meta-path. The final representation of a node is calculated by aggregating its neighbors in the meta-path-based graph. For meta-path-free models, they directly aggregate neighbors in the original graph. In most heterogeneous graphs, a node is of different types with its immediate neighbors. Thus, the final embedding is the aggregation of the nodes with different types.

Take an example as shown in Figure 1, if we want to know the category of a paper, meta-path-based models check the

Fig. 1: Different aggregation schemes on the DBLP dataset.
categories of other papers written by the same author, while meta-path-free models collect information about the author of the paper, the conference where the paper was published, and the term of the paper etc. Overall, the immediate neighbors of a node contain key attributes, and the meta-path-based neighbors can easily supply high-order connectivity information of the same node type. Both meta-path-based neighbors and immediate neighbors are useful, they can complement each other. Therefore, to improve the performance of heterogeneous GNNs, it is essential to design a new representation leaning method that can leverage both meta-path-based neighbors and immediate neighbors.

A straightforward idea is to directly combine the meta-path-based intra-type aggregation with the immediate-neighbor-based aggregation. But as shown in Figure 2, simply combining the typical meta-path-based model (i.e., HAN) with the SOTA meta-path-free model (i.e., SimpleHGN) even leads to performance decreases on both the node classification datasets (i.e., DBLP and IMDB) and the link prediction dataset (i.e., PubMed), especially on the IMDB dataset. The reason for this problem is that existing meta-path-based models suffer from information redundancy and excessive additional parameters, which may lead to over-parameterization as well as over-fitting if directly further combined with meta-path-free models.

Table I shows the issue of information redundancy. In existing meta-path-based models [10, 11], each meta-path corresponds to a graph. The node representations learned from each separate math-path-based graphs are then aggregated with hierarchical attention. For the DBLP dataset in Table I, the meta-path-based-graph produced by Author-Paper-Author (APA) is a subgraph of that produced by Author-Paper-Conference-Paper-Author (APCPA). The meta-path-based graph generated by the meta-path Author-Paper-Conference-Paper-Author and the meta-path Author-Paper-Term-Paper-Author (APTPA) have 61.84% duplicate edges. There exists the similar problem for the ACM dataset. It can be seen that if we build separate meta-path-based graphs for each different meta-paths, unnecessary computation will be squandered on redundant information. Moreover, too many duplicate edges lead to excessive additional learnable parameters for node aggregation, which may degrade the learning performance.

Furthermore, Table I also shows that the number of meta-path-based neighbors is much larger than that of direct neighbors in the original graph and thus too many neighbors in the meta-path-based graph cause difficulties in the learning of attention weights. Existing meta-path-based models only consider the node connectivity in the meta-path-based graph, ignoring the _structural semantic information_ (e.g., the number of path instances following the specific meta-path), which can be exploited to improve the learning of node representation.

Based on the above analysis, we propose a novel framework to utilize the rich type semantic information in heterogeneous graphs comprehensively, namely HAGNN1 (Hybrid Aggregation for Heterogeneous Graph Neural Networks). The core of HAGNN is to leverage the meta-path-based neighbors and the directly connected neighbors simultaneously for node aggregation. Specifically, we divide the overall aggregation process into two phases: meta-path-based intra-type aggregation phase and meta-path-free inter-type aggregation phase. During the intra-type aggregation phase, we first propose a new data structure called _fused meta-path graph_ to avoid information redundancy. For a specific node type, the meta-path neighbor relationships of multiple meta-paths are fused in a single graph, where all nodes have the same type. Also, the fused meta-path graph contains the connectivity information of multiple meta-graphs. Then, we perform attention-based intra-type aggregation in the fused meta-path graph.

Footnote 1: HAGNN is available at [https://github.com/Pasal_ab/HAGNN](https://github.com/Pasal_ab/HAGNN)

To further improve the learning of attention weights, we propose a _structural semantic aware aggregation_ method. For each two neighbor nodes of the fused meta-path graph, we view the number of path instances in the original graph as the structural semantic weight, which is used to guide the learning of attention weights. During the inter-type aggregation phase, we directly perform node aggregation with the self-attention mechanism in the original heterogeneous graph to capture the information of immediate neighbors. Finally, the node embeddings generated by the intra-type aggregation and the inter-type aggregation are combined.

To summarize, the main contributions are highlighted as follows:

* Based on the analysis that both meta-path-based and meta-path-free aggregation should be beneficial to the heterogeneous graph, we propose a novel hybrid aggre

Fig. 2: Comparison between the simple combination of intra-type aggregation and inter-type aggregation (i.e., HAN + SimpleHGN) vs. HAGNN (Proposed)
gation framework consisting of two phases: meta-path-based intra-type aggregation and meta-path-free inter-type aggregation. It is the first attempt of combining two kinds of aggregations.
* To eliminate information redundancy and make the intra-type aggregation more effectively, we propose the fused meta-path graph to capture the meta-path neighbors for a specific node type.
* To improve the leaning of attention weights in the intra-type aggregation phase, we propose a structural semantic aware aggregation mechanism, which leverages the number of path instances as the auxiliary aggregation weights.
* Extensive experimental results on five heterogeneous graph datasets reveal that HAGNN outperforms the existing heterogeneous GNNs on node classification, node clustering, and link prediction tasks. The discussion about HAGNN also provides insightful guidance to the use of meta-paths in the heterogeneous graph neural networks.

## II Related Work

### _Homogeneous Graph Representation Learning_

Graph representation learning aims to learn low-dimensional representations from non-Euclidean graph structure. For homogeneous graphs, most methods learn node representations from neighborhood. Line [21] utilizes the first-order and second-order proximity between nodes to learn node embeddings. DeepWalk [22], node2vec [23], TADW [24], and Struc2vec [25] extract a node sequence by random walk and feed the sequence to a skip-gram model. Graph neural networks, following the message passing framework, have been widely exploited in graph representation learning. GNNs can be divided into spectral-based and spatial-based models [7]. GCN [26] is a typical spectral-based model that achieves spectral graph convolution via localized first-order approximation. While the spatial-based models such as GAT [18] leverage the attention mechanism for node aggregation.

Representation learning models for homogeneous graphs are considered unsuitable for heterogeneous graphs due to ignoring type information, but recent work points out that some homogeneous graph models actually perform well in heterogeneous graphs, which is thought provoking.

### _Heterogeneous Graph Representation Learning_

Heterogeneous graphs have more than one type of nodes or edges. To utilize the semantics encoded in heterogeneous graphs, many models designed for heterogeneous graphs are proposed [10, 27].

Depending on whether using meta-path, we can divide these models into meta-path-based and meta-path-free models. For meta-path-based models, metapath2vec [9] utilizes the node paths traversed by meta-path-guided random walks to model the context of a node. HIN2Vec [28] carries out multiple prediction training tasks to learn latent vectors of nodes and meta-paths. HAN [10] leverages the semantic information of meta-paths, and uses hierarchical attention to aggregate neighboring nodes. MAGNN [10] utilizes RotatE [29] to encode intermediate nodes along each meta-path and mix multiple meta-paths using hierarchical attention. GTN [12] learns a soft selection of edge types and composite relations for generating useful meta-paths. SHGNN [30] uses a tree-based attention module to aggregate information on the meta-path and consider the graph structure in multiple meta-path instances. R-HGNN [31] learns node representations on heterogeneous graphs at a fine-grained level by considering relation-aware characteristics. CKD [13] learns the meta-path-based embeddings by collaboratively distilling the knowledge from intra-meta-path and inter-meta-path simultaneously.

The meta-path-free models extract rich semantic information without meta-path. RGCN [20] introduces relation-specific transformations to handle different edge types. HetGNN [32] uses Bi-LSTM to aggregate node features for each type and among types. SimpleHGN [17] revisits existing models and proposes a simple framework with GAT as backbone.

The difference between meta-path-based and meta-path-free models is that they have conflict on which kind of neighbors is more informative. In this paper, we propose a hybrid aggregation framework that can leverage both the meta-path-based neighbors and immediate neighbors effectively.

## III Preliminaries and Notations

**Definition 1**: **Heterogeneous Graph [6]**_. A heterogeneous graph is a directed graph with the form \(\mathcal{H}=\{\mathcal{V},\mathcal{E},\mathcal{T},\mathcal{R},\phi,\psi\}\), where \(\mathcal{V}\) and \(\mathcal{E}\) denote the node set and the edge set in \(\mathcal{H}\). Each node \(v_{i}\in\mathcal{V}\) is associated with a node type \(\phi(v_{i})=t_{i}\in\mathcal{T}\). Similarly, each edge \(e_{ij}\in\mathcal{E}\) is associated with an edge type \(\psi(e_{ij})=r_{ij}\in\mathcal{R}\). In graph \(\mathcal{H}\), \(|\mathcal{T}|+|\mathcal{R}|>2\). Every node has attribute \(x_{i}\in\mathbb{R}^{d_{i}}\), and \(d_{i}\) varies from the type of node \(v_{i}\). Let \(\mathbf{A}_{t_{i},t_{j}}\) denote the adjacent matrix of type \(t_{i}\) and \(t_{j}\). \(\mathbf{A}_{t_{i},t_{j}}[u][v]=1\) indicates that \(\phi(u)=t_{i}\) and \(\phi(v)=t_{j}\)._
**Definition 2**: **Meta-Path [6]**_. A meta-path \(p\) is a composite relation, which consists of multiple edge types, i.e., \(p=t_{1}\xrightarrow{r_{1}}t_{2}\xrightarrow{r_{2}}\ldots\xrightarrow{r_{1}}t_{l+1}\), where \(t_{1},\ldots,t_{l+1}\in\mathcal{T}\) and \(r_{1},\ldots,r_{l}\in\mathcal{R}\). One meta-path contains many meta-path instances in \(\mathcal{H}\)._

**Definition 3**: **Meta-path-based Neighbors [11]**_. Given a meta-path \(p\) in \(\mathcal{H}\), the meta-path-based neighbors of node \(v\) is defined as the set of nodes that connect with node \(v\) via a meta-path instance of \(p\). If meta-path \(p\) is symmetrical, the meta-path-based neighbors of node \(v\) contain itself._

**Definition 4**: **Meta-path-based Graph [11]**_. Given a meta-path \(p\) in \(\mathcal{H}\), the meta-path-based graph \(G_{p}\) of \(p\) is a graph constructed by all meta-path-based neighbor pairs. \(G_{p}\) is homogeneous if the head and tail node types of \(p\) are the same. The neighbors of node \(v\) in \(G_{p}\) can be donated as \(N_{G_{p}}^{v}\)._

Table II shows the used notations and their definitions.

## IV The proposed methodology

### _Overall Framework_

Figure 3 shows the overall framework of HAGNN, which consists of three phases: meta-path-based intra-type aggregation, meta-path-free inter-type aggregation, and combination of semantic information. In the intra-type aggregation phase, we first construct the fused meta-path graph that contains all meta-path-based neighbors. Then, we leverage the structural semantic weights to guide the node aggregation in the fused meta-path graph. The node embeddings obtained in the intra-type aggregation phase are fed to the inter-type aggregation phase, which performs node aggregation directly on the original heterogeneous graph. The embeddings generated by the two aggregation phases are combined to form the final embeddings of the target node types, which are further used for downstream tasks.

The intra-type aggregation phase aims to capture the information of high-order meta-path neighbors, while the inter-type aggregation phase directly aggregates the attribute information of immediate neighbors. The two phases perform node aggregation from two different perspectives. Next, we introduce each phase of HAGNN.

### _Meta-path-based Intra-type Aggregation_

Since the intra-type aggregation is performed on the fused meta-path graph, we first introduce the definition of the proposed fused meta-path graph.

#### Iv-B1 Fused Meta-path Graph

The core of heterogeneous GNNs is to use the graph topology to perform message aggregation. Thus, the graph topology plays an essential role in heterogeneous GNNs. Most of exiting heterogeneous GNNs construct the meta-path-based graph for node aggregation. Each meta-path corresponds to a meta-path-based graph. As shown in Table I, information redundancy arises when we put meta-path-based neighbors of different meta-paths in separate graphs. A large number of duplicate edges leads to computational redundancy and excessive additional parameters when computing and optimizing attention weights in the meat-path-based graph, leading to negative impacts on the computation efficiency and learning performance. To address the issues, we propose a novel data structure called fused meta-path graph to carry the information of multiple meta-paths.

**Definition 5**: **Type of meta-path:** Given a meta-path \(p\) with length of \(l\), \(p=t_{1}\xrightarrow{r_{1}}t_{2}\xrightarrow{r_{2}}\ldots\xrightarrow{r_{l}}t_{l+1}\), where \(t_{1},\ldots,t_{l+1}\in\mathcal{T}\) and \(r_{1},\ldots,r_{l}\in\mathcal{R}\). The type of \(p\) is the head node type \(t_{1}\) and meta-path \(p\) can also be denote as \(p_{t_{1}}\).

**Definition 6**: **Fused meta-path graph:** Given a node type \(t\in\mathcal{T}\) of a heterogeneous graph \(\mathcal{H}\), the meta-path set of type \(t\) is \(\mathcal{P}_{t}=\{p_{t}^{1},p_{t}^{2},\ldots,p_{t}^{l}\}\), the fused meta-path graph \(\mathcal{G}_{t}\) is a single graph contains all meta-path neighbors in \(\mathcal{P}_{t}\). The neighbors of node \(v\) in \(\mathcal{G}_{t}\) is denoted as \(N_{\mathcal{G}_{t}}^{v}\).

\(\mathcal{G}_{t}\) is a homogeneous graph and defines the topological adjacency relationship between nodes of the same node type. To construct \(\mathcal{G}_{t}\), we first obtain the meta-path-based graph \(G_{p_{t}}\) for \(p_{t}\in\mathcal{P}_{t}\), and then we take the union of \(G_{p_{t}}\), i.e.,

\[\mathcal{G}_{t}=\bigcup_{p_{t}\in\mathcal{P}_{t}}G_{p_{t}} \tag{1}\]

Figure 3 shows the construction process of the fused meta-path graph. Unlike the meta-path-based graph, each node type instead of each meta-path corresponds to a fused meta-path graph. For node \(v\) in fused meta-path graph \(\mathcal{G}_{t}\), \(N_{\mathcal{G}_{t}}^{v}\) contains all meta-path neighbors from \(\mathcal{P}_{t}\). The relationship between \(v\) and its neighbor \(u\in N_{\mathcal{G}_{t}}^{v}\) may belong to multiple meta-paths. Since the repeated meta-path-based neighbor relationships are reflected in a single edge, the information redundancy between different meta-paths can be eliminated.

Table III compares fused meta-path graph with existing methods to extract homogeneous graphs from heterogeneous graphs using meta-paths. For HAN and MAGNN, meta-path-based graphs are directly constructed by meta-path-based neighbors. But as we point out in Table I, there is a serious data redundancy problem between different meta-path graphs. GTN is proposed to learn meta-path graphs. It uses 1x1 convolution to softly select different edge types, and then uses matrix multiplication to generate new meta-paths. Since the selection of edge types is continuous rather than discrete, the meta-path graph generated by GTN is dense. Moreover, the computation overhead of GTN is huge. Overall, the proposed fused meta
path graph can solve the data redundancy problem and achieve better performance with less time overhead.

#### Iii-A2 Type Selection for Intra-type Aggregation

HAGNN is a two-phase aggregation model. As shown in Figure 3, in the latter phase (i.e, the inter-type aggregation phase), the representations of other node types are absorbed in the embeddings of the target node type. Unlike the previous models [10, 11] where other node types only play a role as bridges between nodes of the target type and do not directly participate in the learning of node representation, we also perform intra-type aggregation for non-target node types.

Moreover, not all node types are suitable for participating in the intra-type aggregation phase. We select node types for intra-type aggregation mainly based on the following two aspects. First, in heterogeneous graphs, the quantity of different types of nodes is varied. For example, the DBLP dataset has 14,328 paper nodes, but only 20 conferences nodes. For those types with a small number of nodes, we believe that the relationship between the nodes is clear enough and the intra-type aggregation is not necessary. Second, having a closed meta-path is also an important condition. A meta-path \(p\) is closed if its head and tail types are the same, and only using a closed meta-path can generate homogeneous graphs.

Formally, we denote the type set participating in the intra-type aggregation phase as \(\mathcal{T}^{\prime}\).

\[\mathcal{T}^{\prime}=\{t\ |\ \frac{|\mathcal{V}_{t}|}{|\mathcal{V}|}>threshold \text{ and }t\text{ has closed meta-paths}\} \tag{2}\]

where \(V_{t}\) denotes the number of nodes with type \(t\). In practice, the threshold can be set to 1%.

#### Iii-A3 Type-specific Linear Transformation

Before the intra-type aggregation, since nodes of different types have different feature dimensions, we apply a type-specific linear transformation to each type of node, projecting the features of each type of node into the same latent factor space. For node \(v\in\mathcal{V}\) of \(t\in\mathcal{T}\):

\[\tilde{h}_{v}=\tilde{W}_{t}\cdot x_{v}^{t} \tag{3}\]

where \(x_{v}^{t}\in\mathbb{R}^{d_{t}}\) is the original feature vector of node \(v\). The dimension \(d_{t}\) varies from the node type. \(\tilde{W}_{t}\in\mathbb{R}^{d\times d_{t}}\) is the learnable transformation matrix.

After the linear transformation, all nodes have the same dimension \(d\). Next, the intra-type node aggregation can be carried out to aggregate meta-path-based higher-order neighbors.

#### Iii-A4 Structural Semantic Aware Aggregation

For the specific node type \(t\), once the corresponding fused meta-path graph \(\mathcal{G}_{t}\) is constructed, we perform intra-type node aggregation according to the topology (i.e., the neighborhood of each node) of \(\mathcal{G}_{t}\). For each node \(v\) in the fused meta-path graph, the neighborhood is much larger than that in the original graph. Optimizing the attention weight of each neighbor becomes a challenging task. To address the challenge, we propose a structural semantic aware mechanism to guide the learning of the attention weight for node aggregation.

For meta-path \(p_{t}\), suppose that the length of \(p_{t}\) is \(l\) and the node type sequence of \(p_{t}\) is \(t_{1}\to t_{2}\rightarrow\cdots\to t_{l-1}\to t_{l}\). Let \(\mathcal{A}\) be the weighted adjacent matrix following meta-path \(p_{t}\):

\[\mathcal{A}(\mathcal{H},p_{t})=\prod_{i=1}^{l-1}\mathbf{A}_{t_{i},t_{i+1}} \tag{4}\]

where \(\mathbf{A}_{t_{i},t_{i}+1}\) is the adjacent matrix of type \(t_{i}\) and and \(t_{i+1}\). For the node pair \((u,v)\), \(\mathcal{A}(\mathcal{H},p_{t})[u][v]\) denotes the number of path instances from node \(u\) to node \(v\) in heterogeneous graph

Fig. 3: The overall framework of HAGNN. Different circle colors represent different node types.
\(\mathcal{H}\), and the node types on the path conform to the pattern of meta-path \(p_{t}\).

\(\mathcal{A}\) defines the meta-path-based similarity between nodes of the same type. In the fused meta-path graph \(\mathcal{G}_{t}\), since the neighbor pair \((u,v)\) may contain the meta-path neighbor relationships of multiple meta-paths, we further additively mix these meta-path-based similarities:

\[\delta_{uv}^{t}=\sum_{p_{t}\in\mathcal{P}_{t}}\mathcal{A}(\mathcal{H},p_{t})[u] [v] \tag{5}\]

\(\delta_{uv}^{t}\) can be viewed as the common neighbours [33, 34] of node \(u\) and \(v\) in \(\mathcal{G}_{t}\). We define \(\delta\) as _structural semantic weight_ because it can reflect the structural semantic similarity between nodes. For example, as shown in Figure 4, the number of papers published by author \(A\) and author \(B\) is more than the number of papers published by author \(A\) and author \(C\). Thus, \(A\) and \(B\) should have a stronger relationship than \(A\) and \(C\), and their representations should be more similar.

To make structural semantic weight participate in node aggregation, we normalize it by softmax.

\[\tilde{\delta}_{uv}^{t}=\frac{exp(\delta_{uv}^{t})}{\sum_{(u^{\prime},v^{ \prime})\in\mathcal{G}_{t}}exp(\delta_{u^{\prime}v^{\prime}}^{t})} \tag{6}\]

However, only use \(\delta\) as the attention weight for intra-type aggregation has the following two problems:

* \(\delta\) is not capable of distinguishing the importance of different meta-paths by directly summing all meta-path-based similarity. As the higher-order relationship between nodes, different meta-paths represent different levels of intimacy. For example, there are three nodes \(A_{1}\), \(A_{2}\), \(A_{3}\), and the meta-path instances between them is \(A_{1}-P_{1}-A_{2}\) and \(A_{1}-P_{1}-T_{1}-P_{2}-A_{3}\). Even if the meta-path connected between A1-A2 and A1-A3 are different, \(\delta_{(A_{1},A_{2})}=\delta_{(A_{1},A_{3})}\) in the fused meta-path graph.
* \(\delta\) only reflects the intuitive semantic information. As a fixed value, \(\delta\) does not express small differences within the same meta-path. For example, for the APA meta-path, although co-authors may be in the same field, there are also papers in cross-cutting fields whose authors are in different fields.

Therefore, \(\delta\) contains useful information, but cannot be completely relied upon. Learnable adaptive weights are also necessary. In this paper, we adopt a graph self-attention mechanism to calculate the adaptive weights.

\[\begin{split} u\in N_{\mathcal{G}_{t}}^{v}\\ e_{uv}^{t}=LeakyReLU(a_{t}^{T}\cdot W_{t}[\tilde{h}_{v}||\tilde{h }_{u}])\\ \alpha_{uv}^{t}=\frac{exp(e_{uv}^{t})}{\sum_{s\in N_{\mathcal{G}_ {t}}^{v}}exp(e_{us}^{t})}\end{split} \tag{7}\]

where \(||\) denotes the concatenation operator, \(a_{t}\) and \(W_{t}\) are learnable parameters in the self-attention mechanism that vary with node type \(t\). \(\alpha\) represents learned attention weights, which can be adaptively adjusted according to the performance of downstream tasks. Inspired by [35][17], we introduce \(\delta\) as an edge residual into \(\alpha\):

\[\eta_{uv}^{t}=(1-\beta)\alpha_{uv}^{t}+\beta\tilde{\delta}_{uv}^{t} \tag{8}\]

\(\beta\) is a hyperparameter that controls how much structural semantic information we add into the attention weight. At each layer of intra-type aggregation, the attention weight between nodes perceives the structural semantic information in the fused meta-path graph, and the learnable parameters refine the structural semantic weight, helping the target node to select neighbors more effectively.

Then, we can perform intra-type aggregation as follows:

\[h_{v}^{intra}=\begin{cases}\tilde{h}_{v}&\phi(v)\notin\mathcal{T}^{\prime}\\ \sigma(\sum_{u\in N_{\mathcal{G}_{t}}^{v}}(\eta_{uv}^{t}\cdot\tilde{h}_{u}))& \phi(v)\in\mathcal{T}^{\prime}\end{cases} \tag{9}\]

For the node \(v\) of type \(t\in\mathcal{T}^{\prime}\), the new embedding \(h_{v}^{intra}\) is the weighted sum of its neighbors. Otherwise it stays the same. Then, after the intra-type aggregation phase, \(h_{v}^{intra}\) are fed to the following inter-type aggregation phase.

### _Meta-path-free Inter-type Aggregation_

The inter-type aggregation considers the direct neighbors \(N_{\mathcal{H}}^{v}\) of node \(u\) in the original graph \(\mathcal{H}\). Since nodes and their first-order neighbors often belong to different types, the neighbors of a node reflect its attributes. Thus, the inter-type aggregation is actually the process of continuously integrating the node attributes. When performing information fusion between types, the contribution of different neighbors to the target node is different. For each neighbor \(u\in N_{\mathcal{H}}^{v}\), we can learn a normalized importance weight \(\alpha_{uv}\).

\[\begin{split} e_{uv}=LeakyReLU(a^{T}\cdot W[h_{v}^{intra}||h_{u }^{intra}]),\\ \alpha_{uv}=\frac{exp(e_{uv})}{\sum_{s\in N_{\mathcal{H}}^{v}}exp( e_{us})}\\ h_{v}^{inter}=\sum_{u\in N_{\mathcal{H}}^{v}}\alpha_{uv}\cdot h_{u}^{ intra}\end{split} \tag{10}\]

where \(a\) and \(W\) are learnable parameters shared on all node types. Moreover, to stabilize the learning process and reduce the the large variation caused by the heterogeneity of \(\mathcal{H}\), we further employ the multi-head attention mechanism. Specifically, we implement \(K\) independent attention processes and concatenate their outputs.

\[h_{v}^{inter}=\|_{k=1}^{K}\sum_{u\in N_{\mathcal{H}}^{v}}\alpha_{uv}^{k}\cdot h_ {u}^{intra} \tag{11}\]

Fig. 4: Structural semantic weight of the meta-path APA.
### _Combination of Semantic Information_

Due to the different neighborhoods selected in the aggregation phase, intra-type aggregation and inter-type aggregation actually extract the semantic information encoded in heterogeneous graphs from different perspectives. To explicitly capture the information from these two perspectives, we further propose an information combination method.

\[z_{v}=COMBINE(h_{v}^{intra},W_{m}\cdot h_{v}^{inter}) \tag{12}\]

First, we unify the dimensions of \(h_{v}^{intra}\) and \(h_{v}^{inter}\). \(W_{m}\in\mathbb{R}^{Kd\times d}\) is the learnable transformation matrix. Then, we combine the two embeddings by addition, concatenation or other pooling operations. As we described before, the immediate neighbors of a node contain key attributes, and the meta-path-based neighbors can easily supply high-order connectivity information of the same node type. Hence we choose concatenation as the COMBINE function, that is, we view the representation of the intra-type aggregation phase and the representation of the inter-type aggregation phase as the characteristics of different channels. After the two-phase aggregation, the embedding \(z_{v}\) fusing high-order inter-type information and direct intra-type information is obtained, which can be further used in different downstream tasks. Algorithm 1 shows the details of HAGNN.

```
Input: Heterogeneous graph \(\mathcal{H}\), Selected types \(\mathcal{T}^{\prime}\),  Meta-paths \(\mathcal{P}=\{\mathcal{P}_{t}\ |\ t\in\mathcal{T}^{\prime}\}\),  Node features \(\{x_{v},v\in\mathcal{V}\}\),  Number of intra-type aggregation layers \(L_{intra}\),  Number of inter-type aggregation layers \(L_{inter}\). Output: Embeddings of target nodes fornode type \(t\in\mathcal{T}\)do  perform linear transformation for nodes of type \(t\);  end for /*Building the fused meta-path graph */ for\(t\in\mathcal{T}^{\prime}\)do \(\mathcal{G}_{t}\leftarrow\emptyset\); for\(p_{t}\in\mathcal{P}_{t}\)do  construct \(G^{p_{t}}\); \(\mathcal{G}_{t}=\mathcal{G}_{t}\cup G^{p_{t}}\) ;  end for  end for /*Intra-type aggregation phase */ for\(i=1\to l_{intra}\)do for\(t\in\mathcal{T}^{\prime}\)do for\(v\in\mathcal{V}_{t}\)do for\(u\in N_{\mathcal{G}_{t}}^{v}\)do  calculate learnable weight \(\alpha_{uv}^{t}\) and  structural semantic weight \(\tilde{\delta}_{uv}^{t}\);  combine \(\alpha_{uv}^{t}\) and \(\tilde{\delta}_{uv}^{t}\) with edge residuals;  end for  calculate \(h_{v}^{intra}\) using Equation 9;  end for  end for /*Inter-type aggregation phase */ for\(i=1\to l_{inter}\)do for\(v\in\mathcal{V}\)do  calculate \(h_{v}^{inter}\) using the multi-head attention ;  end for /*Combination of Semantic */ \(z_{v}=h_{v}^{intra}||W_{m}\cdot h_{v}^{inter}\); return\(\{z_{v},\ v\in\mathcal{V}\}\)
```

**Algorithm 1**HAGNN

### _Complexity Analysis_

In this section, we theoretically analyze the time complexity of HAGNN during the training stage and compare it with the typical meta-path-based model HAN and meta-path-free model SimpleHGN. Suppose the number of nodes in the heterogeneous graph is \(N\), the number of edges is \(E\), the dimension of the node raw feature is \(D\), the node embedding dimension after linear transformation is \(D^{\prime}\). For the fused meta-path graph, the number of edges is \(E^{\prime}\) and the number of nodes is \(N^{\prime}\).

For HAGNN, in the linear transformation phase and semantic information combination phase, the complexities are all \(O(NDD^{\prime})\). In the intra-type aggregation phase, attention
is calculated pairwise between nodes in the fused meta-path graph, its complexity is \(O(E^{\prime}D^{\prime}+N^{\prime}D^{\prime 2})\). The inter-type aggregation phase is carried out in the original graph, the time complexity is \(O(ED^{\prime}+ND^{\prime 2})\), which is consistent with GAT.

For SimpleHGN, the complexity of its linear transformation phase is \(O(NDD^{\prime})\). Since it introduces edge features, let the dimension of edge features be \(F\), the number of edge types be \(T\), the complexity of calculating attention and aggregating neighbors is \(O(E(D^{\prime}+F)+ND^{\prime 2}+TF^{2})\).

Overall, the complexity of HAGNN is \(O(NDD^{\prime})+O(E^{\prime}D^{\prime}+ED^{\prime}+ND^{\prime 2})\). The complexity of SimpleHGN is \(O(NDD^{\prime})+O(TF^{2}+EF+ED^{\prime}+ND^{\prime 2})\). It can be seen that the complexity of the two models is similar. The difference is mainly in the intra-type aggregation phase of HAGNN, and the edge feature usage of SimpleHGN.

Meanwhile, the complexity of HAN is \(O(K\epsilon D^{\prime 2}+KD^{\prime 2}+ND^{\prime 2})\), where \(K\) is the number of meta-paths, \(\epsilon\) is the number of edges in each meta-path-based graph. Thanks to the fused meta-path graph, HAGNN can reduce redundant edges and avoid double-layer attention. The experimental results in Section V-D reveal that HAGNN can achieve better performance with higher efficiency than existing HGNNs.

## V Experiments

In this section, we conduct extensive experiments to answer the following questions:

* **RQ1**: How is the effectiveness of the proposed HAGNN compared with existing heterogeneous GNN models?
* **RQ2**: What is the impact of each major component of HAGNN?
* **RQ3**: How about the efficiency of HAGNN?
* **RQ4**: How to evaluate the quality of node representations learned by HAGNN in a visual way?
* **RQ5**: How robust is HAGNN to hyperparameter?
* **RQ6**: Are meta-paths or variants still useful in heterogeneous GNNs? [17] How to select suitable meta-paths?

### _Experimental Setup_

#### V-A1 Experimental Setting

All experiments are conducted under the recently proposed Heterogeneous Graph Benchmark (HGB) [17]. HGB provides unified data split and data pre-processing to ensure the fairness of comparison. In the node classification task, node labels are split according to 24% for training, 6% for validation, and 70% for test in each dataset. In the link prediction task, the test set uses 2-hop neighbors as negative. To prevent data leakage, the evaluation metrics are obtained by submitting predictions to the HGB website2. All experiments are run on a single GPU (NVIDIA Tesla V100) with 32 GB memory.

Footnote 2: [https://www.biendata.xyz/competition/hgb-1/](https://www.biendata.xyz/competition/hgb-1/)

#### V-A2 Datasets

We use five real-world datasets for node classification, node clustering, and link prediction. The statistics of datasets are summarized in Table IV.

* **DBLP** is a computer science bibliography website containing author (A), paper (P), term (T), and conference (C).
* **IMDB** is a movie website, which contains movie (M), director (D), actor (A), and keyword (K).
* **Freebase** is a huge knowledge graph with book (B), film (F), music (M), organization (O), business (U), etc.
* **LastFM** is an online music website containing user (U), artist (A), and tag (T). The target edge type is user-artist.
* **PubMed** is a biomedical literature library, which has gene (G), disease (D), chemical (C), and specie (S). The target is to predict the connection between diseases.

#### V-A3 Implementation Details

The parameters are randomly initialized. We use Adam [36] to optimize parameters. \(\beta\) in intra-type aggregation is set to 0.3. We set the number of intra-type aggregation layers to 2 for all datasets, the number of inter-type aggregation layers to 5 for the IMDB dataset and 2 for other datasets. All GNN models are implemented with PyTorch. The selected meta-paths are listed in Table X (Section V-G).

### _Performance Comparison (RQ1)_

#### V-B1 Node Classification

We select baselines depending on whether or not the meta-path is used.

* **Meta-path-based:** HAN [10], GTN [12], MAGNN [11], HetSANN [37], HGNN-AC [38], R-HGNN [31], and CKD [13].
* **Meta-path-free:** R-GCN [20], HGT [39], HetGNN [32], GCN [26], GAT [18], and SimpleHGN [17].

We run these baselines using the official codes. We adopt the Macro F1 and Micro F1 metrics for node classification. All models are run five times and the mean and standard deviation are reported. The results are shown in Table V.

Neither the meta-path-based models or the meta-path-free models can always achieve better performance on all datasets. For meta-path-based models, HAN, MAGNN, and R-HGNN performs better on the DBLP dataset. R-HGNN and CKD performs better on the IMDB dataset. Nevertheless, in contrast with the meta-path-based models, the meta-path-free models such as the commonly used GAT and GCN can also achieve competitive or even better performance. Also, SimpleHGN serves as a strong baseline, indicating that it is necessary to fuse different types of information in the node classification task. The results show that intra-type aggregation and inter-type aggregation have their own advantages and disadvantages. For relatively large datasets such as Freebase, existing models run out of memory due to either focusing too much on meta-path (e.g., MAGNN, GTN, HetSANN) or too high model complexity (e.g., HetGNN).
In comparison, HAGNN can not only run on large datasets but also consistently achieves the best performance on all three datasets, which demonstrates that intra-type and inter-type aggregations are complementary and making good use of type information in heterogeneous graphs is crucial.

#### Iv-B2 Node Clustering

To verify the quality of node embeddings generated by different models, we conduct node clustering on the IMDB and DBLP datasets. The labeled nodes (i.e., movies in IMDB and authors in DBLP) are clustered with the \(k\)-means algorithm. The number of clusters in \(k\)-means is set to the number of classes for each dataset, i.e., 3 for IMDB and 4 for DBLP. We employ the normalized mutual information (NMI) and the adjusted rand index (ARI) as evaluation metrics. From Table VI, we see that HAGNN regularly outperforms all other baselines in the node clustering task. Note that all models perform significantly worse on IMDB than on DBLP. This is due to the dirty labels of movies in IMDB, i.e., every movie node in the original IMDB dataset has multiple genres, and we only choose the very first one as its class label [11]. As shown in Table VI, the traditional heterogeneous models do not have many advantages over the traditional homogeneous models in node clustering. And the node embedding generated by HAGNN has higher quality, leading to better clustering effect.

#### Iv-B3 Link Prediction

We select widely used link prediction models as baselines, including RGCN, GATNE [40], HetGNN, MAGNN, HGT, GCN, GAT, and SimpleHGN. The link prediction task is performed on the LastFM and Pubmed datasets. We adopt the MRR and ROC-AUC metrics and the mean and standard deviation of five runs are reported. From Table VII, we see that GCN, GAT, and other direct neighbor aggregation models perform better than MAGNN and HGT on the LastFM dataset. This is mainly due to the weak heterogeneity of LastFM, which contains only three node types and three edge types. The heterogeneous graph models on the PubMed dataset can achieve comparable results. In comparison, HAGNN outperforms existing models on both datasets, especially on PubMed, where the MRR metric improves by 2% compared to SimpleHGN.

Overall, HAGNN achieves better performance in different tasks and different datasets. Moreover, HAGNN can easily handle larger datasets.

### _Ablation Study (RQ2)_

We design the following variants of HAGNN.

* **HAGNN-wo-inter** removes the inter-type aggregation phase and directly performs downstream tasks on the embeddings obtained after the intra-type aggregation phase.
* **HAGNN-wo-intra** removes the intra-type aggregation phase and aggregates the direct neighbors of all nodes.
* **HAGNN-wo-sw** follows the two-phase framework, but in the intra-type aggregation phase, the semantic structure information is not utilized.
* **HAGNN-wo-fused** conducts intra-type aggregation on meta-path-based graphs rather than the fused meta-path graph.
* **HAGNN-wo-combine** removes the combination of the embedding obtained in intra-type aggregation and inter-type aggregation.
* **HAGNN-combine-add** uses the _add_ operation to combine the the embedding obtained in intra-type aggregation and inter-type aggregation.

Table VIII shows the performance comparison between HAGNN and its six variants on the node classification (i.e, DBLP and IMDB) and link prediction (i.e., PubMed) tasks. HAGNN-wo-intra performs better than HAGNN-wo-inter on the DBLP dataset. While HAGNN-wo-inter is much better than HAGNN-wo-intra on the PubMed dataset. No matter which variant, its performance is far inferior to HAGNN. That is, both the higher-order intra-type information and direct inter-type information to the target nodes are important, but their importance varies in different datasets. The proposed two-phase aggregation method in HAGNN can leverage the information within and between types simultaneously.

Moreover, the performance of HAGNN-wo-sw is inferior to that of HAGNN, which indicates that the proposed structural semantic weight helps to aggregate intra-type information. Also, HAGNN-wo-fused performs worse than HAGNN, especially on the IMDB and PubMed datasets. The main reason is that the information redundancy between multiple meta-path-based graphs plays an negative effect on the performance. In addition, HAGNN-wo-combine and HAGNN-combine-add perform worse than HAGNN, which indicates that the concatenation combination can work better.

### _Efficiency Comparison (RQ3)_

We further evaluate and compare the efficiency of HAGNN and SimpleHGN from the following perspectives, i.e., parameter size, FLOPs, and runtime per training epoch. Let HANGNN denote the heterogeneous GNN that use HAN's meta-path-based graphs instead of the proposed fused meta-path graph in the intra-type aggregation phase. From Table IX, we see that HAGNN achieves the best efficiency among the three models, which is consistent with the complexity analysis in Section IV-F. Moreover, during the training process, HAGNN is also faster than SimpleHGN and HAN-HAGNN. Although HAGNN is composed two phases (i.e., intra-type and inter-type aggregation phases), HAGNN can achieve better performance with higher efficiency than existing models with one aggregation phase due to the fused meta-path graph and structural semantic aware aggregation mechanism.

### _Visualization (RQ4)_

#### Iv-E1 Quality of node representation

To demonstrate the quality of node representations, we project the low-dimensional node embeddings into the two-dimensional space using t-SNE [41], and the visualization results are shown in Figure 5. Different colors represent different classes. We can see that HAGNN can generate more clear classification boundaries than commonly used models (i.e., MAGNN and GAT). Moreover, the point distribution in the same class is more close, indicating that the node embedding generated by HAGNN has higher quality. The clustering results of other baselines can be seen in Table VI. Overall, visualization results further demonstrate the effectiveness of HAGNN.

#### Iv-E2 Difference Between Intra-type Aggregation and Inter-type Aggregation

Due to the different neighborhoods selected in the aggregation phase, intra-type aggregation and inter-type aggregation actually extract the semantic information encoded in heterogeneous graphs from different perspective. To further verify the difference between the information obtained in the two phases, we project node embeddings of the target type generated by each phase into a three-dimensional space using

Fig. 5: Node visualization on the DBLP dataset.
t-SNE. The result is illustrated in Figure 6. We see that the node embedding of intra-type aggregation is denser while the node embedding of inter-type aggregation is looser. More importantly, the two types of embeddings are distributed on different planes, which indicates that they extract the semantic information from different perspectives.

### _Hyperparameter Sensitivity (RQ5)_

\(\beta\) in Equation 8 controls how much structural semantic information added into the attention weight. We further evaluate the hyperparameter sensitivity on \(\beta\). The value of \(\beta\) is selected in \([0.1,0.2,0.3,0.4,0.5]\). Figure 7 shows the performance comparison of HAGNN with different \(\beta\). It can be seen that HAGNN is very robust to \(\beta\), and the performance change is insignificant. The main reason is that we use the learnable adaptive weight and the structural semantic weight simultaneously, which can increase the robustness of HAGNN.

### _Discussion on Meta-path (RQ6)_

Previous work [17] has questioned the effect of meta-paths. The performance improvement in HAGNN verifies that meta-paths are still indispensable for heterogeneous graphs. Table X shows the selected meta-paths in HAGNN. To verify the effectiveness of meta-paths, we further conduct experiments with different meta-paths in Table XI. As shown in Table XI, for the DBLP dataset, meta-paths need to be selected for all types in \(\mathcal{T}^{\prime}\). If only one type is selected, the performance will be degraded. Moreover, the selection of meta-paths should be comprehensive, which can be reflected on the IMDB dataset.

Another question is, how to choose a suitable meta-path. Meta-paths are important, but arbitrarily choosing meta-paths may cause damage to the model performance. We divide meta-paths into two categories: **strong-relational** and **weak-relational meta-paths**. Strong-relational meta-path-based graph tend to be sparse, while weak-relational meta-path-based graph are denser. Take the DBLP dataset as an example. The meta-path 'APA' represents the co-author relationship, which is a strong relationship, and the average degree of the 'APA'-based-graph is only 3. 'APTPA' is a weak relationship because two articles with the same keyword do not indicate that the two authors are closely related. The average degree of the corresponding meta-path-based graph is 1232.

We believe that the sparse strong relationship is difficult to provide enough information in aggregation, while the dense weak relationship provides too much redundant information. Thus, we can choose the combination of strong relationships and weak relationships. Moreover, due to the propose fused meta-path graph and the structural semantic aware aggregation mechanism, HAGNN can effectively avoid redundant information and introduce the weight of strong relationships into the learning of the attention weight.

## VI Conclusion

To utilize the intra-type and inter-type information in the heterogeneous graph, we proposed a novel hybrid aggregation framework called HAGNN, which consists of two phases: meta-path-based intra-type aggregation and meta-path-free inter-type aggregation. To make the intra-type aggregation more effectively, we proposed the fused meta-path graph to capture the meta-path neighbors for a specific node type. Moreover, we proposed a structural semantic aware aggregation mechanism to improve the leaning of attention weights in the intra-type aggregation. Extensive experimental results on

Fig. 6: Node embedding visualization of intra-type and inter-type aggregation phases on the DBLP dataset.

Fig. 7: Performance comparison of HAGNN under different \(\beta\)
five heterogeneous graph datasets reveal that HAGNN outperforms the existing heterogeneous GNNs on node classification and link prediction tasks. Finally, we discussed the role of meta-paths in the heterogeneous GNNs.

## Acknowledgment

This work was supported by the National Natural Science Foundation of China (#62102177), the Natural Science Foundation of Jiangsu Province (#BK20210181), the Key R&D Program of Jiangsu Province (#BE2021729), Open Research Projects of Zhejiang Lab (#2022PG0AB07), and the Collaborative Innovation Center of Novel Software Technology and Industrialization, Jiangsu, China.
