Graph Convolution Network (GCN) has been recognized as one of the most effective graph models for semi-supervised learning, but it extracts merely the first-order or few-order neighborhood information through information propagation, which suffers performance drop-off for deeper structure. Existing approaches that deal with the higher-order neighbors tend to take advantage of adjacency matrix power. In this paper, we assume a seemly trivial condition that the higher-order neighborhood information may be similar to that of the first-order neighbors. Accordingly, we present an unsupervised approach to describe such similarities and learn the weight matrices of higher-order neighbors automatically through Lasso that minimizes the feature loss between the first-order and higher-order neighbors, based on which we formulate the new convolutional filter for GCN to learn the better node representations. Our model, called higher-order weighted GCN (HWGCN), has achieved the state-of-the-art results on a number of node classification tasks over Cora, Citeseer and Pubmed datasets. Convolutional neural networks (CNNs) have made great achievements on a wide range of image tasks including image classification (Simonyan & Zisserman, 2015; Szegedy et al., 2015; Huang et al., 2017) , object detection (Girshick et al., 2014; Redmon et al., 2016; Liu et al., 2016; Dai et al., 2016; Lin et al., 2017) , semantic segmentation (Long et al., 2015; Badrinarayanan et al., 2017; Chen et al., 2017) , etc. Due to the fact that their underlying data representation has a grid-like structure, CNNs perform highly effective on image processing, and can thus capture local patterns by compressing the hypothesis space and using local filters to learn the parameters. However, lots of real-word data cannot be represented as grid-like structure. For example, social networks and biological networks are usually represented as graphs instead of grid-like structure, while the data defined on 3D meshes is important for many graphical applications. As a result, there is an increasing number of fields that focus on studying non-Euclidean structured data. To address such challenge, inspired by the great success of applying CNNs to computer vision tasks, many research efforts have been devoted to a paradigm shift in graph learning that generalizes convolutions to the graph domain. More specifically, the graph structure is encoded using a convolutional neural network model to operate the neighborhood of each node in graphs. In general, attempts in this direction can be categorized into non-spectral (spatial) approaches and spectral approaches. While recent works are making progress on these two lines of research respectively, here, we focus on the extension of the graph convolution spectral filter. In this respect, the base work that applies a localized neighbor filter to achieve convolutional architecture is the graph convolutional network (GCN) (Kipf & Welling, 2017) . However, GCN merely considers the first-order neighbors, resting on which multiply layers are directly stacked to learn the multi-scale information, while it has been observed in many experiments that deeper GCN could not improve the performance and even performs worse (Kipf & Welling, 2017) . In other words, such convolutional filter limits the representational capacity of the model (Abu-El-Haija et al., 2019) . In this work, we propose a new model, HWGCN, for convolutional filter formulation that is capable of mixing its neighborhood information at different orders to capture the expressive representations from the graph. Considering that convolution kernels of different sizes may extract different aspects or information from the input images, similarly, the size of convolutional filter plays a very important role for neighborhood mixing in graph convolutions. Researchers have recently made some attempts to deal with higher-order neighbors for the convolutional filter (Abu-El-Haija et al., 2019; Liao et al., 2019) . Instead of using adjacency matrix power with potential information overlap at different orders, in our proposed graph model HWGCN, we bring an important insight to leverage node features in addition to graph structure for convolutional filter formulation, which allows a refined architecture to code better with neighbor selection at different distances, and thus learn better node representations from first-order and higher-order neighbors. Our contributions are four-fold. Firstly, we analyze the GCN and demonstrate the importance of similarity between first-order and higher-order information. Secondly, we build the convolutional filters with first-order and higher-order neighbors rather than the local neighborhood considered in previous work. Thirdly, we leverage Lasso and the information of node features and graph structure to minimize the feature loss between the first-order and higher-order neighbors to effectively aggregate the higher-order information in a weighted, orthogonal, and unsupervised fashion, unlike existing models that merely utilize graph structure. Fourthly, we conduct comprehensive experimental studies on a number of datasets, which demonstrate that HWGCN can achieve the state-of-the-art results in terms of classification accuracy. The original GCN updates the state of nodes by the aggregation of feature information from directly neighboring nodes in every convolutional layer, but fails to learn the higher-order neighborhood information through operating multiply layers; its performance suffers a drop-off when it adjusts the number of layers over two. To address this, some recent research efforts have been conducted on mixing neighborhood information at different distances to improve the expressive power of graph convolutions, which are promising yet limiting to adjacency matrix power. In this paper, we propose a novel model HWGCN to formulate the convolutional filter to regularize first-order and higherorder neighbors in a weighted and orthogonal fashion, where node features and graph structure are leveraged to minimize feature loss through Lasso, extract relevant higher-order neighborhood information, and thus learn better node representations. Our method is a generic framework which can be further applied to various graph convolution network models. The experimental results based on the three standard citation network benchmark dataset demonstrate state-of-the-art performance being achieved, which match or outperform other baselines.
We propose HWGCN to mix the relevant neighborhood information at different orders to better learn node representations. The authors propose a variant of GCN, HWGCN, to consider convolution beyond 1-step neighbors, which is comparable to state-of-the-art methods.