Interpretable Graph Neural Networks for Tabular Data
Data in tabular format is frequently occurring in real-world applications.
Graph Neural Networks (GNNs) have recently been extended to effectively handle
such data, allowing feature interactions to be captured through representation
learning. However, these approaches essentially produce black-box models, in
the form of deep neural networks, precluding users from following the logic
behind the model predictions. We propose an approach, called IGNNet
(Interpretable Graph Neural Network for tabular data), which constrains the
learning algorithm to produce an interpretable model, where the model shows how
the predictions are exactly computed from the original input features. A
large-scale empirical investigation is presented, showing that IGNNet is
performing on par with state-of-the-art machine-learning algorithms that target
tabular data, including XGBoost, Random Forests, and TabNet. At the same time,
the results show that the explanations obtained from IGNNet are aligned with
the true Shapley values of the features without incurring any additional
computational overhead.
Amr Alkhatib, Sofiane Ennadir, Henrik Boström, Michalis Vazirgiannis
# Interpretable Graph Neural Networks for Tabular Data

###### Abstract

Data in tabular format is frequently occurring in real-world applications. Graph Neural Networks (GNNs) have recently been extended to effectively handle such data, allowing feature interactions to be captured through representation learning. However, these approaches essentially produce black-box models, in the form of deep neural networks, precluding users from following the logic behind the model predictions. We propose an approach, called IGNNet (Interpretable Graph Neural Network for tabular data), which constrains the learning algorithm to produce an interpretable model, where the model shows how the predictions are exactly computed from the original input features. A large-scale empirical investigation is presented, showing that IGNNet is performing on par with state-of-the-art machine-learning algorithms that target tabular data, including XGBoost, Random Forests, and TabNet. At the same time, the results show that the explanations obtained from IGNNet are aligned with the true Shapley values of the features without incurring any additional computational overhead.

## 1 Introduction

In some application domains, e.g., medicine and law, predictions made by machine learning models need justification for legal and ethical considerations Lakkaraju et al. (2017); Goodman and Flaxman (2017). In addition, users may put trust in such models only with a proper understanding of the reasoning behind the predictions. A direct solution is to use learning algorithms that produce interpretable models, such as logistic regression Berkson (1944), which provides both local (instance-specific) and global (model-level) explanations for the predictions. However, such algorithms often result in a substantial loss in predictive performance compared to algorithms that generate black-box models, e.g., XGBoost Chen and Guestrin (2016), Random Forests Breiman (2001), and deep learning algorithms Pintelas et al. (2020); Mori and Uchihira (2019). Post-hoc explanation techniques, e.g., SHAP Lundberg and Lee (2017), LIME Ribeiro et al. (2016), and Anchors Ribeiro et al. (2018), have been put forward as tools to explain predictions of the black-box models. However, the explanations provided by such techniques are limited in that they either do not show how exactly the predictions are computed, but merely present feature scores, such as LIME and SHAP, or come with no guarantees on the fidelity, i.e., that the provided explanation agrees with the underlying model Yeh et al. (2019); Delaunay, Galarraga, and Largouet (2020). As extensively argued in Rudin (2019), there are hence several reasons to consider generating interpretable models in the first place, if trustworthiness is a central concern.

Graph Neural Networks (GNNs) have emerged as a powerful framework for representation learning of graph-structured data Xu et al. (2019). The application of GNNs has been extended to tabular data, where a GNN can be used to learn an enhanced representation for the data points (rows) or to model the interaction between different features (columns). TabGNN Guo et al. (2021) is an example of the first approach, where each data point is represented as a node in a graph. In comparison, TabularNet Du et al. (2021) and Table2Graph Zhou et al. (2022) follow the second approach, where the first uses a Graph Convolutional Network to model the relationships between features, and the second learns a probability adjacency matrix for a unified graph that models the interaction between features of the data points. GNNs can also be combined with other algorithms suited for tabular data, e.g., as in BGNN Ivanov and Prokhorenkova (2021), which combines gradient-boosted decision trees and a GNN in one pipeline, where the GNN addresses the graph structure and the gradient-boosted decision trees handle the heterogeneous features of the tabular data. To the best of our knowledge, all previous approaches to using GNNs for tabular data result in black-box models and they are hence associated with the issues discussed above when applied in contexts with strong requirements on trustworthiness. In this work, we propose a novel GNN approach for tabular data, with the aim to eliminate the need to apply post-hoc explanation techniques without sacrificing predictive performance.

The main contributions of this study are:

* a novel approach, called Interpretable Graph Neural Network for tabular data (IGNNet), that exploits powerful graph neural network models while still being able to show exactly how the prediction is derived from the input features in a transparent way
* a large-scale empirical investigation evaluating the explanations of IGNNet as well as comparing the predictive performance of IGNNet to state-of-the-art approaches for tabular data; XGBoost, Random Forests, and multi-layer
perceptron (MLP), as well as to an algorithm generating interpretable models; TabNet [12]
* an ablation study comparing the performance of the proposed approach to a black-box version, i.e., not constraining the learning algorithm to produce transparent models for the predictions

In the next section, we briefly review related work. In Section 3, we describe the proposed interpretable graph neural network. In Section 4, results from a large-scale empirical investigation are presented and discussed, in which the explanations of the proposed method are evaluated and the performance is compared both to interpretable and powerful black-box models. Finally, in the concluding remarks section, we summarize the main conclusions and point out directions for future work.

## 2 Related Work

In this section, we provide some pointers to self-explaining (regular and graph) neural networks, and briefly discuss their relation to model-agnostic explanation techniques and interpretable models. We also provide pointers to work on interpretable deep learning approaches for tabular data.

### Self-Explaining Neural Networks

Several approaches to generating so-called self-explaining neural networks have been introduced in the literature; in addition to generating a prediction model, they all incorporate a component for explaining the predictions. They can be seen as model-specific explanation techniques, in contrast to model-agnostic techniques, such as LIME and SHAP, but sharing the same issues regarding fidelity and lack of detail regarding the exact computation of the predictions.

Approaches in this category include the method in [13], which is an early self-explaining neural network for text classification, the Contextual Explanation Network (CEN) [12], which generates explanations using intermediate graphical models, the Self-explaining Neural Network (SENN) [1], which generalizes linear classifiers to neural networks using a concept autoencoder, and the CBM-AUC [14], which improves the efficiency of the former by replacing the decoder with a discriminator. Some approaches generate explanations in the form of counterfactual examples, e.g., CounterNet [15], and Variational Counter Net (VCNet) [16]. Again, such explanations do not provide detailed information on how the original predictions are computed and how exactly the input features affect the outcome.

### Self-Explaining Graph Neural Networks

The Self-Explaining GNN (SE-GNN) [1] uses similarities between nodes to make predictions on the nodes' labels and provide explanations using the most similar K nodes with labels. ProtGNN [17] also computes similarities, but between the input graph and prototypical graph patterns that are learned per class. Cui et al. [18] proposed a framework to build interpretable GNNs for connectome-based brain disorder analysis that resembles the signal correlation between different brain areas.

Similar to the (standard) self-explaining neural networks, the approaches that target graph neural networks provide abstract views of the predictions; the users cannot trace the exact computations, in contrast to when using interpretable models, which in principle allows for the inferences to be executed by hand.

### Interpretable Deep Learning for Tabular Data

In an endeavor to provide an interpretable regression model for tabular data while retaining the performance of deep learning models, and inspired by generalized linear models (GLM), LocalGLMnet was proposed to make the regression parameters of a GLM feature dependent, allowing for quantifying variable importance and also conducting variable selection [15]. TabNet [12] is another interpretable method proposed for tabular data learning, which employs a sequential attention mechanism and learnable masks for selecting a subset of meaningful features to reason from at each decision step. The feature selection is instance-based, i.e., it differs from one instance to another. The feature selection masks can be visualized to highlight important features and show how they are combined. However, it is not obvious how the features are actually used to form the predictions.

## 3 The Proposed Approach: IGNNet

This section describes the proposed method to produce an interpretable model using a graph neural network. We first outline the details of a GNN for graph classification and then show how it can be constrained to produce interpretable models. Afterward, we show how it can be applied to tabular data. Finally, we show how the proposed approach can maintain both interpretability and high performance.

### Interpretable Graph Neural Network

The input to a GNN learning algorithm is a set of graphs denoted by \(\mathcal{G}=(V,E,X,\mathcal{W})\), consisting of a set of nodes \(V\), a set of edges \(E\), a set of node feature vectors \(X\), and a set of edge weights \(\mathcal{W}\), where \(V=\{v_{1},\dots,v_{N}\}\), \(E\subseteq\{(v_{i},v_{j})|v_{i},v_{j}\in V\}\), \(X=\{\mathbf{x}_{1},\dots,\mathbf{x}_{N}\}\), and the weight of edge \((v_{i},v_{j})\) is represented by a scalar value \(\delta_{i,j}\) in the set of edge weights \(\mathcal{W}\), where \(\delta_{i,j}=\mathcal{W}(i,j)\). A GNN algorithm learns a representation vector \(\mathbf{h}_{i}\) for each node \(v_{i}\), which is initialized as \(\mathbf{h}_{i}^{(0)}=\mathbf{x}_{i}\). The key steps in a GNN for graph classification can be summarized by the following two phases [12]:

1. **Message Passing:** Each node passes a message to the neighboring nodes, then aggregates the passed information from the neighbors. Finally, the node representation is updated with the aggregated information. A neural network can also be used to learn some message functions between nodes. The message passing phase can be formulated as:
\[\textbf{h}_{i}^{(l+1)}=\varphi(\textbf{w}^{(l+1)}(\sum_{u\in\mathcal{N}(i)}\delta_ {i,u}\textbf{h}_{u}^{(l)}+\delta_{i,i}\textbf{h}_{i}^{(l)})) \tag{1}\]

where \(\delta_{i,u}\) is the weight assigned to the edge between node \(v_{i}\) and node \(v_{u}\). \(\textbf{h}_{i}^{(l)}\) is the hidden representation of the node \(v_{i}\) in the \(l\)-th layer, \(\textbf{w}^{(l+1)}\) represents the learnable parameters, and \(\varphi\) is a non-linearity function.

The adjacency matrix \(\boldsymbol{A}\) of size \(|V|\times|V|\) contains the edge weights and can be normalized similar to a Graph Convolutional Network (GCN) [10] as shown in (2).

\[\tilde{\boldsymbol{A}}=\boldsymbol{D}^{-\frac{1}{2}}\boldsymbol{A}\boldsymbol {D}^{-\frac{1}{2}} \tag{2}\]

Here \(\boldsymbol{D}\) is the degree matrix \(\boldsymbol{D}_{ii}=\sum_{j}\boldsymbol{A}_{ij}\)[10].

**Graph Pooling (Readout):** A representation of the whole graph \(\mathcal{G}\) is learned using a simple or advanced function [13], e.g, sum, mean, or MLP.

The whole graph representation obtained from the **graph pooling** phase can be submitted to a classifier to predict the class of the graph, which can be trained in an end-to-end architecture [11, 12].

_The pooling function can be designed to provide an interpretable graph classification layer_. Thus, the final hidden representation of each node is mapped to a single value, for instance, through a neural network layer or dot product (\(f(\textbf{h}_{i}^{(l+1)}\in\mathbb{R}^{n})=h_{i}\in\mathbb{R}^{1}\)), and concatenated to obtain the final representation **g** of the graph \(\mathcal{G}\) where a scalar value in **g** corresponds to a node in the graph. Consequently, if a set of weights is applied to classify the graph, we can trace the contribution of each node to the predicted outcome, i.e., the user can find out which nodes contributed to the predicted class. For example, **g** can be used directly as follows:

\[\hat{y}=\text{link}(\sum_{i=1}^{n}\textbf{w}_{i}\textbf{g}_{i}) \tag{3}\]

where \(\textbf{w}_{i}\) is the weight assigned to node \(v_{i}\) represented in \(\textbf{g}_{i}\).The link function is applied to accommodate a valid range of outputs, e.g., the sigmoid function for binary and softmax for multi-class classification. This is equivalent to:

\[\hat{y}=\text{link}(\sum_{i=1}^{n}\textbf{w}_{i}f(\textbf{h}_{i}^{(l+1)})) \tag{4}\]

In the case of binary classification, one vector of weights (**w**) is applied, and for multiple classes, each class has a separate vector of weights.

### Representing Tabular Data Points as Graphs

The proposed readout function in the previous subsection allows for determining the contribution of each node in a prediction, if a white-box classification layer is used for the latter. Therefore, we propose representing each data instance as a graph where _the features are the nodes_ of that graph and the _linear correlation between features are the edge weights_, as we assume that not all features are completely independent. The initial representation of a node is a vector of one dimension, and the value is just the feature value, which can be embedded into a higher dimensionality. The idea is outlined in Algorithm 1 and illustrated in Figure 1.

```
Data: a set of graphs \(\mathbb{G}\) and labels \(y\) Result: Model parameters \(\theta\) Initialize \(\theta\) fornumber of training iterationsdo \(\mathcal{L}\gets 0\) foreach \(\mathcal{G}_{j}\in\mathbb{G}\)do \(\boldsymbol{H}_{j}\leftarrow\)messagePassing\((\mathcal{G}_{j})\) \(\textbf{g}_{j}\leftarrow\)readout\((\textbf{H}_{j})\) \(\hat{y}_{j}\leftarrow\)predict\((\textbf{g}_{j})\) \(\mathcal{L}\leftarrow\mathcal{L}+\)loss\((\hat{y}_{j},y_{j})\) end  Compute gradients \(\nabla_{\theta}\mathcal{L}\) Update \(\theta\leftarrow\theta-\nabla_{\theta}\mathcal{L}\)  end for
```

**Algorithm 1**IGNNet

In order to make a prediction for a test instance, the data point has to be converted to a graph using the same procedure for building the input graphs to IGNNet, and for which graph node representations are obtained using a GNN with parameters \(\theta\). Finally, the output layer is used to form the prediction.

### How can IGNNet achieve high performance while maintaining interpretability?

An expressive GNN can potentially capture complex patterns and dependencies in the graph, allowing nodes to be mapped to distinct representations based on their characteristics and relationships [11]. Moreover, a GNN with an injective aggregation scheme can not only distinguish different structures but also map similar structures to similar representations [13]. Therefore, if the tabular data are properly presented as graphs, GNNs with the aforementioned expressive capacities can model relationships and interactions between features, and consequently approximate complex non-linear mappings from inputs to predictions. On top of that, it has been shown by [10] that GCNs based on 1-Lipschitz continuous activation functions can be improved in stability and robustness with Lipschitz normalization and continuity analysis; similar findings have also been demonstrated on graph attention networks (GAT) [1]. This property is of particular importance when the application domain endures adversarial attacks or incomplete tabular data.

The proposed readout function in subsection 3.1 can produce an interpretable output layer. However, it does not guarantee the interpretability of the whole GNN without message-passing layers that consistently maintain relevant representations of the input features. Accordingly, we constrain the message-passing layer to produce interpretable
models using the following conditions:

1. Each feature is represented in a distinct node throughout the consecutive layers.
2. Each node is bounded to interact with a particular neighborhood, where it maintains correlations with the nodes within that neighborhood. As a result, the aggregated messages could potentially hold significance to the input feature values.

Since the edge weights are the correlation values, they determine the strength and the sign of the messages obtained from a neighborhood, allowing each node to store information not only concerning the original input feature but also features that are correlated with it. The proposed graph pooling function, combined with the constrained message-passing layers that keep representative information about the input features, allows tracking each feature's contribution at the output layer and also through the message-passing layers all the way to the input features.

## 4 Empirical Investigation

This section evaluates both the explanations and predictive performance of IGNNet. We begin by outlining the experimental setup, then by evaluating the explanations produced by IGNNet, and lastly, we benchmark the predictive performance.

### Experimental Setup

A GNN consists of one or more message-passing layers, and each layer can have a different design, e.g., different activation functions and batch normalization, which is the intra-layer design level [20]. There is also the inter-layer design level which involves, for instance, how the message-passing layers are organized into a neural network and if the skip connections are added between layers [20]. While the intra-layer and inter-layer designs can vary based on the nature of the prediction task, we propose a general architecture for our empirical investigation. However, it is up to the user to modify the architecture of the GNN. The number of message-passing layers, number of units in linear transformations, and other hyperparameters were found based on a quasi-random search and evaluation on development sets of the following three datasets: Churn, Electricity, and Higgs. We have six message-passing layers in the proposed architecture, each with a Relu activation function. Multiple learnable weights are also applied to the nodes' representation, followed by a Relu function. Besides three batch normalization layers, four skip connections are added as illustrated in Figure 2. After all the GNN layers, we use a feedforward neural network (FNN) to map the multidimensional representation of each node into a single value. In the FNN, we do not include any activation functions in order to keep the mapping linear, but a sigmoid function is applied after the final layer to obtain a value between 0 and 1 for each node. The FNN is composed of 8 layers with the following numbers of units (128, 64, 32, 16, 8, 4, 2, 1) and 3 batch normalization layers after the second, fourth, and sixth hidden layers. After the FNN, the nodes' final values are concatenated to form a representation of the whole graph (data instance). Finally, the weights that are output are used to make predictions. The GNN is trained end-to-end, starting from the embeddings layer and ending with the class prediction.

We also provide an opaque variant of IGNNet (OGNNet, opaque graph neural net for tabular data), where the FNN, along with the output layer, is replaced by an MLP of one hidden layer with 1024 hidden units and a Relu activation function. All the learned node representations just before the FNN are concatenated and passed to the MLP for class prediction. The OGNNet is introduced to determine, by an ablation study, how much predictive performance we may lose by squashing the learned multidimensional representation of the nodes into scalar values and applying a white box classifier instead of a black box.

In the experiments, 35 publicly available datasets are used.1 Each dataset is split into training, development, and test sets. The development set is used for overfitting detection and early stopping of the training process, the training set is used to train the model, and the test set is used to evaluate the model.2 For a fair comparison, all the compared learning algorithms are trained without hyperparam

Figure 1: **An overview of our proposed approach. Each data instance is represented as a graph by embedding the feature values into a higher dimensionality, and the edge between two features (nodes) is the correlation value. Multiple iterations of message passing are then applied. Finally, the learned node representation is projected into a single value, and a whole graph representation is obtained by concatenating the projected values.**
eters tuning using the default settings on each dataset. In cases where the learning algorithm does not employ the development set to measure performance progress for early stopping, the development and training subsets are combined into a joint training set. The adjacency matrix uses the correlation values computed on the training data split. The weight on edge from the node to itself (self-loop) is a user-adjustable hyperparameter, constitutes between 70% to 90% (on average) of the weighted summation to keep a strong message per node that does not fade out with multiple layers of message-passing. Weak correlation values are excluded from the graph, so if the absolute correlation value is below 0.2, the edge is removed unless no correlation values are above 0.2; in case of the latter, the procedure is repeated using a reduced threshold of 0.05.3 The Pearson correlation coefficient (Pearson 1895) is used to estimate the linear relationship between features. In the data preprocessing step, the categorical features are binarized using one-hot encoding, and all the feature values are normalized using min-max normalization (the max and min values are computed on the training split). The normalization keeps the feature values between 0 and 1, which is essential for the IGNNet to have one scale for all nodes.

Footnote 3: In the technical appendix, we provide an ablation study on the effects of different hyperparameter preferences on predictive performance.

The following algorithms are also evaluated in the experiments: XGBoost, Random Forests, MLP and TabNet. XGBoost and Random Forests are trained on the combined training and development sets. The MLP has two layers of 1024 units with Relu activation function and is trained on the combined training and development sets with early stopping and 0.1 validation fraction. TabNet is trained with early stopping after 20 consecutive epochs without improvement on the development set, and the best model is used in the evaluation.

For imbalanced binary classification datasets, we oversample the minority class in the training set to align the size with the majority class. All the compared algorithms are trained using the oversampled training data. While for multi-class datasets, no oversampling is conducted. The area under the ROC curve (AUC) is used to measure the predictive performance, as it is not affected by the decision threshold. For the multi-class datasets, weighted AUC is calculated, i.e., the AUC is computed for each class against the rest and weighted by the support.

### Evaluation of Explanations

The feature scores produced by IGNNet should ideally reflect the contribution of each feature toward the predicted outcome and, therefore, they should be equivalent to the true Shapley values. As it has been shown that KernelSHAP converges to the true Shapley values when provided with an infinite number of samples (Covert and Lee 2021; Jethani et al. 2022), it is anticipated that the explanations generated by KernelSHAP will progressively converge to more similar values to the scores of IGNNet as the sampling process continues. This convergence arises from KernelSHAP moving towards the true values, while the scores of IGNNet are expected to align with these true values. To examine this conjecture, we explain IGNNet using KernelSHAP and measure the similarity between KernelSHAP's explanations and IGNNet's scores following each iteration of data sampling and KernelSHAP evaluation.4 For the feasibility of the experiment, 500 examples are randomly selected from the test set of each dataset to be explained. The cosine similarity and Spearman rank-order correlation are used to quantify the similarity between explanations. The cosine similarity measures the similarity in the orientation (Han, Kamber, and Pei 2012), while the Spearman rank-order measures the similarity in ranking the importance scores (Rahnama et al. 2021).

Footnote 4: KernelSHAP experiments were conducted using the following open-source implementation: [https://github.com/iancovert/shapley-regression](https://github.com/iancovert/shapley-regression)

The results demonstrate a general trend wherein KernelSHAP's explanations converge to more similar values to IGNNet's scores across various data instances and the 35

Figure 2: **IGNNet default architecture. It starts with the embedding layer, a linear transformation from one dimension to 64 dimensions. A Relu activation function follows each message-passing layer and each green block as well. The feedforward network at the end has no activation functions between layers to ensure a linear transformation into a single value. A sigmoid activation function follows the feedforward network to obtain the final value for each feature between 0 and 1.**
datasets, as depicted in Figure 10. The consistent convergence to more similar values clearly indicates that IGNNet provides transparent models with feature scores aligned with the true Shapley values.56

Footnote 5: The complete results of the 35 datasets are provided in the technical appendix.

Footnote 6: We provide illustrations of individual explanations in the technical appendix.

### Evaluation of Predictive Performance

Detailed results for IGNNet and the five competing algorithms on the 35 datasets are shown in Table 1. The ranking of the five algorithms across 35 datasets, based on their AUC values, reveals OGNNet to exhibit superior performance, claiming the top position, closely followed by IGNNet and XGBoost. In order to investigate whether the observed differences are statistically significant, the Friedman test [14] was employed, which indeed allowed to reject the null hypothesis, i.e., that there is no difference in predictive performance, as measured by the AUC, at the 0.05 level. The result of subsequently applying the post-hoc Nemenyi test [12] to determine what pairwise differences are significant, again at the 0.05 level, is summarized in Figure 4. However, the result shows no specific significant pairwise differences between any of the compared algorithms. Furthermore, the results show that using IGNNet instead of the black-box variant, OGNNet, does not significantly reduce the predictive performance while maintaining performance at the level of other powerful algorithms for tabular data, e.g., XGBoost and Random Forests.

## 5 Concluding Remarks

We have proposed IGNNet, an algorithm for tabular data classification, which exploits graph neural networks to produce transparent models. In contrast to post-hoc explanation techniques, IGNNet does not approximate or require costly computations, but provides the explanation while computing the prediction, and where the explanation prescribes exactly how the prediction is computed.

We have presented results from a large-scale empirical investigation, in which IGNNet was evaluated with respect to explainability and predictive performance. IGNNet was shown to generate explanations with feature scores aligned with the Shapley values without further computational cost. IGNNet was also shown to achieve a similar predictive performance as XGBoost, Random Forests, TabNet, and MLP, which are all well-known for their ability to generate high-performing models.

One direction for future research is to explore approaches to model feature interaction in the adjacency matrix that go beyond linear correlations. Understanding how such non-linear interactions between features may impact the model's interpretability could be an intriguing area of exploration. A second direction is to investigate alternative encoders for categorical features rather than relying on one-hot encoding. It would also be interesting to extend IGNNet to handle non-tabular datasets, including images and text, which would require entirely different approaches to representing each data point as a graph. Another important direction for future work is to use IGNNet for studying possible adversarial attacks on a predictive model. Finally, an important direction would be to complement the empirical evaluation with user-grounded evaluations, e.g., measuring to what extent certain tasks could be more effectively and efficiently solved when the users are provided with a transparent model that shows how the prediction has been computed from the input.

## 6 Acknowledgement

This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
