Editable Graph Neural Network for Node Classifications
Despite Graph Neural Networks (GNNs) have achieved prominent success in many
graph-based learning problem, such as credit risk assessment in financial
networks and fake news detection in social networks. However, the trained GNNs
still make errors and these errors may cause serious negative impact on
society. \textit{Model editing}, which corrects the model behavior on wrongly
predicted target samples while leaving model predictions unchanged on unrelated
samples, has garnered significant interest in the fields of computer vision and
natural language processing. However, model editing for graph neural networks
(GNNs) is rarely explored, despite GNNs' widespread applicability. To fill the
gap, we first observe that existing model editing methods significantly
deteriorate prediction accuracy (up to $50\%$ accuracy drop) in GNNs while a
slight accuracy drop in multi-layer perception (MLP). The rationale behind this
observation is that the node aggregation in GNNs will spread the editing effect
throughout the whole graph. This propagation pushes the node representation far
from its original one. Motivated by this observation, we propose
\underline{E}ditable \underline{G}raph \underline{N}eural \underline{N}etworks
(EGNN), a neighbor propagation-free approach to correct the model prediction on
misclassified nodes. Specifically, EGNN simply stitches an MLP to the
underlying GNNs, where the weights of GNNs are frozen during model editing. In
this way, EGNN disables the propagation during editing while still utilizing
the neighbor propagation scheme for node prediction to obtain satisfactory
results. Experiments demonstrate that EGNN outperforms existing baselines in
terms of effectiveness (correcting wrong predictions with lower accuracy drop),
generalizability (correcting wrong predictions for other similar nodes), and
efficiency (low training time and memory) on various graph datasets.
Zirui Liu, Zhimeng Jiang, Shaochen Zhong, Kaixiong Zhou, Li Li, Rui Chen, Soo-Hyun Choi, Xia Hu
# Editable Graph Neural Network for Node Classifications

###### Abstract

Despite Graph Neural Networks (GNNs) have achieved prominent success in many graph-based learning problem, such as credit risk assessment in financial networks and fake news detection in social networks. However, the trained GNNs still make errors and these errors may cause serious negative impact on society. _Model editing_, which corrects the model behavior on wrongly predicted target samples while leaving model predictions unchanged on unrelated samples, has garnered significant interest in the fields of computer vision and natural language processing. However, model editing for graph neural networks (GNNs) is rarely explored, despite GNNs' widespread applicability. To fill the gap, we first observe that existing model editing methods significantly deteriorate prediction accuracy (up to 50% accuracy drop) in GNNs while a slight accuracy drop in multi-layer perception (MLP). The rationale behind this observation is that the node aggregation in GNNs will spread the editing effect throughout the whole graph. This propagation pushes the node representation far from its original one. Motivated by this observation, we propose Editable Graph Neural Networks (EGNN), a neighbor propagation-free approach to correct the model prediction on misclassified nodes. Specifically, EGNN simply stitches an MLP to the underlying GNNs, where the weights of GNNs are frozen during model editing. In this way, EGNN disables the propagation during editing while still utilizing the neighbor propagation scheme for node prediction to obtain satisfactory results. Experiments demonstrate that EGNN outperforms existing baselines in terms of effectiveness (correcting wrong predictions with lower accuracy drop), generalizability (correcting wrong predictions for other similar nodes), and efficiency (low training time and memory) on various graph datasets.

## 1 Introduction

Graph Neural Networks (GNNs) have achieved prominent results in learning features and topology of graph data (Ying et al., 2018; Hamilton et al., 2017; Ling et al., 2023; Zeng et al., 2020; Hu et al., 2020; Zhou et al., 2021; Jiang et al., 2022; Han et al., 2022b, a; Ling et al., 2023; Duan et al., 2022; Zhou et al., 2021). Based on spatial message passing, GNNs learn each node through aggregating representations of its neighbors and the node itself recursively. Once trained, the model is typically deployed as static artifacts to make decisions on a wide range of tasks, such as credit risk assessment in financial networks (Petrone and Latora, 2018) and fake news detection in social networks (Shu et al.,
2017). However, the cost of making a wrong decision could be higher in these graph applications. Over-trusted creditworthiness on borrowers can lead to severe loss for lenders, and failure detection of fake news has a serious negative impact on society.

Ideally, it is desirable to correct these serious errors and generalize corrections to similar mistakes, while preserving the model's prediction accuracy on unrelated input samples. To obtain generalization ability for similar samples, the most prevalent method is to fine-tune the model with a new label on the single example to be corrected. However, this approach often spoils the model prediction on other unrelated samples. To cope with the challenge, many model editing frameworks have been proposed to adjust model behaviors by correcting errors as they appear (Sinitsin et al., 2020; Mitchell et al., 2021, 2022; De Cao et al., 2021). Specifically, these editors usually require an additional training phase to help the model "prepare" for the editing process before applying any edits (Sinitsin et al., 2020; Mitchell et al., 2021, 2022; De Cao et al., 2021).

Although model editing has shown promise to modify vision and language models, to the best of our knowledge, there is no existing work tackling the critical mistakes in graph data. Despite the straightforward concept, it is challenging to efficiently change GNNs' behaviors on the massively connected nodes. First, due to the message-passing mechanism in GNNs, editing the model behavior on a single node can propagate changes across the entire graph, significantly altering the node's original representation, which may destroy the prediction performance on the training dataset. Therefore, compared to the neural networks for computer vision or natural language processing, it is harder to maintain the model prediction on other input samples. Second, unlike other types of neural networks, the input nodes are connected in the graph domain. Thus, when editing the model prediction on a single node using gradient descent, the representation of each node in the whole graph is required (Liu et al., 2022; Han et al., 2023; Hamilton et al., 2017). This distinction introduces complexity and computational challenges when making targeted adjustments to GNNs, especially on large graphs.

In this work, we delve into studying the graph model editing problem, which is more challenging than the independent sample edits. We first observe the existing editors significantly harm the overall node classification accuracy although the misclassified nodes are corrected. The test accuracy drop is up to 50%, which prevents GNNs from being practically deployed. We experimentally study the rationale behind this observation from the lens of loss landscapes. Specifically, we visualize the loss landscape of the Kullback-Leibler (KL) divergence between node embeddings obtained before and after the model editing process in GNNs. We found that a slight weight perturbation can significantly enlarge the KL divergence. In contrast, other types of neural networks, such as Multi-Layer Perceptrons (MLPs), exhibit a much flatter region of the KL loss landscape and display greater robustness against weight variations. Such observations align with our viewpoint that after editing on misclassified samples, GNNs are prone to widely propagating the editing effect and affecting the remaining nodes.

Based on the sharp loss landscape of model editing in GNNs, we propose Editable Graph Neural Network (EGNN ), a neighbor propagation-free approach to correct the model prediction on the graph data. Specifically, suppose we have a well-trained GNN and we found want to correct its prediction on some of the misclassified nodes. EGNN stitches a randomly initialized MLP to the trained GNN. We then train the MLP for a few iterations to ensure that it does not significantly alter the model's prediction. When performing the edit, we only update the parameter of the stitched MLP while freezing the parameter of GNNs during the model editing process. In particular, the node embeddings from GNNs are first inferred offline. Then MLP learns an additional representation, which is then combined with the fixed embeddings inferred from GNNs to make the final prediction. When a misclassified node is received, the gradient is back propagated to update the parameters of MLP instead of GNNs'. In this way, we decouple the _neighbor propagation process_ of learning the
structure-aware node embeddings from the _model editing process_ of correcting the misclassified nodes. Thus, EGNN disables the propagation during editing while still utilizing the neighbor propagation scheme for node prediction to obtain satisfactory results. Compared to directly applying the existing model editing methods to GNNs:

* We can leverage the GNNs' structure learning meanwhile avoiding the spreading edition errors to guarantee the overall node classification task.
* The experimental results validate our solution which could address all the erroneous samples and deliver up to **90% improvement in overall accuracy**.
* Via freezing GNNs' part, EGNN is scalable to address misclassified nodes in the million-size graphs. We save more than \(2\times\) in terms of memory footprint and model editing time.

## 2 Preliminary

Graph Neural Networks.Let \(\mathcal{G}=(\mathcal{V},\mathcal{E})\) be an undirected graph with \(\mathcal{V}=(v_{1},\cdots,v_{|\mathcal{V}|})\) and \(\mathcal{E}=(e_{1},\cdots,e_{|\mathcal{E}|})\) being the set of nodes and edges, respectively. Let \(\mathbf{X}\in\mathbb{R}^{|\mathcal{V}|\times d}\) be the node feature matrix. \(\mathbf{A}\in\mathbb{R}^{|\mathcal{V}|\times|\mathcal{V}|}\) is the graph adjacency matrix, where \(\mathbf{A}_{i,j}=1\) if \((v_{i},v_{j})\in\mathcal{E}\) else \(\mathbf{A}_{i,j}=0\). \(\tilde{\mathbf{A}}=\tilde{\mathbf{D}}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I})\tilde{\mathbf{D}}^{- \frac{1}{2}}\) is the normalized adjacency matrix, where \(\tilde{\mathbf{D}}\) is the degree matrix of \(\mathbf{A}+\mathbf{I}\). In this work, we are mostly interested in the task of node classification, where each node \(v\in\mathcal{V}\) is associated with a label \(y_{v}\), and the goal is to learn a representation \(\mathbf{h}_{v}\) from which \(y_{v}\) can be easily predicted. To obtain such a representation, GNNs follow a neural message passing scheme (Kipf and Welling, 2017). Specifically, GNNs recursively update the representation of a node by aggregating representations of its neighbors. For example, the \(l^{\text{th}}\) Graph Convolutional Network (GCN) layer (Kipf and Welling, 2017) can be defined as:

\[\mathbf{H}^{(l+1)}=\text{ReLU}(\tilde{\mathbf{A}}\mathbf{H}^{(l)}\mathbf{\Theta}^{(l)}), \tag{1}\]

where \(\mathbf{H}^{(l)}\) is the node embedding matrix containing the \(\mathbf{h}_{v}\) for each node \(v\) at the \(l^{\text{th}}\) layer and \(\mathbf{H}^{(0)}=\mathbf{X}\). \(\mathbf{\Theta}^{(l)}\) is the weight matrix of the \(l^{\text{th}}\) layer.

The Model Editing Problem.The goal of model editing is to alter a base model's output for a misclassified sample \(x_{e}\) as well as its similar samples via model finetuning only using a single pair of input \(x_{e}\) and desired output \(y_{e}\) while leaving model behavior on unrelated inputs intact (Sinitsin et al., 2020; Mitchell et al., 2021, 2022). We are the first to propose the model editing problem in graph data, where the decision faults on a small number of critical nodes can lead to significant financial loss and/or fairness concerns. For the node classification, suppose a well-trained GNN incorrectly predicts a specific node. **Model editing** is used to correct the undesirable prediction behavior for that node by using the node's features and desired label to update the model. Ideally, the model editing ensures that the updated model makes accurate predictions for the specific node and its similar samples while maintaining the model's original behavior for the remaining unrelated inputs. Some model editors, such as the one presented in this paper, require a training phase before they can be used for editing.

## 3 Proposed Methods

In this section, we first empirically show vanilla model editing performs extremely worse for GNNs compared with MLPs due to node propagation (Section 3.1). Intuitively, due to the message-passing
mechanism in GNNs, editing the model behavior on a single node can propagate changes across the entire graph, significantly altering the node's original representation. Then through visualizing the loss landscape, we found that for GNNs, even a slight weight perturbation, the node representation will be far away from the original one (Section 3.2). Based on the observation, we propose a propagation-free GNN editing method called EGNN (Section 3.3).

### Motivation: Model Editing may Cry in GNNs

**Setting:** We train GCN, GraphSAGE, and MLP on Cora, Flickr, Reddit, and ogbn-arxiv, respectively, following the training setup as described in Section 5. To evaluate the difficulty of editing, _we ensured that the node to be edited was not present during training_, meaning that _the models were trained inductively._ Specifically, we trained the model on a subgraph containing only the training node and evaluated its performance on the validation and test set of nodes. Next, we selected a misclassified node from the validation set and applied gradient descent only on that node until the model made a correct prediction for it. Following previous work (Sinitsin et al., 2020; Mitchell et al., 2022), we perform 50 independent edits and report the averaged test accuracy before and after performing a single edit.

**Results:** As shown in Table 1, we observe that **(1)** GNNs consistently outperform MLP on all the graph datasets before editing. This is consistent with the previous graph analysis results, where the neural message passing involved in GNNs extracts the graph topology to benefit the node representation learning and thereby the classification accuracy. **(2)** After editing, the accuracy drop of GNNs is significantly larger than that of MLP. For example, GraphSAGE has an almost 50% drop in test accuracy on ogbn-arxiv after editing even a single point. MLP with editing even delivers higher overall accuracies on Flickr and ogbn-arxiv compared with GNN-based approaches. One of the intuitive explanations is the slightly fine-tuned weights in MLP mainly affect the target node, instead of other unrelated samples. However, due to the message-passing mechanism in GNNs, the edited node representation can be propagated over the whole graph and thus change the decisions on a large area of nodes. These comparison results reveal the unique challenge in editing the correlated nodes with GNNs, compared with the conventional neural networks working on isolated samples. **(3)** After editing, the test accuracy of GCN, GraphSAGE, and MLP become too low to be practically deployed. This is quite different to the model editing problems in computer vision and natural

\begin{table}
\begin{tabular}{c c c c c} \hline \hline  & & GCN & GraphSAGE & MLP \\ \hline \multirow{3}{*}{Cora} & w./o. edit & **89.4** & 86.6 & 71.8 \\  & w./ edit & **84.36** & 82.06 & 68.33 \\  & \(\Delta\) Acc. & 5.03\(\downarrow\) & 4.53\(\downarrow\) & **3.46**\(\downarrow\) \\ \hline \multirow{3}{*}{Flickr} & w./o. edit & **51.19** & 49.03 & 46.77 \\  & w./ edit & 13.94 & 17.15 & **36.68** \\  & \(\Delta\) Acc. & 37.25\(\downarrow\) & 31.88\(\downarrow\) & **10.08**\(\downarrow\) \\ \hline \multirow{3}{*}{Reddit} & w./o. edit & 95.52 & **96.55** & 72.41 \\  & w./ edit & **75.20** & 55.85 & 69.86 \\  & \(\Delta\) Acc. & 20.32\(\downarrow\) & 40.70\(\downarrow\) & **2.54**\(\downarrow\) \\ \hline \multirow{3}{*}{ogbn-arxiv} & w./o. edit & **70.20** & 68.38 & 52.65 \\  & w./ edit & 23.70 & 19.06 & **45.15** \\ \cline{1-1}  & \(\Delta\) Acc. & 46.49\(\downarrow\) & 49.31\(\downarrow\) & **7.52\(\downarrow\)** \\ \hline \hline \end{tabular}
\end{table}
Table 1: The test accuracy (%) before (“w./o. edit”) and after editing (“w./ edit”) on one single data point. \(\Delta\) Acc is the accuracy drop before and after performing the edit. All results are averaged over 50 simultaneous model edits. The best result is highlighted by **bold faces.**
language processing, where the modified models only suffer an acceptable accuracy drop.

### Sharp Locality of GNNs through Loss Landscape

Intuitively, due to the message-passing mechanism in GNNs, editing the model behavior for a single node can cause the editing effect to propagate across the entire graph. This propagation pushes the node representation far from its original one. **Thus, we hypothesized that the difficulty in editing GNNs as being due to the neighbor propagation of GNNs.** The model editing aims to correct the prediction of the misclassified node using the cross-entropy loss of desired label. Intuitively, the large accuracy drop can be interpreted as the low model prediction similarity before and after model editing, named as the locality.

To quantitatively measure the locality, we use the metric of KL divergence between the node representations learned before and after model editing. The higher KL divergence means after editing, the node representation is far away from the original one. In other words, the higher KL divergence implies poor model locality, which is undesirable in the context of model editing. Particularly, we visualize the locality loss landscape for Cora dataset in Figure 1. We observe several **insights:** (1) GNNs (e.g., GCN and GraphSAGE) suffer from a much sharper loss landscape. Even slightly editing the weights, KL divergence loss is dramatically enhanced. That means GNNs are hard to be fine-tuned while keeping the locality. (2) MLP shows a flatter loss landscape and demonstrates much better locality to preserve overall node representations. This is consistent to the accuracy analysis in Table 1, where the accuracy drop of MLP is smaller.

To deeply understand why model editing fails to work in GNNs, we also provide a pilot theoretical analysis on the KL locality difference between before/after model editing for one-layer GCN and MLP in Appendix D. We theoretically show that when model editing corrects the model predictions on misclassified nodes, GNNs are susceptible to altering the predictions on other connected nodes. This phenomenon results in an increased KL divergence difference.

### Egnn Neighbor Propagation Free GNN Editing

In our previous analysis, we hypothesized that the difficulty in editing GNNs as being due to the neighbor propagation. However, as Table 1 suggested, the neighbor propagation is necessary for obtaining good performance on graph datasets. On the other hand, MLP could stabilize most of the node representations during model editing although it has worse node classification capability. Thus, we need to find a way to "disable" the propagation during editing while still utilizing the neighbor propagation scheme for node prediction to obtain satisfactory results. Following the motivation, we propose to combine a compact MLP to the well-trained GNN and only modify the MLP during editing. In this way, we can correct the model's predictions through this additional MLP while freezing the neighbor propagation. Meanwhile during inference, both the GNN and MLP are used together for prediction in tandem to harness the full potential of GNNs for prediction. The whole algorithm is shown in Algorithm 1.

Figure 1: The loss landscape of various model architectures on Cora dataset. Similar results can be found in Appendix C
```
procedureMLP training procedure: Input: MLP \(g_{\Phi}\), dataset \(\mathcal{D}\), the node embedding \(\mathbf{h}_{v}\) for each node \(v\) in \(\mathcal{D}\) for\(t=1,\cdots,T\)do  Sample \(\mathbf{x}_{v}\), \(y_{v}\sim\mathcal{D}^{\text{train}}\) \(\mathcal{L}_{\text{loc}}=\text{KL}(\mathbf{h}_{v}+g_{\Phi}(\mathbf{x}_{v})||\mathbf{h}_{v})\) \(\mathcal{L}_{\text{task}}=-\log p_{\Phi}(y_{v}|\mathbf{h}_{v}+g_{\Phi}(\mathbf{x}_{v}))\) \(\mathcal{L}=\mathcal{L}_{\text{task}}\,+\,\alpha\mathcal{L}_{\text{loc}}\) \(\mathbf{\Phi}\leftarrow Adam(\mathbf{\Phi},\nabla\mathcal{L})\)  end for  end for procedureEGNN EDIT PROCEDUTE: Input: data pair \(x_{e},y_{e}\) to be edited, the node embedding \(\mathbf{h}_{e}\) for node \(e\) \(\hat{y}=\arg\max_{y}p_{\Phi}(y|\mathbf{x}_{e},\mathbf{h}_{e})\) while\(\hat{y}\neq y_{v}\)do \(\mathcal{L}=-\log p_{\Phi}(y|\mathbf{x}_{e},\mathbf{h}_{e})\) \(\mathbf{\Phi}\gets Adam(\mathbf{\Phi},\nabla\mathcal{L})\)  end for  end for
```

**Algorithm 1**Proposed EGNN

**Before editing.** We first stitch a randomly initialized compact MLP to the trained GNN. To mitigate the potential impact of random initialization on the model's prediction, we introduce a training procedure for the stitched MLP, as outlined in Algorithm 1 "MLP training procedure": we train the MLP for a few iterations to ensure that it does not significantly alter the model's prediction. By freezing GNN's weights, we first get the node embedding \(\mathbf{h}_{v}\) at the last layer of the trained GNN by running a single forward pass. We then stitch the MLP with the trained GNNs. Mathematically, we denote the MLP as \(g_{\Phi}\) where \(\Phi\) is the parameters of MLP. For a given input sample \(\mathbf{x}_{v},\mathbf{y}_{v}\), the model output now becomes \(\mathbf{h}_{v}+g_{\Phi}(\mathbf{x}_{v})\). We calculate two loss based on the

Figure 2: The overview of EGNN. We fix the backbone of GNNs (in blue), while only update the small MLPs (in orange) during editing. The wrongly predicted nodes are highlighted with orange color, and the MLP only require its node feature.
prediction, i.e., the task-specific loss \(\mathcal{L}_{\text{task}}\) and the locality loss \(\mathcal{L}_{\text{loc}}\). Namely,

\[\mathcal{L}_{\text{task}} =-\log p_{\boldsymbol{\Phi}}(y_{v}|\boldsymbol{h}_{v}+g_{ \boldsymbol{\Phi}}(\boldsymbol{x}_{v})),\] \[\mathcal{L}_{\text{loc}} =\text{KL}(\boldsymbol{h}_{v}+g_{\boldsymbol{\Phi}}(\boldsymbol{x }_{v})||\boldsymbol{h}_{v}),\]

where \(\boldsymbol{h}_{v}+g_{\boldsymbol{\Phi}}(\boldsymbol{x}_{v})\) is the model prediction with the additional MLP and \(p_{\boldsymbol{\Phi}}(y_{v}|\boldsymbol{h}_{v}+g_{\boldsymbol{\Phi}}( \boldsymbol{x}_{v}))\) is the probability of class \(y_{v}\) given by the model. \(\mathcal{L}_{\text{task}}\) is the cross-entropy between the model prediction and label. \(\mathcal{L}_{\text{loc}}\) is the locality loss, which equals KL divergence between the original prediction \(\boldsymbol{h}_{v}\) and the prediction with the additional MLP \(\boldsymbol{h}_{v}+g_{\boldsymbol{\Phi}}(\boldsymbol{x}_{v})\). The final loss \(\mathcal{L}\) is the weighted combination of two parts, i.e., \(\mathcal{L}=\mathcal{L}_{\text{task}}\) + \(\alpha\mathcal{L}_{\text{loc}}\) where \(\alpha\) is the weight for the locality loss. \(\mathcal{L}\) is used to guide the MLP to fit the task while keep the model prediction unchanged.

**When editing.**EGNN **freezes the model parameters of GNN and only updates the parameters of MLP.** Specifically, as outlined in Algorithm 1 "EGNN Edit Procedure", we update the parameters of MLP until the model prediction for the misclassified sample is corrected. Since MLP only relies on the node features, we can easily perform these updates in mini-batches, which enables us to edit GNNs on large graphs.

Lastly, we visualize the KL locality loss landscape of EGNN (including GCN-MLP and SAGE-MLP) in Figure 1. It is seen that the proposed EGNN shows the most flattened loss landscape than MLP and GNNs, which implied that EGNN can preserve overall node representations better than other model architectures.

## 4 Related Work and Discussion

Due to the page limit, below we discuss the related work on model editing. We also discuss the limitation in Appendix B.

Model Editing.Many approaches have been proposed for model editing. The most straightforward method adopts standard fine-tuning to update model parameters based on misclassified samples while preserving model locality via constraining parameters travel distance in model weight space (Zhu et al., 2020; Sotoudeh and Thakur, 2019). Work (Sinitsin et al., 2020) introduces meta-learning to find a pre-trained model with rapid and easy finetuned ability for model editing. Another way to facilitate model editing relies on external learned editors to modify model editing considering several constraints (Mitchell et al., 2021; Hase et al., 2021; De Cao et al., 2021; Mitchell et al., 2022). The editing of the activation map is proposed to correct misclassified samples in (Dai et al., 2021; Meng et al., 2022) due to the belief of knowledge attributed to model neurons. While all these works either update base model parameters or import external separate modules for model prediction to induce desired prediction change, the considered data is i.i.d. and may not work well in graph data due to essential node interaction during neighborhood propagation. In this paper, we propose EGNN, using a stitched MLP module to edit the output space of the base GNN model, for node classification tasks. The key insight behind this solution is the sharp locality of GNNs, i.e., the prediction of GNNs can be easily altered after model editing.

## 5 Experiments

The experiments are designed to answer the following research questions. **RQ1:** Can EGNN correct the wrong model prediction? Moreover, what is the difference in accuracy before and after editing using EGNN? **RQ2:** Can the edits generalize to correct the model prediction on other similar inputs? **RQ3:** What is the time and memory requirement of EGNN to perform the edits?
### Experimental Setup

Datasets and Models.To evaluate EGNN, we adopt four small-scale and four large-scale graph benchmarks from different domains. For small-scale datasets, we adopt Cora, A-computers (Shchur et al., 2018), A-photo (Shchur et al., 2018), and Coauthor-CS (Shchur et al., 2018). For large-scale datasets, we adopt Reddit (Hamilton et al., 2017), Flickr (Zeng et al., 2020), _ogbn-arxiv_ (Hu et al., 2020), and _ogbn-products_ (Hu et al., 2020). We integrate EGNN with two popular models: GCN (Kipf and Welling, 2017) and GraphSAGE (Hamilton et al., 2017). _To avoid creating confusion, GCN and GraphSAGE are all trained with the whole graph at each step._ We evaluate EGNN under the **inductive setting**. Namely, we trained the model on a subgraph containing only the training node and evaluated it on the whole graph. Details about the hyperparameters and datasets are in Appendix A.

Compared Methods.We compare our EGNN editor with the following two baselines: the vanilla gradient descent editor (GD) and Editable Neural Network editor (ENN) (Sinitsin et al., 2020). GD is the same editor we used in our preliminary analysis in Section 3. **We note that for other model editing, e.g., MEND (Mitchell et al., 2021), SERAC (Mitchell et al., 2022) are tailored for NLP applications, which cannot be directly applied to the graph area**. Specifically, GD applies the gradient descent **on the parameters of GNN** until the GNN makes right prediction. ENN trains **the parameters of GNN** for a few steps to make it prepare for the following edits. Then similar to GD editor, it applies the gradient descent **on the parameters of GNN** until the GNN makes right prediction. For EGNN, we only train **the stitched MLP** for a few steps. Then we only update **weights of MLP** during edits. Detailed hyperparameters are listed in Appendix A.

Evaluation Metrics.Following previous work (Sinitsin et al., 2020; Mitchell et al., 2022, 2021), we evaluate the effectiveness of different methods by the following three metrics. **DrawDown (DD)**, which is the mean absolute difference of test accuracy before and after performing an edit. A smaller drawdown indicates a better editor locality. **Success Rate (SR)**, which is defined as the rate of edits, where the editor successfully corrects the model prediction. **Edit Time**, which is defined as the wall-clock time of a single edit that corrects the model prediction.

### The Effectiveness of EGNN Editing GNNs

In many real-world applications, it is common to encounter situations where our trained model produces incorrect predictions on unseen data. It is crucial to address these errors as soon as they are identified. To assess the usage of editors in real-world applications (**RQ1**), **we select misclassified nodes from the validation set, which is not seen during the training process.** Then we

\begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & \multirow{2}{*}{Editor} & \multicolumn{3}{c}{Cora} & \multicolumn{3}{c}{A-computers} & \multicolumn{3}{c}{A-photo} & \multicolumn{3}{c}{Count-CS} \\ \cline{3-13}  & & Acc\({}^{\dagger}\) & DD\({}_{\ddagger}\) & SR\({}^{\dagger}\) & Acc\({}^{\dagger}\) & DD\({}_{\ddagger}\) & SR\({}^{\dagger}\) & Acc\({}^{\dagger}\) & DD\({}_{\ddagger}\) & SR\({}^{\dagger}\) & Acc\({}^{\dagger}\) & DD\({}_{\ddagger}\) & SR\({}^{\dagger}\) \\ \hline \multirow{3}{*}{GCN} & GD & 84.37\(\pm\)5.84 & 5.03\(\pm\)6.40 & 1.0 & 44.78\(\pm\)22.41 & 43.09\(\pm\)22.32 & 1.0 & 28.70\(\pm\)21.26 & 65.08\(\pm\)20.13 & 1.0 & 91.07\(\pm\)3.23 & 3.30\(\pm\)2.22 & 1.0 \\  & ENN & 37.16\(\pm\)3.80 & 52.24\(\pm\)4.76 & 1.0 & 55.11\(\pm\)10.99 & 72.36\(\pm\)10.87 & 1.0 & 16.71\(\pm\)14.81 & 77.07\(\pm\)15.20 & 1.0 & 4.94\(\pm\)3.78 & 89.43\(\pm\)3.34 & 1.0 \\ \cline{2-13}  & GNN & **87.80\(\pm\)**2.34 & **1.80\(\pm\)**2.13 & **1.0** & **82.85\(\pm\)**5.20 & **2.32\(\pm\)**5.11 & 0.98** & **91.97\(\pm\)**5.55 & **2.39\(\pm\)**5.34 & **1.0** & **94.54\(\pm\)**0.07 & **-0.17\(\pm\)**0.07 & **1.0** \\ \cline{2-13}  & GD & 82.06\(\pm\)4.33 & 4.54\(\pm\)5.23 & 1.0 & 21.68\(\pm\)20.98 & 61.15\(\pm\)20.33 & 1.0 & 38.98\(\pm\)30.24 & 55.32\(\pm\)29.35 & 1.0 & 91.55\(\pm\)5.55 & 50.15\(\pm\)5.32 & 1.0 \\ \cline{2-13}  & SAGE & ENN & 31.61\(\pm\)4.5 & 53.44\(\pm\)2.23 & 1.0 & 16.89\(\pm\)16.98 & 65.94\(\pm\)16.75 & 1.0 & 15.06\(\pm\)11.92 & 79.24\(\pm\)11.25 & 1.0 & 13.71\(\pm\)2.73 & 81.45\(\pm\)2.11 & 1.0 \\ \cline{2-13}  & GNN & **85.65\(\pm\)**1.23 & **0.55\(\pm\)**1.26 & **1.0** & **84.34\(\pm\)**4.84 & **2.72\(\pm\)**5.03 & 0.94 & **92.53\(\pm\)**2.90 & **1.83\(\pm\)**3.22 & **1.0** & **95.27\(\pm\)**10.08 & **-0.01\(\pm\)**0.10 & **1.0** \\ \hline \hline \end{tabular}
\end{table}
Table 2: The results on four small scale datasets after applying one single edit. The reported number is averaged over 50 independent edits. **SR** is the edit success rate, **Acc** is the test accuracy after editing, and **DD** are the test drawdown, respectively. “OOM” is the out-of-memory error.
employ the editor to correct the model's predictions for those misclassified nodes, and measure the drawdown and edit success rate on the test set.

The results after editing on a single node are shown in Table 2 and Table 3. We observe that

_Unlike editing Transformers on text data (Mitchell et al., 2021, 2022; Huang et al., 2023), all editors can successfully correct the model prediction in graph domain_. As shown in Table 3, all editors have 100% success rate when edit GNNs. In contrast, for transformers, the edit success rate is often less than 50% and drawdown is much smaller than GNNs (Mitchell et al., 2021, 2022; Huang et al., 2023). This observation suggests that **unlike transformers, GNNs can be easily perturbed to produce correct predictions. However, at the cost of huge drawdown on other unrelated nodes. Thus, the main challenge lies in maintaining the locality between predictions for unrelated nodes before and after editing.** This observation aligns with our initial analysis, which highlighted the interconnected nature of nodes and the edit on a single node may propagate throughout the entire graph.

_EGNN significantly outperforms both GD and ENN in terms of the test drawdown_. This is mainly because both GD and ENN try to correct the model's predictions by updating the parameters of Graph Neural Networks (GNNs). This process inevitably relies on neighbor propagation. In contrast, EGNN has much better test accuracy after editing. Notably, for Reddit, the accuracy drop decreases from roughly 80% to \(\approx 1\%\), which is significantly better than the baseline. This is because EGNN decouples the neighbor propagation with the editing process. Interestingly, ENN is significantly worse than the vanilla editor, i.e., GD, when applied to GNNs. As shown in Appendix C, we found that this discrepancy arises from the ENN training procedure, which significantly compromises the model's performance to prepare it for editing.

In Figure 3, 4, and 5 we present the ablation study under the sequential setting. This is a more challenging scenario where the model is edited sequentially as errors arise. In particular, we plot the test accuracy drawdown against the number of sequential edits for GraphSAGE on the ogbn-arxiv dataset. We observe that

_EGNN consistently surpasses both GD and ENN in the sequential setting_. However, the drawdown is considerably greater than that in the single edit setting. For instance, EGNN exhibits a 0.64% drawdown for GraphSAGE on the ogbn-arxiv dataset in the single edit setting, which escalates up to a 20% drawdown in the sequential edit setting. These results also highlight the hardness of maintaining the locality of GNN prediction after editing.

\begin{table}
\begin{tabular}{c c c c c c c c c c c c c} \hline \hline  & \multirow{2}{*}{Editor} & \multicolumn{3}{c}{Flickr} & \multicolumn{3}{c}{Reddit} & \multicolumn{3}{c}{ogbn-} & \multicolumn{3}{c}{ogbn-} \\  & & Acc! & DD\(\downarrow\) & SR\(\uparrow\) & Acc! & DD\(\downarrow\) & SR\(\uparrow\) & Acc! & DD\(\downarrow\) & SR\(\uparrow\) & Acc! & DD\(\downarrow\) & SR\(\uparrow\) \\ \hline \multirow{3}{*}{GCN} & GD & 13.95\(\pm\)11.0 & 37.25\(\pm\)10.2 & 1.0 & 75.20\(\pm\)12.3 & 20.32\(\pm\)11.3 & 1.0 & 23.71\(\pm\)16.9 & 46.50\(\pm\)14.9 & 1.0 & OOM & OOM & 0 \\  & ENN & 25.82\(\pm\)14.9 & 25.38\(\pm\)16.9 & 1.0 & 11.65\(\pm\)1.5 & 84.36\(\pm\)1.1 & 1.0 & 16.39\(\pm\)7.7 & 53.62\(\pm\)6.7 & 1.0 & OOM & OOM & 0 \\  & ENN & **44.91\(\pm\)**12.2 & **6.34\(\pm\)**10.3 & **1.0** & **94.46\(\pm\)**10.4 & 1.03\(\pm\)0.6 & **1.0** & **67.34\(\pm\)**8.7 & **2.67\(\pm\)**4.4 & **1.0** & **74.19\(\pm\)**3.4 & **0.81\(\pm\)**0.23 & **1.0** \\ \hline \multirow{3}{*}{Graph-SAGE} & GD & 17.16\(\pm\)12.2 & 31.88\(\pm\)12.2 & 1.0 & 55.85\(\pm\)22.5 & 40.71\(\pm\)20.3 & 1.0 & 19.07\(\pm\)14.1 & 36.68\(\pm\)10.1 & 1.0 & OOM & OOM & 0 \\  & ENN & 28.73\(\pm\)5.6 & 20.31\(\pm\)5.6 & 1.0 & 5.88\(\pm\)3.9 & 90.68\(\pm\)4.3 & 1.0 & 8.14\(\pm\)8.6 & 47.61\(\pm\)7.6 & 1.0 & OOM & OOM & 0 \\  & ENN & **43.52\(\pm\)**10.8 & **5.12\(\pm\)**10.8 & **1.0** & **96.50\(\pm\)**0.1 & **0.05\(\pm\)**0.1 & **1.0** & **67.91\(\pm\)**2.9 & **0.64\(\pm\)**2.3 & **1.0** & **76.27\(\pm\)**0.6 & **0.17\(\pm\)**0.10 & **1.0** \\ \hline \hline \end{tabular}
\end{table}
Table 3: The results on four large scale datasets after applying one single edit. “OOM” is the out-of-memory error.

Figure 3: Sequential edit drawdown of GCN on four small scale datasets.
### The Generalization of the Edits of Egnn

Ideally, we aim for the edit applied to a specific node to generalize to similar nodes while preserving the model's initial behavior for unrelated nodes. To evaluate the generalization of the Egnn edits, we conduct the following experiment:

**(1)** We first select a particular group (i.e., class) of nodes based on their labels.

**(2)** Next, we randomly flip the labels of 10% of the training nodes within this group and train a GNN on the modified training set.

**(3)** For each flipped training node, we correct the trained model's prediction for that node back to its original class and assess whether the model's predictions for other nodes in the same group are also corrected. If the model's predictions for other nodes in the same class are also corrected after modifying a single flipped node, it indicates that the Egnn edits

Figure 4: Sequential edit drawdown of GraphSAGE on four small scale datasets.

Figure 5: Sequential edit test drawdown of GCN and GraphSAGE on Reddit and ogbn-arxiv dataset.

Figure 6: The subgroup and overall test accuracy before and after one single edit. The results are averaged over 50 independent edits.

Figure 7: T-SNE visualizations of GNN embeddings before and after edits on the Cora dataset. The flipped nodes are all from class 0, which is marked in red color.
can effectively generalize to address similar erroneous behavior in the model.

To answer **RQ2**, we conduct the above experiments and report the **subgroup and overall test accuracy** after performing a single edit on the flipped training node. The results are shown in Figure 6. We observe that: _From Figure 5(a) and Figure 5(c), EGNN significantly improves the subgroup accuracy after performing even a single edit_. Notably, the subgroup accuracy is significantly lower than the overall accuracy. For example, on Flickr dataset, both GCN and GraphSAGE have a subgroup accuracy of less than \(5\%\) before editing. This is mainly because the GNN is trained on the graph where \(10\%\) labels of the training node in the subgroup are flipped. However, even after editing on a single node, the subgroup accuracy is significantly boosted. These results indicate that the EGNN edits can effectively generalize to address the wrong prediction on other nodes in the same group. In Figure 7, we also visualize the node embeddings before and after editing by EGNN on the Cora dataset. We note that all of the flipped nodes are from class 0, which is marked in red color in Figure 7. Before editing, the red cluster has many outliers that lie in the embedding space of other classes. This is mainly because the labels of some of the nodes in this class are flipped. In contrast, after editing, the nodes in the red cluster become significantly closer to each other, with a substantial reduction in the number of outliers.

### The Efficiency of Egnn

We want to patch the model as soon as possible to correct errors as they appear. Thus ideally, the editor should be efficient and scalable to large graphs. Here we summarize the edit time and memory required for performing the edits in Table 4. We observe that EGNN is about \(2\sim 5\times\) faster than the GD editor in terms of the wall-clock edit time. This is because EGNN only updates the parameters of MLP, and totally gets rid of the expensive graph-based sparse operations (Liu et al., 2022b, a; Han et al., 2023b). Also, updating the parameters of GNNs requires storing the node embeddings in memory, which is directly proportional to the number of nodes in the graph and can be exceedingly expensive for large graphs However, with EGNN, we only use node features for updating MLPs, meaning that memory consumption is not dependent on the graph size. Consequently, EGNN can efficiently scale up to handle graphs with millions of nodes, e.g., ogbn-products, whereas the vanilla editor raises an OOM error.

## 6 Conclusion

In this paper, we explore a and important problem, i.g., GNNs model editing for node classification. We first empirically observe that the vanilla model editing method may not perform well due to node aggregation, and then theoretically investigate the underlying reason through the lens of locality loss landscape with quantitative analysis. Furthermore, we propose EGNN to correct misclassified samples while preserving other intact nodes, via stitching a trainable MLP. In this way, the power of GNNs for prediction and the editing-friendly MLP can be integrated together in EGNN.

\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & \multirow{2}{*}{Editor} & \multicolumn{3}{c}{Flickr} & \multicolumn{2}{c}{Reddit} & \multicolumn{2}{c}{ogbn-} & \multicolumn{2}{c}{ogbn-} \\  & & Edit & Peak & Edit & Peak & Edit & Peak & Edit & Peak \\  & & Time (ms) & Memory (MB) & Time (ms) & Memory (MB) & Time (ms) & Memory (MB) & Time (ms) & Memory (MB) \\ \hline \multirow{2}{*}{GCN} & GD & 379.86 & 707 & 1835.24 & 3429 & 663.17 & 967 & OOM & OOM \\  & EGNN & 246.63 & 315 & 765.15 & 2089 & 299.71 & 248 & 5122.53 & 5747 \\ \hline Graph- & GD & 712.07 & 986 & 4781.92 & 5057 & 668.77 & 1109 & OOM & OOM \\ SAGE & EGNN & 389.37 & 328 & 1516.68 & 2252 & 174.82 & 260 & 5889.59 & 6223 \\ \hline \hline \end{tabular}
\end{table}
Table 4: The edit time and memory required for editing.