Graph Neural Networks (GNNs) are a powerful representational tool for solving problems on graph-structured inputs. In almost all cases so far, however, they have been applied to directly recovering a final solution from raw inputs, without explicit guidance on how to structure their problem-solving. Here, instead, we focus on learning in the space of algorithms: we train several state-of-the-art GNN architectures to imitate individual steps of classical graph algorithms, parallel (breadth-first search, Bellman-Ford) as well as sequential (Prim's algorithm). As graph algorithms usually rely on making discrete decisions within neighbourhoods, we hypothesise that maximisation-based message passing neural networks are best-suited for such objectives, and validate this claim empirically. We also demonstrate how learning in the space of algorithms can yield new opportunities for positive transfer between tasks---showing how learning a shortest-path algorithm can be substantially improved when simultaneously learning a reachability algorithm. A multitude of important real-world tasks can be formulated as tasks over graph-structured inputs, such as navigation, web search, protein folding, and game-playing. Theoretical computer science has successfully discovered effective and highly influential algorithms for many of these tasks. But many problems are still considered intractable from this perspective. Machine learning approaches have been applied to many of these classic tasks, from tasks with known polynomial time algorithms such as shortest paths (Graves et al., 2016; Xu et al., 2019) and sorting (Reed & De Freitas, 2015) , to intractable tasks such as travelling salesman (Vinyals et al., 2015; Bello et al., 2016; Kool et al., 2018) , boolean satisfiability (Selsam et al., 2018; Selsam & Bjørner, 2019) , and even probabilistic inference (Yoon et al., 2018) . Recently, this work often relies on advancements in graph representation learning (Bronstein et al., 2017; Hamilton et al., 2017; with graph neural networks (GNNs) (Li et al., 2015; Kipf & Welling, 2016; Gilmer et al., 2017; Veličković et al., 2018) . In almost all cases so far, ground-truth solutions are used to drive learning, giving the model complete freedom to find a mapping from raw inputs to such solution 1 . Many classical algorithms share related subroutines: for example, shortest path computation (via the Bellman-Ford (Bellman, 1958 ) algorithm) and breadth-first search both must enumerate sets of edges adjacent to a particular node. Inspired by previous work on the more general tasks of program synthesis and learning to execute (Zaremba & Sutskever, 2014; Kaiser & Sutskever, 2015; Kurach et al., 2015; Reed & De Freitas, 2015; , we show that by learning several algorithms simultaneously and providing a supervision signal, our neural network is able to demonstrate positive knowledge transfer between learning different algorithms. The supervision signal is driven by how a known classical algorithm would process such inputs (including any relevant intermediate outputs), providing explicit (and reusable) guidance on how to tackle graph-structured problems. We call this approach neural graph algorithm execution. Given that the majority of popular algorithms requires making discrete decisions over neighbourhoods (e.g. "which edge should be taken?"), we suggest that a highly suitable architecture for this task is a message-passing neural network (Gilmer et al., 2017 ) with a maximisation aggregator-a claim we verify, demonstrating clear performance benefits for simultaneously learning breadth-first search for reachability with the Bellman-Ford algorithm for shortest paths. We also verify its applicability to sequential reasoning, through learning Prim's algorithm (Prim, 1957) for minimum spanning trees. Note that our approach complements Reed & De Freitas (2015) : we show that a relatively simple graph neural network architecture is able to learn and algorithmically transfer among different tasks, do not require explicitly denoting subroutines, and tackle tasks with superlinear time complexity. 2 PROBLEM SETUP Parallel algorithm execution In order to evaluate how faithfully the neural algorithm executor replicates the two parallel algorithms, we propose reporting the accuracy of predicting the reachability 2 (for breadth-first search; Table 1 ), as well as predicting the predecessor node (for Bellman-Ford; Table 2 ). We report this metric averaged across all steps t (to give a sense of how well the algorithm is imitated across time), as well as the last-step performance (which corresponds to the final solution). While it is not necessary for recovering the final answer, we also provide the mean squared error of the models on the Bellman-Ford distance information, as well as the termination accuracy (computed at each step separately)-averaged across all timesteps-in Table 3 . The results confirm our hypotheses: the MPNN-max model exhibits superior generalisation performance on both reachability and shortest-path predecessor node prediction. Even when allowing for hardening the attention of GAT-like models (using entropy or Gumbel softmax), the more flexible computational model of MPNN is capable of outperforming them. The performance gap on predicting the predecessor also widens significantly as the test graph size increases. Our findings are compounded by observing the mean squared error metric on the intermediate result: with the MPNN-max being the only model providing a reasonable level of regression error at the 100-node generalisation level. It further accentuates that, even though models like the MPNN-sum model may also learn various thresholding functions-as demonstrated by (Xu et al., 2018 )-aggregating messages in this way can lead to outputs of exploding magnitude, rendering the network hard to numerically control for larger graphs. We perform two additional studies, executing the shortest-path prediction on MPNN-max without predicting reachability, and without supervising on any intermediate algorithm computations-that is, learning to predict predecessors (and termination behaviour) directly from the inputs, x i . Note that this is the primary way such tasks have been tackled by graph neural networks in prior work. We report these results as no-reach and no-algo in Table 2 , respectively. Looking at the no-reach ablation, we observe clear signs of positive knowledge transfer occurring between the reachability and shortest-path tasks: when the shortest path algorithm is learned in isolation, the predictive power of MPNN-max drops significantly (while still outperforming many other approaches). In Appendix B, we provide a brief theoretical insight to justify this. Similarly, considering the no-algo experiment, we conclude that there is a clear benefit to supervising on the distance information-giving an additional performance improvement compared to the standard approach of only supervising on the final downstream outputs. Taken in conjunction, these two results provide encouragement for studying this particular learning setup. Lastly, we report the performance of a curriculum learning (Bengio et al., 2009 ) strategy (as curriculum): here, BFS is learnt first in isolation (to perfect validation accuracy), followed by finetuning on Bellman-Ford. We find that this approach performs worse than learning both algorithms simultaneously, and as such we do not consider it in further experiments. Desirable properties of the MPNN-max as an algorithm executor persist when generalising to even larger graphs, as we demonstrate in Table 4 -demonstrating favourable generalisation on graphs up to 75× as large as the graphs originally trained on. We note that our observations also still hold Figure 3 : The per-step algorithm execution performances in terms of reachability accuracy (left), distance mean-squared error (middle) and predecessor accuracy (right), tested on 100-node graphs after training on 20-node graphs. Please mind the scale of the MSE plot. when training on larger graphs (Appendix C). We also find that there is no significant overfitting to a particular input graph category-however we do provide an in-depth analysis of per-category performance in Appendix D. Additional metrics The graphs we generate may be roughly partitioned into two types based on their local regularity-specifically, the ladder, grid and tree graphs all exhibit regular local structure, while the remaining four categories are more variable. As such, we hypothesise that learning from a graph of one such type only will exhibit better generalisation for graphs of the same type. We verify this claim in Table 5 , where we train on either only Erdős-Rényi graphs or trees of 20 nodes, and report the generalisation performance on 100-node graphs across the seven categories. The results directly validate our claim, implying that the MPNN-max model is capable of biasing itself to the structural regularities found in the input graphs. Despite this bias, the model still achieves generalisation performances that outperform any other model, even when trained on the full dataset. Further, we highlight that our choices of aggregation metrics may not be the most ideal way to assess performance of the algorithm executors: the last-step performance provides no indication of faithfulness to the original algorithm, while the mean-step performance may be artificially improved by terminating the algorithm at a latter point. While here we leave the problem of determining a better single-number metric to future work, we also decide to compound the results in Tables 1-2 by also plotting the test reachability/predecessor accuracies for each timestep of the algorithm individually (for 100-node graphs): refer to Figure 3 . Such visualisations can help identify cases where neural executors are "cheating", by e.g. immediately predicting every node is reachable: in these cases, we can see a characteristic-initially weak but steadily improving-performance curve. It also further solidifies the outperformance of MPNN-max. Lastly, in Appendix E we apply the recently proposed GNNExplainer model to detecting which graph substructures contributed the most to certain predictions. Sequential algorithm execution We demonstrate results for all considered architectures on executing Prim's algorithm within Table 6 . We provide the accuracy of predicting the next MST node GAT* (Veličković et al., 2018) 27.94% / 61.74% 22.11% / 58.66% 10.97% / 53.80% GAT-full* (Vaswani et al., 2017) 29.94% / 64.27% 18.91% / 53.34% 14.83% / 51.49% MPNN-mean (Gilmer et al., 2017) 90.56% / 93.63% 52.23% / 88.97% 20.63% / 80.50% MPNN-sum (Gilmer et al., 2017) 48.05% / 77.41% 24.40% / 61.83% 31.60% / 43.98% MPNN-max (Gilmer et al., 2017) 87 (computed against the algorithm's "ground-truth" ordering), as well as the accuracy of reconstructing the final MST (via the predecessors). As anticipated, our results once again show strong generalisation outperformance of MPNN-max. We additionally compared against a non-sequential version (no-algo), where the MPNN-max model was trained to directly predict predecessors (without requiring sequentially chosing nodes). This resulted in poor generalisation to larger graphs, weaker than even the LSTM sequential baseline. The insights from our setup verify that our neural graph execution paradigm is applicable to sequential algorithm execution as well-substantially expanding its range of possible applications. In this manuscript, we have presented the neural graph algorithm execution task, where-unlike prior approaches-we optimise neural networks to imitate individual steps and all intermediate outputs of classical graph algorithms, parallel as well as sequential. Through extensive evaluation-especially on the tasks of reachability, shortest paths and minimum spanning trees-we have determined a highly suitable architecture in maximisation-based message passing neural networks, and identified clear benefits for multi-task learning and positive transfer, as many classical algorithms share related subroutines. We believe that the results presented here should serve as strong motivation for further work in the area, attempting to learn more algorithms simultaneously and exploiting the similarities between their respective subroutines whenever appropriate. i : is i reachable from s in ≤ t hops? : has the algorithm terminated? Bellman-Ford i : predecessor of i in the shortest path tree (in ≤ t hops) Prim's algorithm (built from s after t steps)? p To aid clarity, within Table 7 , we provide an overview of all the inputs and outputs (supervision signals) for the three algorithms considered here (breadth-first search, Bellman-Ford and Prim).
We supervise graph neural networks to imitate intermediate and step-wise outputs of classical graph algorithms, recovering highly favourable insights. Suggests training neural networks to imitate graph algorithms by learning primitives and subroutines rather than the final output.