Causal Temporal Graph Convolutional Neural Networks (CTGCN)
Many large-scale applications can be elegantly represented using graph
structures. Their scalability, however, is often limited by the domain
knowledge required to apply them. To address this problem, we propose a novel
Causal Temporal Graph Convolutional Neural Network (CTGCN). Our CTGCN
architecture is based on a causal discovery mechanism, and is capable of
discovering the underlying causal processes. The major advantages of our
approach stem from its ability to overcome computational scalability problems
with a divide and conquer technique, and from the greater explainability of
predictions made using a causal model. We evaluate the scalability of our CTGCN
on two datasets to demonstrate that our method is applicable to large scale
problems, and show that the integration of causality into the TGCN architecture
improves prediction performance up to 40% over typical TGCN approach. Our
results are obtained without requiring additional domain knowledge, making our
approach adaptable to various domains, specifically when little contextual
knowledge is available.
Abigail Langbridge, Fearghal O'Donncha, Amadou Ba, Fabio Lorenzi, Christopher Lohse, Joern Ploennigs
# Causal Temporal Graph Convolutional Neural Networks (CTGCN)

###### Abstract.

Many large-scale applications can be elegantly represented using graph structures. Their scalability, however, is often limited by the domain knowledge required to apply them. To address this problem, we propose a novel Causal Temporal Graph Convolutional Neural Network (CTGCN). Our CTGCN architecture is based on a causal discovery mechanism, and is capable of discovering the underlying causal processes. The major advantages of our approach stem from its ability to overcome computational scalability problems with a divide and conquer technique, and from the greater explainability of predictions made using a causal model. We evaluate the scalability of our CTGCN on two datasets to demonstrate that our method is applicable to large scale problems, and show that the integration of causality into the TGCN architecture improves prediction performance up to 40 % over typical TGCN approach. Our results are obtained without requiring additional domain knowledge, making our approach adaptable to various domains, specifically when little contextual knowledge is available.

Graph Neural Networks, Causal Inference, Time Series, Scaling AI, Spatiotemporal

## 1. Introduction

Numerous real-world processes contain interconnected dynamics characterised by a complex organizational structure. Examples include environmental systems, energy grids, and epidemic outbreaks. Many classical mathematical frameworks exist to model these dynamics such as Navier-Stokes or convective-diffusion equations. While these explicitly encode known relationships, machine learning provides opportunity to resolve established and latent dynamics.

Graphs have been historically used to model many of the underlying structures and physical behaviours of spatial systems such as road networks (Srivastava et al., 2017), building thermodynamics (Srivastava et al., 2017), and water and energy grids (Koshelev et al., 2017). This allows practitioners to explicitly encode their domain knowledge. A fundamental assumption in these models is that the underlying structure of these dynamics are well known to be modelled. But, most real-world applications do not admit a-priori knowledge of these structures. Therefore, we aim to create a graph model that learns this structure using causal discovery.

Recent Temporal Graph Convolutional Neural Networks (TGCN) (Koshelev et al., 2017; Goyal et al., 2017) utilise this available domain knowledge in graphs by combining learning over temporal and graphical features. These TGCN models also assume that spatial information or similar prior knowledge regarding connections is available (Goyal et al., 2017; Goyal et al., 2017). However, in practise, many use cases for spatiotemporal modelling do not have well-defined graph structures. In contrast, in most practical applications we see clients collect new timeseries data faster than they are able to specify contextual information which grows the problem.

This lack or uncertainty of a-priori spatial knowledge is intensified for downstream tasks of the TGCN. First, incorrect graph models will strongly influence the prediction performance of the model limiting optimization and diagnostic tasks. Second, recent works have investigated the efficacy of post-hoc explanations for graph convolutional neural
network (GCNN) predictions (Sriramulu et al., 2019), with some approaches using causal inference methods (Han et al., 2019), but these fundamentally rely on the correctness and completeness of the graph over which they are explaining.

Some work has investigated treating the graph as a learnable parameter to optimise for a given downstream task (Sriramulu et al., 2019; Chen et al., 2020). Sriramulu et al. (Sriramulu et al., 2019) proposed an efficient method to construct a dynamic dependency graph based on statistical structure learning models. Prominent limitations of these methods is that they are prone to learning spurious correlations and not the underlying causes. Other work looks into identifying the graph from physics equations from scientific papers utilizing NLP approaches (Chen et al., 2020), but it assumes that systems exhibit static physical behaviour.

Also traditional Bayesian structure learning approaches are often not applicable to high-dimensional, real-world data due to their computational cost and unrealistic assumptions which include stationarity, the absence of latent confounders in data, and that none of the underlying relationships are contemporaneous (Kang et al., 2019).

In this work, we present a novel, scalable method for deducing causal relationships in large observational data, which we integrate into a TGCN architecture to overcome the limitation of requiring a-priori domain knowledge. This gives our causal-informed TGCN (CTGCN) visibility of the underlying structural causal model in an automated way and facilitates its self-adaptation and scalability in diverse large-scale applications. Furthermore, we investigate the computational complexity and predictive skill of the causal discovery process in order to improve the performance for large scale, real-world applications and non-IID datasets. We further study the effects of decomposing the causal discovery problem on different downstream predictive task, as we posit that integrating structural causal models into the graph convolution layer of a TGCN model improves model performance, robustness and explainability.

The main contributions of this work are:

* We develop a novel causal-informed TGCN discovery method in order to facilitate adoption for large-scale, interconnected, and complex applications.
* We extend existing causal discovery methods with spatial and temporal decomposition to improve their scalability for large-scale applications.
* We demonstrate that the integration of causal structure information into predictive models through a graph convolution layer improves forecasting performance.

## 2. Related Work

First, we provide a background on spatiotemporal graph convolutional neural networks (GCNN). We then discuss the utility of existing causal inference methods for large-scale data.

### Spatiotemporal GCNN

Our approach relies on GCNN (Han et al., 2019), initially introduced by Bruna et al. (Bruna et al., 2020), and extended by Duvenaud et al. (Duvenaud et al., 2020) with fast localised convolutions. Approaches such as TGCN (Han et al., 2019; Sriramulu et al., 2019; Sriramulu et al., 2019) augment this method with sequence-to-sequence learning to encode temporal dynamics. The penetration of GCNN into sensor-driven applications using time series data led to the extension of GCNN with sequence-to-sequence learning methods such as Gated Recurrent Unit (GRU) or Long Short-Term Memory (LSTM) for forecasting applications (Kirshman et al., 2019). The combination of GCNN with sequence-to-sequence learning methods is generally motivated by the need to simultaneously capture spatial and temporal dependencies. In this case, the GCNN is used to capture spatial dependencies by building a filter that acts on the nodes and their \(n\)-th order neighbourhood (usually first order). This enables the filter to capture spatial features between the nodes, and further the GCNN can be developed by stacking multiple convolutional layers. The sequence-to-sequence learning method is employed to capture dynamic changes of the systems.
However, modelling complex topological structures with sequence-to-sequence learning usually fails in capturing a system's underlying causal processes and thus restricts the input-output relationships to correlations. Scholkopf highlights the robustness of similar approaches as a key problem, suggesting the integration of causal modelling in ML could overcome this (Scholkopf, 2017). Many real-world problems that we are interested in studying violate the IID assumption, which underpins many correlational approaches. We propose an extension of the TGCN architecture that introduces a scalable causal convolution to capture the characteristics of the underlying system.

### Causal Discovery

Time-series data has long been a challenging problem in causal discovery: while the canonical order of data facilitates the directing of causal links (thus overcoming Markov equivalence), strong autocorrelation and the presence of unobserved confounders reduces detection power (Kal
To evaluate the performance of our approach, we define the spatiotemporal forecasting problem as learning the mapping function f using the structural information provided by the adjacency matrix \(A\) and the features \(\mathcal{X}\).

A GCNN model constructs the mapping function f as filter in the Fourier domain. The filter then acts on the nodes of the graph and its first order neighbourhood. This allows the topological structure and the spatial features between the nodes to be captured. Subsequently, the GCNN model can be established by stacking multiple convolutional layers. The GCNN is given by

\[\text{f}\left(\mathcal{X},A\right)=\sigma\left(\hat{A}\ \operatorname{ReLU} \left(\hat{A}\mathcal{X}W^{(0)}\right)\ W^{(1)}\right), \tag{1}\]

where \(\mathcal{X}\) is the feature matrix, \(A\) represents the adjacency matrix, \(\hat{A}=\tilde{D}^{-\frac{1}{2}}\tilde{A}\tilde{D}^{-\frac{1}{2}}\) is a preprocessing step, \(\tilde{A}=A+I_{N}\) is a matrix that considers the features of the nodes for which the learning is conducted, \(\tilde{D}\) is a degree matrix, where \(\tilde{D}=\sum_{j}\tilde{A}_{ij}\), \(W^{(0)}\) and \(W^{(1)}\) represent the weight matrix in the first and second neighborhood, and \(\sigma\), and \(\operatorname{ReLU}\) represent the activation functions.

### Causal Inference of the Adjacency Matrix

To solve the above GCNN problem, we need to identify the adjacency matrix \(A\). We frame this as a causal learning problem on the structure of the underlying causal processes:

\[\mathcal{X}_{t}^{j}=\mathrm{g}_{j}\left(\mathcal{P}(\mathcal{X}_{t}^{j}),\eta _{t}^{j}\right), \tag{2}\]

where \(\mathrm{g}_{j}\) are arbitrary measurable functions that depend non-trivially on the causal parents \(\mathcal{P}\) of the given node \(j\), and \(\eta\) is independent noise obscuring the causal processes.

As described in Section 1.3, there are various methods for discovering these causal relationships. In this work, we adapt the constraint-based method PCMCI\({}^{\star}\) presented by Runge (2007). This method increases the detection power over highly autocorrelated data compared to seminal methods (Kang et al., 2017), and its ability to detect contemporaneous links makes it particularly suitable for dealing with noisy real-world spatiotemporal data.

Causal discovery methods such as PCMCI\({}^{\star}\) are bound by assumptions that the observations are faithful to and fully representative of the underlying processes, and that these processes are stationary and acyclic. PCMCI\({}^{\star}\) is based on Runge's definition of a momentary conditional independence (MCI) test (Kang et al., 2017) that, for each lagged observation \(\mathcal{X}_{t-T}^{j}\), \(\mathcal{X}_{t}^{k}\) of a feature pair \(j,k\), tests for the existence of a causal relationship given a lag \(\tau\in(1,\ldots,\tau_{max})\). If the \(p\)-value of the test is above the significance threshold \(\alpha\), we consider a binary causal relationship between \(\mathcal{X}_{t-\tau}^{j}\), \(\mathcal{X}_{t}^{k}\) where \(c_{j,k,t,\tau}=\mathrm{L}(p_{j,k,t,\tau}>\alpha)\) with the logical function \(L(b)\) which is \(1\) if condition \(b\) evaluates true and \(0\) otherwise.

Figure 1. Block diagram of the proposed causal TGCN architecture.
The biggest limitation of the approach is its scalability. As an MCI test is computed for each lagged combination of features, we observe a worst-case time complexity of \(\mathcal{O}(P\cdot((N\cdot\tau_{max})^{2}+e^{N}))\) for PCMCI\({}^{\star}\). While this is a significant improvement on the original PC algorithm's worst case \(\mathcal{O}(P\cdot e^{N\cdot\tau_{max}})\), it is still intractable for large \(N\) and \(P\). The choice of CI tests also affects the runtime, with methods more robust to non-linear relationships, and therefore more generalisable, increasing computational cost.

### Temporal Decomposition

A key limitation of existing causal discovery methods is the assumption of stationarity, which is unrealistic for data spanning months or years and encompassing varying temporal dynamics. To overcome this, and improve the scalability of our method, we propose splitting the data along the time axis into approximately stationary periods with \(P_{T}\) observations.

These periods are typically derived based on generalised domain expertise or statistical analysis of exogenous features. Human systems such as buildings, energy, transport, or finance often exhibit daily, weekly, or monthly patterns, while natural systems such as oceans, agriculture, and weather often exhibit daily, seasonal, or annual dynamics.

### Spatial Decomposition

Modern systems in manufacturing, smart cities, and the automotive industry are highly monitored with thousands of IoT devices. The large spatial dimensionality is computationally challenging for existing CI algorithms. Intelligent spatial decompostion can dramatically reduce computational expense. Many decomposition approaches exist such as statistical, domain-inspired, or rule-based methods.

Dynamic time warping (DTW) is a pattern-matching approach to the alignment of time-series data first proposed for speech recognition by Sakoe & Chiba (2018). It has more recently been popularised as a method for the unsupervised classification of large temporal data in applications from finance to astronomy (Bahdan et al., 2017). The principal advantage of the DTW approach over Euclidean distances lies in the fact that one-to-many relationships can be mapped between candidate timeseries, which facilitates the identification of shared trends even if they occur on different timescales.

By utilising unsupervised DTW clustering on the feature axis, we can decompose the causal discovery problem into several sub-problems which we solve independently. We evaluate PCMCI\({}^{\star}\) within each cluster and for each temporal period. This divide-and-conquer approach reduces the worst-case complexity of PCMCI\({}^{\star}\) to \(\mathcal{O}(D\cdot P_{T}\cdot((N_{C}\cdot\tau_{max})^{2}+e^{N_{C}}))\) with \(P_{T}\) being the temporal decomposition period, \(D\) being the cluster number and \(N_{C}\) being the maximum cluster size. We posit that by clustering in this way, we minimise the number of cross-cluster relationships in the underlying causal model, and therefore maximise the recall of the decomposed problem.

Further, as DTW clustering is an unsupervised method, this decomposition requires few additional parameters to be run automatically as shown in Table A3.

### Adjacency Matrix Construction

The causal discovery steps outlined above produce results \(c_{j,k,t,\tau}=\mathrm{L}(p_{j,k,t,\tau}>\alpha)\) for each \(\tau\)-lagged timestep \(t\) and detect causal relationships which span from contemporaneous (\(\tau=0\)) up to a maximum lag (\(\tau_{max}\)) which exceed some significance threshold \(\alpha\). These parameters and their selection are summarised in Table A1.

After the temporal and spatial decomposition we retrieve \(c_{j,k,t,\tau}\) for all temporal and spatial clusters and aggregate them in form of our causal adjacency matrix \(A\). We first aggregate all test results within each temporal \(\mathcal{T}\) and spatial \(\mathcal{S}\)
sample set and perform a majority vote

\[\hat{c}_{j,k,\mathcal{T},\mathcal{S}}=\operatorname{M}\!\left(\sum_{t=0}^{P_{T}} \sum_{\tau=0}^{\min(t,\tau_{max})}\frac{1}{P_{T}(\tau_{max}-1)}c_{j,k,t,\tau}\right) \tag{3}\]

with the majority voting function \(\operatorname{M}\!\operatorname{V}(v)\) that is \(1\) if \(v>0.5\) and \(0\) otherwise. This filters out causal relationships that were only discovered in specific time steps, but, are not common within a temporal and spatial sample set. We then aggregate the votes from all temporal and spatial sample sets in our causal adjacency matrix \(A\). We use the following aggregation strategies:

\[a_{j,k}^{\operatorname{ANY;W}}=\sum_{\mathcal{S}}\sum_{\mathcal{T}}\hat{c}_{ j,k,\mathcal{T},\mathcal{S}}, \tag{4}\]

The weighted aggregation \(a_{j,k}^{\operatorname{ANY;W}}\) computes the total sum of causality test results. The weighted majority vote \(a_{j,k}^{\operatorname{MT;W}}\) considers the weight of relationships occurring in a majority of sample sets in the number \(T\) of temporal sets. The unweighted aggregations \(a_{j,k}^{\operatorname{ANY;UW}}\) and \(a_{j,k}^{\operatorname{MT;UW}}\) reduce the weighted adjacency matrix to a binary one using the binary function \(\operatorname{B}(v)\) that is \(1\) if \(v>0\) and \(0\) otherwise. If we do not model the directed behaviour of the system, we can simplify this to an undirected matrix via \(a_{j,k}=\operatorname{B}(a_{j,k}+a_{k,j})\).

### Evaluating Performance

Forecasting is a common task in spatiotemporal applications for monitoring, anomaly detection, optimisation, etc. We consider the task of windowed forecasting, where a model is trained to infer the subsequent \(q\) measurements given a finite history of length \(\lambda\).

Despite the growing popularity of sequence-to-sequence learners for forecasting applications, we consider the foundational case of a simple one-dimensional convolution along the time axis to capture the temporal behaviour of the system. This has the dual benefit of reducing training overhead for improved scalability and clearly demonstrating the effectiveness of our method even with a simplistic architecture. The simpler network structure also enables explainability (Srivastava et al., 2017) that is important for practical applications.

We design a forecasting architecture which begins with a temporal convolution, followed by the causal graph convolution layer. We posit that this ordering allows the temporal dynamics of the system to be captured at each node, and then these distilled features can be used to inform forecasting at causally-related nodes.

The TGCN we employ builds on Kipf & Welling (2014) and is implemented in PyTorch Geometric (Kipf and Welling, 2021). A final linear layer produces a forecast of length \(q\). The model is trained in batches using RMSE loss. Hyperparameters are tuned for each dataset using the adaptive grid search method provided by Optuna (2021).

## 4. Experiments

We demonstrate the performance of our method in two contexts, selected to demonstrate the generalisability of the approach. Experiments were conducted on an Apple M1 Max 32GB MacBook Pro (2021) running MacOS Monterey 12.6 and Python 3.9.12.

Code to replicate our experiments is provided at [https://tinyurl.com/ctgcn-code](https://tinyurl.com/ctgcn-code) and will be open-sourced
### Building Heating System

The first dataset is heterogeneous and multivariate, and generated from a simulated heating, ventilation, air conditioning (HVAC) system which controls the temperature of two rooms and an interconnecting corridor (Zhou et al., 2017). It consists of data from thirty sensors measuring the system variables including the internal status of the system's boiler, chiller and ventilation units, the temperature within each room and externally, and room occupancy. The data is sampled every 10 minutes over a total period of one year.

We included this dataset in our experiments as, due to its simulated nature, we have a ground truth about the causal relationships which we can use to investigate the quality of the causal discovery, as well as to measure the performance of the CTGCN on the downstream forecasting task. The dataset is further strongly heterogenous with datapoints representing sensors with different characteristics and value ranges. It is known that this is a challenging task for GCNNs (Chen et al., 2016; Chen et al., 2016), and may also adversely effect the causal discovery, rendering this a particularly interesting study.

The ground truth adjacency matrix is defined based on the underlying physics of the simulation (Zhou et al., 2017), and contains 50 relationships between the 30 sensors.

For this dataset we evaluate performance for both temporal and spatiotemporal decomposition in terms of the accuracy of causal discovery, the performance of forecasting the temperature of one of the rooms, and the runtime.

Our temporal decomposition consists of a daily temporal split with \(P_{T}=144\) based on auto-correlation analysis, yielding 365 causal sub-problems. We select one hour (\(\tau_{max}=6\)) as the maximal time lag based on system dynamics. For the spatial decomposition we use 10 clusters after running an elbow test.

### Highway Traffic Flow

We further verify our method on homogeneous real-world traffic flow data collected by the California Department of Transport. The data consists of 30-second traffic flow speed measurements collected from 228 locations across the California state highway system over weekdays in May and June 2012. We downsample the data to five-minute intervals to align with work by Yu et al. (Yu et al., 2017) and facilitate comparison. In that work, Yu et al. also proposes a distance-thresholding mechanism to identify the adjacency matrix

\[w_{ij}=\begin{cases}exp\left(-\frac{d_{i,j}^{2}}{\sigma^{2}}\right),&\text{if }i \neq j\text{ and }exp\left(-\frac{d_{i,j}^{2}}{\sigma^{2}}\right)\geq\epsilon;\\ 0,&\text{otherwise}.\end{cases} \tag{5}\]

We evaluate the effectiveness of our method against their results using their parameters \(\sigma^{2}=10\) and \(\epsilon=0.5\).

To overcome the non-stationarity of the data, we split the data by date, conducting causal discovery for each of the 44 days of data. We also conduct spatial decomposition with 25 clusters estimated using an elbow test. The maximal lag we consider is \(\tau_{max}=9\) to align with Yu et al. (Yu et al., 2017).

For this dataset, we demonstrate the runtime improvement of spatiotemporal decomposition over temporal, and evaluate the compromise between runtime and prediction accuracy.
## 5. Results

### Building Heating System

We first compare the ground-truth with the temporal TGCN and the spatiotemporal (DTW clustered) TGCN approach with different aggregation strategies. Table 1 shows the precision, accuracy and F1 score for the different approaches outlined in Section 3.5.

The accuracy of all approaches is higher than 83 % due to a high true negative rate as the association matrix is sparse, hence precision is a more relevant metric. Precision is only 14.2 % to 16.6 % for the temporal approach, which may be surprising as this approach should be able to discover all potential causal relationships. However, this results in a high false positive rate with about 150 causal relationships discovered. The spatiotemporal approach has a significantly higher precision with 32.8 % to 39.2 % discovering an average of 25 relationships. Despite this, the method is not able to discover all relationships due to the heterogenous nature of the dataset and our spatial decomposition. We see that data are grouped semantically by the DTW clustering step, e. g. separate clusters are created for temperature, CO2, occupancy and humidity. As our algorithm does not consider causal relationships to exist between clusters, we miss these relationships (See Appendix C for details). Nonetheless, the spatiotemporal approach has a higher accuracy, precision and a significantly lower compute time. The temporal approach took 288 hours (12 days) to compute, while the spatiotemporal approach computed in 49 hours (2 days, see Table 3).

The matrix aggregation step also improves the result. One performance improvement approach could be to sample individual days from the dataset to estimate the adjacency matrix. This approach (AVG) has a mean precision of 32.8 %. But, analysing the full dataset and applying the Majority Threshold (MT) or the ANY aggregation we improve precision to 38.4 % and 39.2 %, respectively.

We further compare the prediction performance of the discovered adjacency matrix to the ground truth when forecasting the room temperature. This has multiple causal relationships and relevant practical applications in energy management. As detailed in Section 3.6, we utilise a CTGCN that is configured with the different association matrices resulting from the temporal and spatiotemporal clustering and the different aggregation methods for prediction. Table 2 compares the results. We provide also the RMSE for an unconnected and fully connected TGCN for context.

The RMSE for the ground truth prediction is significantly better than the unconnected and fully connected TGCN demonstrating the well-known benefits of using domain knowledge in TGCN configuration. The unweighted temporal and spatiotemporal CTGCN outperform the unconnected TGCN but not the ground truth. This is expected given that not all causal relationships were discovered. However, the temporal and spatiotemporal CTGCN is able to outperform the ground truth TGCN when weighted by frequency. The frequency clearly encodes important information about the relevance of a causal relationship that is not contained in the binary ground truth, and these weights allow the TGCN to compensate for the missing edges. In result, the spatiotemporal weighted majority threshold had the lowest RMSE, showing that our CTGCN can outperform the ground truth in a prediction problem.

\begin{table}
\begin{tabular}{l l l l} \hline \hline Approach & Precision & Accuracy & F1 \\ \hline Temporal AVG & 14.2 \% & 83.1 \% & 14.1 \% \\ Temporal MT & 16.6 \% & 84.7 \% & 15.2 \% \\ Temporal ANY & 15.2 \% & 83.9 \% & 14.6 \% \\ \hline Spatiotemporal AVG & 32.8 \% & 88.2 \% & 21.9 \% \\ Spatiotemporal MT & 38.4 \% & 88.8 \% & 26.3 \% \\ Spatiotemporal ANY & 39.2 \% & 88.8 \% & 28.2 \% \\ \hline \hline \end{tabular}
\end{table}
Table 1. Results for the building dataset.
### Highway Traffic Flow

The mean runtime of each day of the temporally-decomposed traffic data was 35 hours, corresponding to an estimated total runtime of more than 64 days (Table 3). As such, causal discovery was not conducted for every temporal split with this method. Instead, we selected the first, last, and central five days of data to get an overview of the entire period and include variations in causal relationships over time. Forecasting performance was measured across the entire dataset to test the representativity of the causal discovery method when forecasting on unseen data.

As evidenced in Fig. 2 and Appendix Table B4, downstream performance is significantly improved by introducing the temporally decomposed causal adjacency matrix into the TGCN architecture. Forecasting RMSE was on average 30.4% improved over the benchmark, with the best-performing and worst-performing days yielding 33.5% and 23.6% improvements respectively.

To combine the causal graphs calculated from different days of data, the majority threshold aggregation method was used. This aggregation improves the performance of the temporal decomposition, resulting in an RMSE 36.0 % lower than the benchmark, and lower than any of the daily graphs. This result is notable as it demonstrates that by combining causal graphs from different sections of the data we can better capture the overall causal relationships.

To compare our results to a state-of-the art approach, we added the STGCN results from Yu et al. (Yu et al., 2019). Note that the RMSE of the same distance based benchmark is higher than the STGCN due to our simpler sequence-to-sequence learner. But, the improved adjacency matrix of the Temporal MT graph outperforms even the more advanced STGCN.

As shown in Table 3, introducing spatial decomposition accelerates causal discovery on this data by more than 14 times. Despite this, Fig. 2 demonstrates that predictive performance was not negatively affected, with performance improving in some cases. The minimum, mean and maximum improvement over the benchmark are 26.1%, 33.5% and 37.4% respectively. Spatiotemporal performance is further improved through MT aggregation, with RMSE 39.1% lower than the benchmark and 4.9% lower than Temporal MT.

\begin{table}
\begin{tabular}{l c} \hline \hline Approach & RMSE \\ \hline Ground Truth TGCN & **0.8297** \\ \hline Unconnected TGCN & 0.8981 \\ Fully connected TGCN & 0.8548 \\ \hline Temporal MT Unweighted CTGCN & 0.8668 \\ Temporal MT Weighted CTGCN & 0.8469 \\ Temporal ANY Unweighted CTGCN & 0.8735 \\ Temporal ANY Weighted CTGCN & **0.8250** \\ \hline Spatiotemporal MT Unweighted CTGCN & 0.8668 \\ Spatiotemporal MT Weighted CTGCN & **0.8209** \\ Spatiotemporal ANY Unweighted CTGCN & 0.8306 \\ Spatiotemporal ANY Weighted CTGCN & 0.8735 \\ \hline \hline \end{tabular}
\end{table}
Table 2. Comparison of the prediction accuracy of different aggregation methods for the building dataset.

\begin{table}
\begin{tabular}{l c c c} \hline \hline Dataset & Temporal & Spatiotemporal & Factor \\ \hline Building & 287.7 h & 49.8 h & 5.8 x \\ Traffic & 1540.0 h & 106.2 h & 14.5 x \\ \hline \hline \end{tabular}
\end{table}
Table 3. Comparison of causal discovery runtimes in hours for the datasets.
## 6. Discussion

Our experiments have shown that using an automatically discovered causal adjacency matrix with a TGCN architecture can improve the forecasting performance over typical spatial approaches. Further, we demonstrate that through context-aware decomposition of the causal discovery problem we accelerate the discovery process such that previously intractable problems can be computed on a single machine. This is particularly notable for the traffic dataset, which is accelerated 14 times when the problem is both spatially and temporally decomposed over a temporal decomposition.

Importantly, our approach reports high predictive skill with a simplistic sequence-to-sequence learner. Due to the lightweight architecture--constructed of a simple 1-D convolution and a graph convolution layer--we posit that the predictions from this model are more interpretable than more complex SOTA architectures (Srivastava et al., 2017). Simple weight visualisation or gradient based approaches can provide efficient insight into model performance for such 1-layer systems (Kang et al., 2018). A notable advantage of the proposed framework is that a causal model accurately representing underlying system dynamics allows us to retain performance with the lightweight architecture.

Causal discovery can generate a more practical representation of latent and contemporaneous relationships. The power of GNNs has been demonstrated across many applications in recent years such as estimating travel times in Google Maps, powering content recommendations in Pinterest, and providing product recommendations in Amazon (Srivastava et al., 2017). We propose an agnostic and scalable framework for effective graph generation for such applications.

We also illustrate that the method used to aggregate causal results from sub-problems is important, with the weighted majority threshold method yielding performance improvements over any of its constituent graphs. We posit that this is due to its ability to capture information about the relevance of most common causal relationships in the data and filters non-stationary relationships.

However, our results are not exhaustive: we suggest that by implementing a more balanced clustering method, we might see improved precision. Also, the scalability remains a problem for very large datasets (\(N>>1000\)) with dense

Figure 2. Forecasting RMSE on the traffic dataset compared to STGCN (Srivastava et al., 2017)
causal models where the spatial decomposition capability is limited. There also remains opportunity to expand this work toward the explainability of the CTGCN model.

## 7. Conclusion

While TGCN demonstrates excellent forecasting skill across a wide variety of applications, the need for manually configured adjacency matrices hinder large-scale deployments. We demonstrate that causal discovery methods generate a more robust graph structure that better capture system dynamics, and demonstrate improved predictive skill on two real-world datasets and in comparison to state of the art. We address computational scalability and non-stationarity by implementing efficient spatial and temporal decomposition.

We show that CTGCN demonstrates particularly impressive performance when data is decomposed based on stationarity, causal relationships are calculated for each sub-problem, and aggregated to incorporate these temporal dynamics. Future work in this space is in the direction of improving the clustering method, automatic parameter identification, and an evaluation of explainability.
