Futures Quantitative Investment with Heterogeneous Continual Graph
  Neural Network
This study aims to address the challenges of futures price prediction in
high-frequency trading (HFT) by proposing a continuous learning factor
predictor based on graph neural networks. The model integrates multi-factor
pricing theories with real-time market dynamics, effectively bypassing the
limitations of existing methods that lack financial theory guidance and ignore
various trend signals and their interactions. We propose three heterogeneous
tasks, including price moving average regression, price gap regression and
change-point detection to trace the short-, intermediate-, and long-term trend
factors present in the data. In addition, this study also considers the
cross-sectional correlation characteristics of future contracts, where prices
of different futures often show strong dynamic correlations. Each variable
(future contract) depends not only on its historical values (temporal) but also
on the observation of other variables (cross-sectional). To capture these
dynamic relationships more accurately, we resort to the spatio-temporal graph
neural network (STGNN) to enhance the predictive power of the model. The model
employs a continuous learning strategy to simultaneously consider these tasks
(factors). Additionally, due to the heterogeneity of the tasks, we propose to
calculate parameter importance with mutual information between original
observations and the extracted features to mitigate the catastrophic forgetting
(CF) problem. Empirical tests on 49 commodity futures in China's futures market
demonstrate that the proposed model outperforms other state-of-the-art models
in terms of prediction accuracy. Not only does this research promote the
integration of financial theory and deep learning, but it also provides a
scientific basis for actual trading decisions.
Min Hu, Zhizhong Tan, Bin Liu, Guosheng Yin
# Futures Quantitative Investment with Heterogeneous Continual Graph Neural Network

###### Abstract

It is a challenging problem to predict trends of futures prices with traditional econometric models as one needs to consider not only futures' historical data but also correlations among different futures. Spatial-temporal graph neural networks (STGNNs) have great advantages in dealing with such kind of spatial-temporal data. However, we cannot directly apply STGNNs to high-frequency future data because future investors have to consider both the long-term and short-term characteristics when doing decision-making. To capture both the long-term and short-term features, we exploit more label information by designing four heterogeneous tasks: price regression, price moving average regression, price gap regression (within a short interval), and change-point detection, which involve both long-term and short-term scenes. To make full use of these labels, we train our model in a continual manner. Traditional continual GNNs define the gradient of prices as the parameter important to overcome catastrophic forgetting (CF). Unfortunately, the losses of the four heterogeneous tasks lie in different spaces. Hence it is improper to calculate the parameter importance with their losses. We propose to calculate parameter importance with mutual information between original observations and the extracted features. The empirical results based on 49 commodity futures demonstrate that our model has higher prediction performance on capturing long-term or short-term dynamic change.

## 1 Introduction

Futures prices are regarded as an authoritative indicator of market conditions. As the core of futures quantitative investment, high-precision trend prediction of futures prices is of great practical significance. For decades, classical trend predict methods of time series such as vector autoregression (VAR)[22], autoregressive integrated moving average (ARIMA) model[15], generalized autoregressive conditional heteroskedasticity (GARCH) [1], which assume specific models to characterize financial time series data, were widely considered to be a general approach to capture the temporal linear dependencies.

However, more often than not, financial systems are complex and their underlying dynamics are not known [23]. At least two significant characteristics of the futures data have been ignored or partially ignored by the existing works. First, in quantitative investment, especially in high-frequency trading, faster and more precise predictions can provide reliable signals for trading strategies so as to timely and accurately make trading decisions. But in practice, it is extremely harder to predict the short-term trend than the long-term trend. Most of the existing methods ignore the interaction between the short-term and long-term features, which may cause the model to fail to obtain enough information, thus limiting the accuracy of the prediction [3]. Second, the price of one future usually shows a strong dynamic correlation with other futures [1]. That is, each variable (futures contract) depends not only on its historical values but also on the observation of other variables. For example, a shortage of petroleum may impact the price of synthetic resin at the downstream industrial chain. These two features of real-time series data cannot be reflected by analytical equations containing parameters [22, 14].

The deep learning models are proven to be more efficient than classical econometric models. Lots of deep learning methods have been extended to predict the multivariate time series [16, 20, 21]. For example, ConvLSTM [16] has been proposed to model the temporal dependence with the recurrent structure of LSTM and capture the spatial correlation with a traditional convolution. However, LSTM variants do not model the pair-wise dependencies among variables explicitly, which weakens model interpretability [20]. Performing node feature aggregation and updates, graph neural network (GNN) [1, 1] extends the deep learning methodology to graphs that can capture these complex dependencies naturally.

In this work, we focus on modeling the multiple futures series by considering the two aforementioned characteristics simultaneously. To capture the spatial correlation among futures, we propose to use STGNNs to model the multivariate
future data as a base model. However, if we only focus on price forecasting or change-point detection (CPD), there will be only one kind of supervised label, which is insufficient to learn comprehensive features to balance long- and short-term considerations. To alleviate this problem, we propose four heterogeneous tasks, futures' close price forecasting, gap regression, moving average (MA) regression, and change-point detection (CPD). To learn holistic long-term and short-term features, we train the proposed model with these four tasks in a continual manner.

The rationale is that the task of gap regression captures the law behind the spreads between the maximum and the minimum within a fixed time interval. The task of real-time futures price forecasting aims to capture the short-term feature from local fluctuation. On the contrary, the task of MA depicts the average settle prices over a window of time to show the historical volatility of futures prices, which tries to extract the long-term signal. The task of CPD helps us to find abrupt changes in data when a property of the time series changes, which takes into account both long-term and short-term features. In addition, we force our model to learn an overall experience from the four tasks in a continual way to avoid "catastrophic forgetting" on the long-term and short-term features [11].

Traditionally, training artificial neural networks on new tasks typically causes them to forget previously learned tasks, which refers to the issue of catastrophic forgetting [11]. Zenke et al.[14] propose a quadratic surrogate loss to solve this problem. The key point of the surrogate loss is a per-parameter regularization strength. It tries to "remember" the best model parameters for the last task as well as to adapt to the new task. The existing works [14, 11] approximate the per-parameter regularization strength with the ratio of loss dropping (the gradient of loss). In our problem, however, the tasks are heterogeneous with each other, therefore their corresponding losses are heterogeneous as well. Hence, the loss dropping based on per-parameter strength is inappropriate for futures price modeling. In this paper, we creatively introduce mutual information between the extracted features and original features to substitute the traditional loss change ratio. The rationale is that the mutual information among different tasks is much smoother than their losses. Experimental results show that our model has higher prediction accuracy and can effectively predict the dynamic trend of multi-future time series data.

The main achievements, including contributions to the field, can be summarized as follows:

* Conceptually, we propose four heterogeneous tasks to balance both long-term and short-term quantitative trading requirements.
* Technically, we propose a novel Heterogeneous Continual Graph Neural Network (HCGNN) to learn temporal (long-term and short-term) and spatial dependencies. We design a mutual information-based parameter strength to adapt to heterogeneous tasks.
* Empirically, we evaluate the proposed model on a real-world dataset that involves 49 types of futures in the Chinese futures market.

## 2 Related Works

### Deep Learning in Financial Quantitative Investment

A lot of literature have studied deep learning in financial quantitative investment. Solving the problem of long-term dependence of the series well, LSTM [13] and its variants [12][11] are the preferred choice of most researchers in the field of financial time series forecasting, including but not limited to stock price forecasting[14], index forecasting [1, 15, 16], commodity price forecasting[17], forex price forecasting[18], volatility forecasting[19]. Salinas et al. [16] embedded LSTM or its simplified version gated recurrent units (GRU) as a module in the main architecture. Meanwhile, hybrid models were used in some of the papers. For example, Wu et al.[20] combined CNN (Convolutional Neural Networks) and LSTM to predict the rise and fall of the stock market. They exploited the ability of convolutional layers for extracting useful financial knowledge as the input of LSTM. Combining complementary ensemble empirical mode decomposition (CEEMD), PCA, and LSTM, Zhang et al. [21] constructed a deep learning hybrid prediction model for stock markets based on the idea of "decomposition-reconstruction-synthesis". For multivariate time series, Lai et al. [10] proposed a deep learning framework, namely a long- and short-term time-series network (LSTNet) which employed convolutional neural networks to capture local dependencies. LSTNet cannot fully exploit latent dependencies between pairs of variables and is difficult to scale beyond a few thousand-time series. Similar attempts were made in [15] and [16]. In general, these methods often completely disregard available relational information or can not take full advantage of nonlinear spatial-temporal dependencies.

As a popular model emerging in recent years, GNN has been applied to different tasks and domains. Due to its good performance in graph modeling methods, GNN has been continuously promoted in the financial field by transforming financial tasks into node classifications. To represent relational data in the financial field, graphs are usually constructed, including user relations graphs [23], company-based relationship graphs [24], and stock relationship graphs [11, 12]. Sawhney et al. [25] concentrated on stock movement forecasting by blending chaotic multi-modal signals including inter-stock correlations via graph attention networks (GATs) [26]. Taking the lead-lag effect in the financial market into account, Cheng et al. [2] constructed heterogeneous graphs to learn from multi-modal inputs and proposed a multi-modality graph neural network (MAGNN) for financial time series prediction. Besides constructing graphs from prior knowledge, for multivariate
time series forecasting, researchers have been trying to discover graphical relationships between variables from the data.

With multiple ways of graph construction, it becomes challenging to obtain and select the relation for graph construction [20, 14]. For example, Cao et al. [14] proposed a StemGNN, which constructed a graph with the attention mechanism. Then they decompose the spatial-temporal signal into a spectral domain and frequency domain with GFT (graph Fourier transform) and DFT (discrete Fourier transform) respectively for multivariate time-series forecasting. Departing from their work, we extend StemGNN to a continual learning scene.

### Multi-task Continual Learning

In the area of multi-task continual learning (CL), a well-known problem is catastrophic forgetting (CF) [13]. That is when training a new task the mode will update existing network parameters, which may cause accuracy degradation of the task's former tasks that the model has learned. Using regularization to mitigate CF by penalizing changes to important parameters learned in the past is a popular approach [15]. Rehearsal is another popular approach, which stores or generates examples of previous tasks and replays them in training a new task [21]. Almost all online CL methods are based on rehearsal. Farajtabar et al. [1] addressed the CF issue from a parameter space perspective and studied an approach to restricting the direction of the gradient updates to avoid forgetting previously-learned data. Another alternative CL solution is Elastic Weight Consolidation (EWC) [16] and its variations [15, 17, 1]. The idea of these methods is trying to preserve the optimal parameters inferred from the previous tasks while optimizing the parameters for the next task. Liu et al. [13] extended EWC to GNNs by proposing a topology-aware weight preserving (TWP) term. However, all these models assume that all the tasks are isomorphic. In this work, we proposed a new framework for a heterogeneous multi-task setting.

## 3 Problem Formulation

We study 49 commodity futures in China market, which are multivariate time series data. In order to effectively represent the structural relationship among future varieties, we use the graph to describe the inter-series correlations. Let \(\mathcal{G}=(\mathbf{A},\mathbf{X})\), \(\mathbf{A}\in\mathbb{R}^{N\times N}\) denotes an adjacent matrix of the futures and \(\mathbf{X}=\{x_{i,t}\}\in\mathbb{R}^{N\times T}\) represents the input of multivariate time series, \(N\) is the number of future varieties.

### Four Tasks for Future Price Modeling

We propose four well-designed tasks for future data modeling, including **price forecasting**, **gap regression**, **moving average (MA) regression**, and **change-point detection (CPD)**.

**Future Price Forecasting.** Given the historical observations of previous \(T\) timestamps \(\mathbf{X}_{t-T},\ldots,\mathbf{X}_{t-2},\mathbf{X}_{t-1}\), where \(\mathbf{X}_{t}\in\mathbb{R}^{N}\), the goal for a multivariate time series forecasting is to learn a mapping function from the historical values to the next \(T^{{}^{\prime}}\) timestamps \(\mathbf{X}_{t},\mathbf{X}_{t+1},\ldots,\mathbf{X}_{t+T^{\prime}-1}\) in the graph \(\mathcal{G}\),

\[[\mathbf{X}_{t-T},\ldots,\mathbf{X}_{t-1},\mathbf{A};\Theta;W]\rightarrow[ \mathbf{X}_{t},\ldots,\mathbf{X}_{t+T^{\prime}-1}],\]

where \(\Theta\), \(W\) are necessary parameters.

**Gap Regression.** We encourage our model to make a regression on the dispersion ratio within a fixed time window to capture short-term features from local fluctuation as shown in Equation 1. We call it gap regression,

\[\Delta\mathbf{X}_{t}^{(l)}=\frac{\mathbf{X}_{max}-\mathbf{X}_{min}}{l} \tag{1}\]

where \(\mathbf{X}_{max}=\max(\mathbf{X}[:,t:t\!+\!l\!-\!1])\) (\(\mathbf{X}_{min}=\min(\mathbf{X}[:,t:t+l-1])\)) is a row maximum (minimum) operation within the sliding window, which returns row maximum (minimum) values of the slice \(\mathbf{X}[:,t:t+l-1]\), \(l\) is the window length.

**Moving Average Regression.** A moving average is a type of convolution. It can be viewed as an example of a low-pass filter that can smooth signals. In this work, we resort to the moving average to capture the long-term change trends of futures prices. Given a discrete time series with observations \(\mathbf{X}_{t}\), a moving average of \(\mathbf{X}_{t}\) can be calculated as follows,

\[\bar{\mathbf{X}}_{t}=\left(\mathbf{h}*\mathbf{X}\right)_{t}=\sum_{i=-L}^{i=L} \mathbf{h}\left(i\right)\mathbf{X}\left(t-i\right),\]

where \(\bar{\mathbf{X}}_{t}\) is the smoothed time series and \(\mathbf{h}\) denotes a discrete density function (that is \(\sum_{i=-L}^{i=L}\mathbf{h}\left(i\right)=1\)). The input sequence \(\mathbf{X}_{t}\) could be the futures' historical settle prices.

**Predicting change-points.** During futures trading, it is of great importance for investors to detect trend changes in price series in a timely and accurate manner, whether they are manual traders or programmed traders. In a way, if an investor can accurately predict a trend reversal, he can not only increase excess returns for the investor but also avoid a lot of unnecessary losses. We have therefore investigated the task of change-point detection to increase the excess returns of the investments. We calculate the change-points of the real price data with ruptures [15] and then try to classify them in our model. The labels of change-points are defined as follows,

\[y_{x_{i,t_{k}}}=\left\{\begin{array}{ll}1&\quad\text{change-point at $t_{k}$}\\ 0&\quad\text{otherwise}\end{array}\right.\]

The four tasks in this article are described as follows,

\[[\mathbf{X}_{t-T},\ldots,\mathbf{X}_{t-1},\mathbf{A};\Theta;W]\overset{F}{ \rightarrow}\begin{cases}\left[\mathbf{X}_{t},\ldots,\mathbf{X}_{t+T^{\prime }-1}\right]\\ \Delta\mathbf{X}_{t}^{(l)}\\ y(x_{i,t_{k}}),t_{k}\geq t\end{cases} \tag{2}\]

where these values can be inferred by an overall forecasting model \(F:=g(f(x;\Theta);W)\) with parameter \(\Theta\) and \(W\) as shown in Figure 1. \(f(x;\Theta)\) is a spatial-temporal module and \(g(f;W)\) is a link function to adapt the downstream tasks as we stated before. Usually, \(g\) is a multi-layer perception. To learn the common knowledge of all tasks while balancing the learning performance of all individual tasks, we conduct a continual learning the four tasks, which will be introduced in next section.
Model

In this section, we provide a detailed formulation of a Heterogeneous Continual Graph Neural Network (HCGNN).

### Overview

Figure 1 illustrates the overall framework of the proposed method. With the inputs of multivariate futures data from the four tasks, we extract their features \(f\) with the spatial-temporal modules in \(f(\mathbf{X};\Theta)\). The spatial-temporal modules are connected in a residual mode. Then we adapt them to the four downstream tasks (as shown in Equation 2) with the module \(g(f;W)\), which could be a multi-layer perception. The feature extraction process can be defined by the composite function \(F:=g(f(\mathbf{X};\Theta);W)\) as we stated in section 3.1. Finally, we train the four tasks in a continual manner.

Suppose there are four related tasks for futures data modeling, \(\mathcal{T}:=\{\mathcal{T}^{1},\mathcal{T}^{2},\mathcal{T}^{3},\mathcal{T}^{4}\}\). As shown in Equation 2, all four tasks share the feature \(\mathbf{X}_{t-T,t-1}\), but their target spaces are heterogeneous. That is, \(\mathbf{Y}_{t}^{1}=\mathbf{X}_{t,t+T^{\prime}-1}\), \(\mathbf{Y}_{t}^{2}=\Delta\mathbf{X}_{t}^{(l)}\), \(\mathbf{Y}_{t}^{3}=\bar{\mathbf{X}}_{t}\), \(\mathbf{Y}_{t}^{4}\in\{0,1\}\). We train the proposed model in a continual learning manner. Unfortunately, the traditionally continual learning failed to consider a heterogeneous setting of tasks. In this work, we propose a mutual information based method to satisfy such a heterogeneous requirement for continual learning as follows,

\[L_{k+1}(\Theta,W) =L_{k+1}^{\text{new}}(\Theta,W)+\lambda_{1}\sum_{n=1}^{k}\Omega^ {\Theta}\odot\|\Theta-\Theta_{n}^{*}\|_{2}^{2}\] \[+\lambda_{2}\sum_{n=1}^{k}\Omega^{W}\odot\|W-W_{n}^{*}\|_{2}^{2},\]

where \(\lambda_{1}\) and \(\lambda_{2}\) are regularization penalties, \(L_{k+1}^{\text{new}}(\Theta,W)\) denotes the \((k+1)\)-task-related loss function, e.g., it could be a cross-entropy loss for the classification task 4 or mean square error for regression tasks. \(\Theta_{n}^{*}\) and \(W_{n}^{*}\) are the optimal parameters of the \(n\)-th task. \(\odot\) is an element-wise product. \(\Omega^{\Theta}\) and \(\Omega^{W}\) play a role of "indicator" matrices that try to "divide" the network parameters \(\Theta\) (\(W\)) into groups for each task. Large values in \(\Omega^{\Theta}\) (\(\Omega^{W}\)) indicate the corresponding parts in \(\Theta_{n}^{*}\) (\(W_{n}^{*}\)) are important memories for the previous task \(n\) (\(n=0,...,k-1\)), hence we have to keep them when training the task \(k+1\). In other words, we can only allow updating the parts of \(\Theta\) (\(W\)) where the indicator values in \(\Omega^{\Theta}\) (\(\Omega^{W}\)) are small. This learning strategy can not only ensure that the parameters of low-importance scores can be freely changed to adapt to new tasks, but also punish the parameters of high-importance scores so that the model can still perform well on previous tasks.

The key part of the continual learning framework is the calculation of the per-parameter regularization strength \(\Omega:=\{\Omega^{W},\Omega^{\Theta}\}\)[20]. In [20], \(\Omega\) was defined with the gradient of the loss. However, considering the heterogeneous tasks in our problem, we proposed to calculate \(\Omega\) with mutual information that will be introduced in section 4.2.

### Mutual Information Based \(\Omega\)

As we mentioned before, traditionally, people define the per-parameter regularization strength \(\Omega\) as the magnitude of the gradient of loss [20],

\[\Omega=\left\|\frac{\partial L}{\partial\Theta}\right\|,\]

however, for our problem, the four tasks (section 3.1) are heterogeneous because the targets (Equation (2)) of them lie in different spaces. It suggests that the corresponding losses of the four tasks will present in different scales. For example, the price regression loss may be totally different from the change-point classification loss. Even for two regression tasks, the moving average regression loss and price regression loss may lie in different metric spaces because the fluctuation of real-time price is more severe than the smooth average.

The heterogeneous setting of tasks motivates us to develop new parameter strength. In this work, we replace the gradient of the training loss with _the gradient of mutual information between the original feature and extracted feature described by the model parameters_.

Specifically, we calculate both the mutual information between the original feature \(\mathbf{X}^{k}\) of task \(k\) (\(k\in\{1,2,3,4\}\)) and the feature map \(f(\mathbf{X}^{k};\Theta)\) learned from the deep neural network. For an infinitesimal parameter perturbation on \(\Delta\Theta=\{\Delta\boldsymbol{\theta}_{p}\}\), the changes in mutual information can be approximated by

\[\text{MI}(\mathbf{X}^{k},f(\mathbf{X}^{k};\Theta+\Delta\Theta))- \text{MI}(\mathbf{X}^{k},f(\mathbf{X}^{k};\Theta)) \tag{3}\] \[\approx\sum_{m}\frac{\partial\text{MI}(\mathbf{X}^{k},f(\mathbf{ X}^{k};\Theta))}{\partial\boldsymbol{\theta}_{p}}\Delta\boldsymbol{\theta}_{p},\]

where \(\text{MI}(x,y)\) is the mutual information between \(x\) and \(y\).

From Equation (3), we observe that the contribution of the parameter \(\boldsymbol{\theta}_{p}\) to the change of mutual information can be approximated by the gradient of the mutual information with respect to \(\boldsymbol{\theta}_{p}\).

Similarly, we can calculate the mutual information between the predicted value \(g(\mathbf{Z}^{k};W)\) and ground-truth target \(\mathbf{X}^{k}\), where \(\mathbf{Z}^{k}=f(\mathbf{X}^{k};\Theta)\) is the feature map output by a deep spatial-temporal module as shown in Fig 1. Consequently, the change of the mutual information with respect to \(\Delta W=\{\Delta\boldsymbol{w}_{q}\}\) can be approximated as follows,

\[\text{MI}(g(\mathbf{Z}^{k};W+\Delta W),\mathbf{X}^{k})-\text{MI}(g (\mathbf{Z}^{k};W),\mathbf{X}^{k}) \tag{4}\] \[\approx\sum_{m}(\frac{\partial\text{MI}(g(\mathbf{Z}^{k};W), \mathbf{X}^{k})}{\partial\boldsymbol{w}_{q}})\Delta\boldsymbol{w}_{q}.\]

Figure 1: \(f(\mathbf{X};\Theta)\) is a spatial-temporal module, \(g(f;W)\) is a feed-forward neural network.
According to Equation. (3) and (4), we define the per-parameter regularization strength \(\Omega:=\{\Omega^{W},\Omega^{\Theta}\}\) as follows,

\[\Omega^{\Theta} =\bigg{\{}\Big{[}\frac{\partial[\text{MI}(\mathbf{X}^{k+1};\mathbf{ Z}^{k+1})+\text{MI}(g(\mathbf{Z}^{k+1};W);\mathbf{X}^{k+1})]}{\partial\mathbf{\theta}_{p}} \Big{]}_{ij}\bigg{\}}, \tag{5}\] \[\Omega^{W} =\bigg{\{}\Big{[}\frac{\partial\text{MI}(g(\mathbf{Z}^{k+1};W); \mathbf{X}^{k+1})}{\partial\mathbf{w}_{q}}\Big{]}_{ij}\bigg{\}}.\]

### The Calculation of \(\Omega\)

In practice, the mutual information in Equation 5 is notoriously difficult to calculate. To overcome this problem, inferring the lower bound and then maximizing the tractable objective becomes a popular choice. Oord et al.(2018) proposed the InfoNCE loss as a proxy objective to maximize the Mutual Information. According to their research framework, we assume that \(\mathbf{X}^{{}^{\prime}}\) is a different view of the input variable \(\mathbf{X}\) created through data transformation. We have

\[\max_{\Theta}\text{MI}(\mathbf{X};f(\mathbf{X};\Theta))\geq\max_{ \Theta}\text{MI}(f(\mathbf{X};\Theta);f(\mathbf{X}^{{}^{\prime}};\Theta)) \tag{6}\] \[\max_{W}\text{MI}(g(\mathbf{Z};W);\mathbf{X})\geq\max_{W}\text{ MI}(g(\mathbf{Z};W);\mathbf{X}^{{}^{\prime}})\]

According to [11],

\[\max_{\Theta}\text{MI}(f(\mathbf{X};\Theta);f(\mathbf{X}^{{}^{\prime}};\Theta ))\geq\log B+\text{InfoNCE}(\{\mathbf{X}_{i}\}_{i=1}^{B}) \tag{7}\]

\[\text{InfoNCE}(\{\mathbf{X}_{i}\}_{i=1}^{B})=\frac{1}{B}\sum_{B}^{i=1}log\frac {s(\mathbf{X}_{i},\mathbf{X}^{{}^{\prime}}_{i})}{\sum_{j=1}^{B}s(\mathbf{X}_{i },\mathbf{X}^{{}^{\prime}}_{j})} \tag{8}\]

where \(s(\mathbf{X}_{i},\mathbf{X}^{{}^{\prime}}_{j})=e^{\frac{f(\mathbf{X}_{i}; \Theta)^{T}f(\mathbf{X}^{{}^{\prime}};\Theta)}{r}}\) can be regarded as calculating the similarity of \(\mathbf{X}_{i}\) and \(\mathbf{X}^{{}^{\prime}}_{j}\) and \(r\) is the temperature. \(\{\mathbf{X}_{i}\}_{i=1}^{B}\) are samples from variable \(\mathbf{X}\) and \(B\) is the batch size. we can calculate MI(\(\mathbf{X}\); \(f(\mathbf{X};\Theta))\) with Equation 6 and 7[11]. MI\((g(\mathbf{Z};W);\mathbf{X})\) can be calculated in the same way.

## 5 Experiments

In this section, we evaluate our model on the tick-level high-frequency futures data on 4 related and heterogeneous tasks. To explore the best structure of the HCGNN model, we perform exhaustive ablation studies on the proposed model to validate and analyze the contribution of various components.

### Datasets

The data used in our experiment is the tick-level high-frequency future market series from 14 February 2022 to 28 February 2022. Including the close price (buy price), the highest price within 1 minute, and the lowest price within 1 minute. There are 388,800 sample points. Considering that there are missing data for some futures varieties and there are also some newly listed futures varieties that are not available for previous trading records, we 1) delete commodity futures that more than 50% of their log-return is zero; 2) exclude commodity futures with no trading records for the first five days or the last five days; 3) delete commodity futures with missing slot length over 15 minutes within the study period; 4) for a small number slot of missing values, we fill them with the mean. All the future series data are normalized by the Z-Score method. We finally keep 49 active commodity futures for the following analysis. We split the data into training set, validation set, and test set with a ratio of 7:2:1 in Table 1.

We make a similar experimental setting as in Cao et al.(2020). Specifically, the channel size of each graph convolution layer is set as 64 and the kernel size of 1D convolution is 3. The number of training epochs is set as 50 and the learning rate is initialized by 0.001 and decayed with a rate of 0.7 after every 5 epochs. We select mean Absolute Errors (MAE), Root Mean Square Errors (RMSE), and Mean Absolute Percentage Errors (MAPE) to evaluate the three regression tasks and report precision, recall, accuracy, and F1 to measure the ability of change-point prediction.

### Baselines Methods

The baseline methods include **StemGNN**[14], **MTGNN**[15], **STGCN**[15], **SSTGNN**[16].

### Results

#### Performance

Table 2 illustrates the comparisons of our method with the baseline methods. We see that HCGNN has an obvious advantage on modeling the high-frequency time series data. It achieve the best MAE, RMSE, and MAPE on 1 minute and 15 minutes predictions.

#### Visualizations

Figure 2 visualizes predicted prices of twelve representative future varieties. The real values and predictions are plotted in pink and blue respectively. We observe that the proposed method has strong ability of predicting future price, especially on capturing the significant events.

We also conduct change-point detection to estimate the locations of the events to provide early warning. Part of the results of the change-point detection are shown in Figure 3. The red vertical lines are real change-points and the green ones are the predictions. The line was colored in both red and blue suggesting a correct prediction. We see that the proposed approach can capture most of the important change-points.

The bar charts shown in Figure 4 visualize the dispersion degree (gap) of the price within twenty ticks (20 seconds). We compare the predicted gaps with their corresponding ground truth in 15 minutes. We see that our method can predict the

\begin{table}
\begin{tabular}{c l} \hline \hline Datasets & Period (\# samples) \\ \hline Training set & 02/14-2/25 (278479) \\ Validation set & 02/25-2/27 (79565) \\ Test set & 02/27-2/28 (39782) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Data splitting on 49 futures.
dispersion degree (gap) well, even when there are some big fluctuations at these time slots.

### Ablation Studies

To better validate the effectiveness of the proposed model, we conduct an ablation study as follows,

* **w/o GAP+CPD+MA**: HCGNN without GAP, CPD, and

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{Models} & \multicolumn{3}{c}{1min} & \multicolumn{3}{c}{15min} \\ \cline{2-7}  & MAE & RMSE & MAPE(\%) & MAE & RMSE & MAPE(\%) \\ \hline StemGNN & 17.04 & 52.35 & 10.67 & 10.64 & 32.87 & 11.04 \\ MTGNN & 84.29 & 171.09 & 1.19 & 64.58 & 126.08 & 0.77 \\ STGCN & 16.79 & 36.88 & 0.1925 & 16.940 & 37.230 & 0.1938 \\ SSTGNN & 223.80 & 389.30 & 4.34 & 68.68 & 135.75 & 0.9976 \\
**HCGN** & **6.48** & **12.40** & **0.04** & **13.70** & **42.15** & **0.1** \\ \hline \hline \end{tabular}
\end{table}
Table 2: Comparisons of the performance of close price regression between the state-of-the-art approaches and our method. We report 1-minute and 15-minute results.

Figure 4: Visualizations of gap predictions. Predicted gap and the corresponding ground truth are plotted in blue and brown bars respectively.

Figure 3: Change-point classification results. The pink curves are the price fluctuation. The change-points of ground truth and predictions are plotted in red and green dotted lines respectively.

Figure 2: Close price forecasting results. The ground truth and predictions are colored pink and blue respectively.
MA task. We train the model only on the single Forecast task.
* **w/o CPD+MA+Forecast**: HCGNN without CPD, MA, and Forecast task. We train the model only on the single GAP task.
* **w/o GAP+MA+Forecast**: HCGNN without GAP, MA, and Forecast task. We train the model only on the single CPD task.
* **w/o GAP+CPD+Forecast**: HCGNN without GAP, CPD, and Forecast task. We train the model only on the single MA task.
* **w/o GAP+CPD**: HCGNN without GAP and CPD task. We train the model on forecasting and MA tasks in continuous learning with mutual information.
* **w/o MA+CPD**: HCGNN without MA and CPD. We train the model on forecasting and gap tasks in continuous learning with mutual information.
* **w/o GAP+MA**: HCGNN without gap and MA task. We train the model on forecasting and CPD tasks in continuous learning with mutual information.
* **w/o CM**: HCGNN without MA task. We train the model on forecasting, gap, and CPD tasks in continuous learning with mutual information.
* **w/o GAP**: HCGNN without gap task. We train the model on forecasting, MA, and CPD tasks in continuous learning with mutual information.
* **w/o CPD**: HCGNN without CPD task. We train the model on forecasting, MA, and gap task in continuous learning with mutual information.
* **w/o MI**: HCGNN without mutual information. We train the model on all four tasks continuously but replace mutual information with the traditional gradient of loss.

For all the ablation experiments, we only back-propagate gradient of the activated task. Take the setting w/o GAP+CPD+MA as an example, we set the requires_gradient as True for the price **Forecast** task while restraining the gradients of GAP, CPD, and MA.

Based on the ablation experimental design approach described above, we evaluated the various variants described above on the high-frequency trading dataset and present all the experimental results in Table 3. Values in bold are the best results of these settings. We have several observations based on the results of 1-minute and 15-minute: 1) The best performance values appear in almost all the settings, it suggests that the promotion relationships among the four tasks are complex. 2) Most of the best values appear in the setting of more tasks, it suggests that the overall performance get improved when adding more tasks. Together with the results of Table 2 (last row), the best performance is achieved when four tasks are engaged simultaneously. 3) Mutual information is more appropriate to define parameter strength than the traditional loss-based methods with the heterogeneous multi-task setting.

## 6 Conclusion and Future Work

In this work, we propose a novel deep learning model based graph called Heterogeneous Continual Graph Neural Network (HCGNN). It models spatial-temporal futures data and captures the correlations between different varieties. Meanwhile, it can catch long-term and short-term trends through multi-task continuous learning. Furthermore, we creatively take use of the mutual information maximization mechanism to address the problem of CF in continuous learning. The experiment shows HCGNN outperforms existing approaches consistently in futures time-series forecasting. In future works, we will continue exploring the applicability of HCGNN in stock, bond, forex, option, and other financial markets.
