# A Natural Gas Consumption Forecasting System for Continual Learning
Scenarios based on Hoeffding Trees with Change Point Detection Mechanism

Radek Svoboda<EMAIL_ADDRESS>Sebastián Basterrech<EMAIL_ADDRESS>Jędrzej Kozal<EMAIL_ADDRESS>Jan Platoš<EMAIL_ADDRESS>Michał
Woźniak<EMAIL_ADDRESS>
###### Abstract

Forecasting natural gas consumption, considering seasonality and trends, is
crucial in planning its supply and consumption and optimizing the cost of
obtaining it, mainly by industrial entities. However, in times of threats to
its supply, it is also a critical element that guarantees the supply of this
raw material to meet individual consumers’ needs, ensuring society’s energy
security.

This article introduces a novel multistep forecasting of natural gas
consumption with change point detection integration for model collection
selection with continual learning capabilities using data stream processing.
The performance of the forecasting models based on the proposed approach is
evaluated in a complex real-world use case of natural gas consumption
forecasting. We employed Hoeffding tree predictors as forecasting models and
the Pruned Exact Linear Time (PELT) algorithm for the change point detection
procedure. The change point detection integration enables selecting a
different model collection for successive time frames. Thus, three model
collection selection procedures (with and without an error feedback loop) are
defined and evaluated for forecasting scenarios with various densities of
detected change points. These models were compared with change point agnostic
baseline approaches. Our experiments show that fewer change points result in a
lower forecasting error regardless of the model collection selection procedure
employed. Also, simpler model collection selection procedures omitting
forecasting error feedback leads to more robust forecasting models suitable
for continual learning tasks.

###### keywords:

incremental learning , time series forecasting , multivariate time series ,
change point detection , machine learning , data stream processing

††journal: Expert Systems with Applications

[label1]organization=Faculty of Electrical Engineering and Computer Science,
VŠB–Technical University of Ostrava, addressline=17. listopadu 2172/15,
city=Ostrava, postcode=70833, country=Czechia [label2]organization=Department
of Applied Mathematics and Computer Science, Technical University of Denmark,
city=Kongens Lyngby, country=Denmark [label3]organization=Department of
Systems and Computer Networks, University of Science and Technology,
city=Wroclaw, country=Poland

##  1 Introduction

As a result of ongoing trends, the energy industry is experiencing a growing
demand for consumption forecasting tools. In the past, these tools focused
mainly on electricity load forecasting, as they presented complex and costly
storage challenges. Consequently, the development of similar tools for other
areas was postponed. However, the importance of natural gas is now on the
rise, driven by the increasing environmental initiatives undertaken by
governments and private companies. Despite this, effective logistics and
delivery planning in the natural gas sector presents significant challenges.

A primary driver for energy consumption forecasting stems from the prevalent
use of take-or-pay contracts and clauses in commodity trading [37, 20, 38].
These contractual agreements require customers to pay for the specified
quantity, regardless of actual usage. The financial consequences of such
agreements underscore the importance of improving the quality of the
forecasting models. Clearly, the vital landscape of energy consumption shows a
need for more effective forecasting models with continual adaptation
capabilities. This adaptability ensures that the models remain relevant and
effective in capturing the nuances of changing consumption patterns, market
dynamics, and external influences. This responsiveness to new data improves
the overall reliability of the system. Creating a more resilient framework
capable of addressing the challenges inherent in the energy sector. Therefore,
recognizing the pivotal role of continual model adaptation is crucial for
stakeholders navigating the intricacies of energy consumption forecasting in
the context of take-or-pay contracts.

Change point detection techniques are useful for understanding and predicting
patterns in time series forecasting. A change point event refers to a specific
moment in a sequence where the underlying behavior or trend shifts abruptly.
Various factors can cause these shifts, such as policy changes, economic
events, technological advancements, or changes in consumer behavior.
Identifying change points is crucial to developing an accurate and robust
forecasting model. Change points can significantly impact the underlying
distribution of time series. In the particular case of energy consumption, it
can cause modifications in the near-term energy demand. Furthermore,
understanding change points and seasonality in time series forecasting enables
energy industry stakeholders to make more informed decisions. By identifying
abrupt changes in consumption behavior, energy providers can anticipate shifts
in energy demand, adjust their strategies accordingly, and improve energy grid
management.

Recently, Continual Learning (CL) holds great importance in the field of time
series forecasting. With technological advances and the availability of large-
scale datasets, CL allows forecasting methods to incorporate advanced
approaches and continuously integrate new information, enhancing their
forecasting capabilities. The ability to adapt and learn from new data and
methodologies ensures that forecasting tools remain reliable and effective in
meeting the evolving demands of the energy industry. By continually updating
and refining predictive models in the context of CL, predictive tools can
capture distribution changes in input patterns and provide different weights
to the latest data, enabling more accurate predictions of energy demand.

The main contributions of the work are listed below.

  * (i)

We introduce a general framework for multistep forecasting of multivariate
time series as CL task. The innovative framework is composed of several
hierarchical independent modules, such as dataset preprocessing, change point
detection, model collection setup, model collection selection schema, and
model training, which are fully described in the paper. The modular
architecture enhances flexibility and makes it easy to exchange specific
modules if the characteristics of the problem change.

  * (ii)

We study the performance of the framework in a wide range of scenarios. We
investigate several approaches for forecasting natural gas consumption, using
the integration of the detection of change points with the selection approach
of model collection (with or without an error feedback loop).

  * (iii)

We present a sensitive analysis of the integration of the PELT algorithm into
the CL system. We also discuss the threat to validity of our research. The
experimental analysis includes a comparison with two baseline approaches that
do not consider change point detection. Furthermore, we study the impact of
number of detected change points on the overall predictive performance. The
evaluation was carried out on a recently collected real-world dataset from the
natural gas domain. We evaluated the developed system using three standard
metrics (MSE, MAE and SMAPE) to better capture the time series behavior.

##  2 Related works

In this section, we focus on four pivotal topics central to our research. Our
primary emphasis lies in forecasting natural gas consumption, given its vital
role in energy planning and trading. Additionally, we delve into the
literature of the CL paradigm in the area of energy consumption forecasting.
The Hoeffding Decision Tree is explored for its algorithmic relevance, and we
review the literature of Change Point Detection methods for identifying abrupt
shifts in the time series data. By synthesizing insights from these areas, our
study aims to build a solid foundation for improving the predictive
capabilities of natural gas consumption models.

###  2.1 Natural gas consumption forecasting

Statistical methods have gained popularity in forecasting gas consumption
during the late 90s [7]. A nonlinear regression model was applied to model the
gas consumption behavior of individual and small commercial customers [59].
Due to the success of Neural Networks (NNs) for solving machine learning
problems, several works also applied their potential. Oil market data and
weather information were studied to create a forecasting model employing NNs
for Belgian gas consumption [50]. Other families of NNs have also been
studied, a two-stage model that combined feedforward and functional link NNs
[30], while a hybrid of traditional networks and fuzzy NNs was developed [58].
Hourly sampled data was analyzed using linear methods, support vector
machines, and forward NNs [48]. The authors showed how different time windows
of the day present different consumption trends. Another learning application
was over the gas consumption in Turkish regions [55, 11]. The SARIMAX model,
multilayer perceptron ANN, and RBF models were evaluated for the prediction of
daily gas consumption [55]. The impact of including exogenous variables in
specific forecasting models for the gas consumption was also investigated,
especially the relevance of the air temperature feature [55].

Multilayer feedforward neural network also showed to be effective as
forecasting method of hourly gas consumption when the input layer has included
features with weather data information [52]. However, the relationship between
external variables and gas consumption was not always evident and direct.
Several studies considered the thermal memory capacity of the buildings. For
example, air temperature and solar radiation may have some delays in the
impacts of gas consumption [48, 43]. Forecasting based on the LSTM model and
evolutionary algorithms for hyperparameter optimization were also studied
[49]. Although NNs have been widely applied to solve the forecasting problem,
other tools have also reported good results. Support vector machines were also
employed with structure calibration [6]. According to their results, kernel-
based methods outperformed classic backpropagation models in predicting daily
gas consumption. Two reviews of the literature on the area of natural gas
forecasting are available in [47, 53].

###  2.2 Continual Learning

Despite recent advances in the machine learning domain and the large number of
applications, most learning tools are designed to model stationary learning
data. The CL paradigm has been introduced to overcome some classic machine
learning limitations. A CL approach defines an algorithm that can learn in
incremental mode, which is suitable for not stationary data modeling [18]. The
greatest challenge in such a setting is overcoming catastrophic forgetting
[23] – a tendency for NNs to quickly forget what they have learned from
previously seen data, especially when those data are no longer available. The
optimal solution for the CL problem was shown to be NP-hard [33], and many
heuristics have been proposed so far. Most of the current CL methods are based
on rehearsal [39]. In such methods, historical samples are stored in a small
buffer and could be used to prevent forgetting when training with new data.
Rehearsal algorithms can be quickly adapted to the online setting by
utilization of reservoir sampling [17]. In [4], instead of randomly selecting
the learning examples from the rehearsal buffer when learning a new task,
samples were selected based on the loss change when performing network
updates. The authors of [15] observed a shift of old data representations in
the network feature space during training with new learning examples. They
also introduced a method to correct this problem. The work of [62] provided
theoretical insight into the risk of overfitting when using a small memory
buffer. Dark experience replay [14] extended the standard rehearsal to save
not only historical samples, but also logits with predictions that are later
used for knowledge distillation. This method was later improved [12] by
introducing bias correction, adding pre-training for future tasks, and
updating existing logits in memory. Other approaches to CL can include the
inclusion of additional regularization terms [32, 61], or the extension of the
neural network architecture for new tasks [44, 22]. The area is advancing
fast, and CL was also previously applied to solve time series problems. In
[25] a new method was introduced for multi-sensor time series based on task-
specific generative models and classifiers. The authors of [24] proposed a new
method based on Variational Auto Encoders and Dilated Convolutional NNs aimed
at multivariate time series anomaly detection problems. The work of [19]
studied CL with recurrent NNs by empirically evaluating existing algorithms
and introducing new benchmarks for CL with sequential data.

###  2.3 The Hoeffding decision tree

Standard tree-based classification or regression algorithms are not well
suited for online learning [13, 41, 42]. They learn by splitting the data into
separate manifolds that contain either homogenous samples from a single class
or low target variable variance. For this reason, to update the structure of
the tree is not as easy as in the case of NNs. Domingos and Hulten introduced
a new method that allows tree structure modification [21]. It uses the
Hoeffding bound to calculate the number of samples in the node necessary to
decide with a high probability that the best attribute provides a better split
than the second-best attribute. They also showed that trees optimized using an
online schema are asymptotically similar to trees learned offline using the
whole data. Another tree variation introduced fast splits with non-zero
information gain [36]. Forecasting capacity of Streaming Random Forest with
Hoeffding trees as base models was also investigated [2]. Ensemble methods
using Hoeffding trees were also investigated [9]. A parallel version of a
Hoeffding tree was also developed, it computes information gain on multiple
computational nodes decreasing the overall computing time [16].

###  2.4 Change point detection

Let $(X_{t})_{t\geq 0}$ be a time series of independent and identically
distributed (i.i.d.) observations. In a non-stationary environment, it may
occur that the probability distribution of $X$ changes over time, then there
exists a point $t\in\\{0,1,...\\}$ such that the underlying distribution of
$\\{\ldots,X_{t-2},X_{t-1},X_{t}\\}$ is different from the distribution of
$\\{X_{t+\Delta},X_{t+\Delta+1},\ldots\\}$, for a $\Delta>0$. A change point
detection task is based on minimizing a cost function that should evaluate the
homogeneity of the found segments, and they are usually linked to parametric
probabilistic models (e.g., using maximum likelihood method) or non-parametric
approaches (as rank-based algorithms) [35]. Another critical component is a
search method that could return optimal or approximate solutions [57]. Some
algorithms also employ penalty terms, which are responsible for minimizing
overfitting [26]. Older methods that require knowing the number of change
points may not be practical in real-world applications. Multiple approaches
have been proposed to solve the problem of detecting change points [5].
However, the most well-known method seems to be the Pruned Exact Linear Time
(PELT) [31] algorithm. That it can be seen as an improvement of the Optimal
Partitioning method [29]. It minimizes the cost function:

$\sum_{i=1}^{M+1}[C(y_{\tau_{i-1}+1})+\beta],$ (1)

where $\tau$ is a change point and $M$ is the total number of change points,
$\beta$ is a penalization parameter.

The partitioning procedure uses a pruning method that improves computational
efficiency by reducing the number of change points managed in a single
iteration. There are a plethora of other change point detectors that have been
proposed so far such as At Most Single Changepoint (AMOC), Binary Segmentation
algorithm [45], or Bayesian Online Changepoint Detection [3].

Interest in these methods is not waning, and attempts are being made to
develop change point detection methods for increasingly complex problems,
including taking into account not only the non-stationarity of time series but
also the fact that nowadays, more and more tasks could be multidimensional
[5]. An interesting proposal is the WATCH algorithm that allows change point
detection for multidimensional time series [1]. The solution is based on the
analysis of distances between distributions using the Wasserstein metric.
Also, interesting proposals using specific penalty functions may be found.
Several works provide a recipe for estimating an interval with a useful
penalty value – see, for example, the CROPS algorithm [26] or the ALPIN scheme
for optimal penalty setting [57].

##  3 Proposed methodology

Existing research in the energy consumption forecasting in general is
predominantly focused on static or non-adaptive models, overlooking the
importance of continual adaptation [56, 28, 46]. Our methodology addresses
this gap by emphasizing the challenges and methodologies associated with the
continual improvements of forecasting models in the context of energy
consumption. Moreover, we put an emphasis on the integration of change point
detection methods. Recognizing that energy consumption is characterized by
sudden changes throughout the year, we highlight the critical role of change
point detection in enhancing the adaptability of forecasting models. By
incorporating change point detection methods, our approach goes beyond
traditional static models, allowing identification of significant shifts in
consumption patterns.

A general methodology for multistep forecasting of multivariate time series
with CL capabilities employing time series in the form of data stream
processing is depicted in Fig. 1. The methodology consists of a modular schema
which forms the following pipeline:

Change point detection procedure. This step is optional and is used in the
Multi Collection scenario. This step aims to divide the time series into
several segments. Each segment should have different statistical properties
(e.g., trend direction, variance, or mean). Thus, each segment will use the
different model collection for forecasting. The purpose of diving the time
series into segments is to simplify the forecasting task, as each model
collection should be trained on data with similar properties. Change points
can be set manually by knowledge regarding the domain (e.g., based on
knowledge about the heating season in the natural gas consumption case) or by
a change point detection algorithm such as PELT or Binary segmentation. The
detected change point locations in a chosen period (e.g., the first year of
the data) are then re-used in subsequent periods.

Model collection setup. The defined approaches may utilize one or more
forecasting models. The forecasting model, in general, is a mathematical
representation designed to make predictions about future values in a short,
medium, or long-term time horizon based on historical data and patterns, for
example, to forecast $n$ hours ahead of gas consumption or $m$ days ahead of
electricity load. The first one is a Single Model Collection approach, which
does not divide the data into segments but uses only one model collection for
the whole dataset. The model collection is an ordered set of partial models
which provides forecasts for the defined forecast horizon (for the number of
steps after the forecast origin). The number of models ($m$) in the collection
is equal to the length of the forecast horizon. The reason for using direct
forecasting with multiple models instead of the cumulative approach, which
reuses the forecasts from the previous timestep in predicting the next value,
is to prevent the accumulation of errors through the forecast horizon. The
second approach uses Multi Model Collections. The number of collections ($k$)
depends on the number of segments into which the time series was divided, thus
each segment has its own assigned model collection, i.e. if we divide the time
series into $k$ segments, $k$ model collections will be used, one for each
segment. The rationale behind this approach is that there should be a benefit
in having different strategies for distinct parts of the timeseries as each
segment might have different properties. The number of partial forecasting
models is the same as in the Single Model Collection approach. The specific
model used is a decision made during the application of the methodology. The
model has to support incremental learning, so models such as Hoeffding Tree or
Stochastic Gradient Tree can be used for this task.

Model collection selection procedure. After the model collections are set, the
only step left before the training phase is a logic for model collection
selection. The simplest case is a Single Model Collection approach as it uses
only one collection, and thus this step could be skipped. For Multi-Model
Collections, there are two approaches for the collection selection, also
called model aggregation scheme. In simple terms, the model aggregation scheme
is the plan for deciding which set of prediction models to use based on the
changes observed in the time series data. The first one, called the Pure
Change Point-Divided schema, selects the model collection based on the current
time series segment (segments are divided by change points from the Change
point detection step). When a change point occurs, the current model
collection is exchanged for the subsequent one. The rationale behind this
schema is that when the time-series distribution shifts, then it is beneficial
to switch to a different group of forecasting models. The second approach,
called Mixed Change Point-Divided schema, is based on the hypothesis that the
change points do not occur at the exact same location in every period, but the
location varies in some close subsegments. Thus, this approach works with a
boundary of $b$ steps, which defines a segment of length $2b$ ($b$ step prior
to the change point and $b$ steps after it). Both model collections (before
and after the change point) are used inside this boundary. They are trained on
the new instances and produce forecasts. A forecast error measure $E$ is
computed using an arbitrarily selected metric (e.g. MAE, MSE, etc.) for both
model collections. Error calculation is used at timestep $t+1$ for the final
forecast construction, and this process can be performed in one of two ways:

  1. 1.

WAVG procedure: Let $R_{WA}$ be a collection of $m$ forecasting models
generated by the WAVG procedure
($R^{(t)}_{WA}=(r^{(t)}_{1},r^{(t)}_{2},...,r^{(t)}_{m})$). Furthermore, we
denote by $C_{1}$ and $C_{2}$ two disjoint collections of partial models
$R^{(t)}_{C_{1}}$ and $R^{(t)}_{C_{2}}$ before and after a specific change
point. The forecast value of model $r_{i}$ at timestep $t$ ($f_{i}^{(t)}$) is
calculated as a weighted average of the forecasts produced by the following
two model collections $R^{(t)}_{C_{1}}$ and $R^{(t)}_{C_{2}}$:

$f^{(t)}_{i}=\frac{w_{C_{1}}f^{(t)}_{i,C_{1}}+w_{C_{2}}f^{(t)}_{i,C_{2}}}{w_{C_{1}}+w_{C_{2}}},$
(2)

where the weights $w_{C_{1}}$ and $w_{C_{2}}$ are defined according to the
previous performances (errors $E^{(t-1)}_{C_{1}}$ and $E^{(t-1)}_{C_{2}}$):

$\displaystyle w_{C_{1}}$
$\displaystyle=1-\frac{E^{(t-1)}_{C_{1}}}{E^{(t-1)}_{C_{1}}+E^{(t-1)}_{C_{2}}},$
$\displaystyle w_{C_{2}}$
$\displaystyle=1-\frac{E^{(t-1)}_{C_{2}}}{E^{(t-1)}_{C_{1}}+E^{(t-1)}_{C_{2}}}.$

  2. 2.

SWITCH procedure: Let $R_{SW}$ be a collection of $m$ forecasting models
generated by the SWITCH procedure
($R^{(t)}_{SW}=(r^{(t)}_{1},r^{(t)}_{2},...,r^{(t)}_{m})$). The forecast at
the timestep $t$ is also based on the forecasts produced by the two model
collections, as in the WAVG procedure case. However, in this procedure, only
the forecast at the timestep $t$ by the model collection with lower $E$ at the
timestep $t-1$ is used, thus the models switch during the boundary period. The
forecast value of model $r_{i}$ ($f_{i}^{(t)}$) at the timestep $t$ is
produced by the partial model of the selected collection ${C_{1}}$ or
${C_{2}}$:

$f^{(t)}_{i}=\begin{cases}f^{(t)}_{i,C_{1}},\>\>\text{if
}E^{(t-1)}_{C_{1}}<E^{(t-1)}_{C_{2}},\\\
f^{(t)}_{i,C_{2}},\>\>\text{otherwise.}\end{cases}$ (3)

In both cases, these procedures help to decide how to select different
prediction models based on their past performance around detected change
points in the data. WAVG combines predictions from different models with
weighted averages, giving more influence to historically accurate models.
SWITCH, on the other hand, selects predictions from one model based on its
past performance, adapting to changing circumstances in the data. After the
boundary sub-segment ends, only a single model collection is used, and its
choice is the same as in the Pure Change Point-Divided schema case.

Model training. When the model collections are set up and the selection schema
is selected, the model collections could be used in a training phase. The
methodology is meant for data stream incremental learning; thus, the models
are trained after each data instance. Before training, the models produce the
forecasted values and error metrics are computed and used if the Mixed Change
Point-Divided schema is used.

Figure 1: The proposed multistep forecasting pipeline with CL support.

##  4 Experiments

The conducted experiments aim to verify the accuracy and efficiency of the
proposed method on real-world data and evaluate the defined model collection
selection schemas as beneficial over naive baseline models. We intend to
answer the following research questions:

  * •

RQ1: How does the performance of a single-model continuous learning approach
(SMCA) compare to a multimodel approach aggregated by year quarter (QDMDC) in
the context of real-world data?

  * •

RQ2: May we benefit from the model aggregation scheme based on change points
detected by the PELT algorithm compared to the quarter aggregation?

  * •

RQ3: Does the number of detected change points affect the accuracy of the
forecast?

  * •

RQ4: Is there an advantage in employing forecast ensembling (MCPDMC-WA) or
model switching (MCPDMC-SW) in proximity to change points, with the main
consideration being the Symmetric Mean Absolute Percentage Error (SMAPE)
metric?

###  4.1 Experiment setup

Dataset. The data covers eight years, from January 1, 2013, to December 31,
2020, with data available at an hourly frequency [51]111The dataset is
available online: https://ai.vsb.cz/natural-gas-forecasting. It consists of
70,104 data points, compiled from three main components. The first component
comprises consumption data, focusing on Prague, the capital of the Czech
Republic. The Prague distribution network served 422,926 customers in 2018,
with a total consumption of 3.82 billion m3. The second component incorporates
weather variables obtained from the Prague LKPR airport weather station. These
variables are derived from periodic METAR reports issued by airports and are
archived for long-term preservation. The third component that represents
economic characteristics is natural gas price data. We have obtained price
data from the Czech Energy Regulation Office and included them in the dataset.
The natural gas consumption time series is shown in Fig. 2, as well as the
temperature (Fig. 3), which is the most important exogenous variable, since
gas consumption is highly dependent on the outside temperature because of the
household heating.

Figure 2: Natural gas consumption (endogenous variable) from January 1, 2013,
to December 31, 2020 (years are divided by colors). Figure 3: Temperature
(exogenous variable with the strongest correlation coefficient ($-0.871$) to
the natural gas consumption) from January 1, 2013, to December 31, 2020 (years
are divided by colors). Table 1: Description of the features included in the
natural gas dataset.

Feature name Description Measuring unit Year 2013-2018 - Month 1-12 - Day 1-31
- Hour 0-23 - Day of week 1-7 - Before holiday Is the next day a holiday? -
Holiday Is the current day a holiday? - Temperature Air temperature at a
2-meter height $\mathrm{\SIUnitSymbolCelsius}$ above the earth’s surface
Pressure Atmospheric pressure at weather station level $\mathrm{mm}$ Hg
Pressure (sea level) Atmospheric pressure reduced to mean sea level mm Hg
Humidity Relative humidity at height of 2 meters above the % earth’s surface
Wind direction Mean wind direction at a height of $10-12$ meters compass
points above the earth’s surface over a 10-min period Wind speed Mean wind
speed at a height of $10-12$ meters m/s above the earth’s surface over a
10-min period Phenomena Special present weather phenomena observed at Text
value or near the aerodrome Recent phenomena Recent weather phenomena of
operational Text value significance Visibility Horizontal visibility km
Dewpoint Dewpoint temperature at a height of 2 meters
$\mathrm{\SIUnitSymbolCelsius}$ above the earth’s surface Cloud cover Total
cloud cover Text value Price Weighed average EUR / MWh Consumption Consumption
of the distribution network m3/h for the current hour Temperature YRNO
Forecasted temperature $\mathrm{\SIUnitSymbolCelsius}$

Natural gas consumption over multiple years exhibits a clear cyclical
phenomenon, as shown in Fig. 2. Cycles reflect seasonal variation of the
outside temperature, affecting heating and cooling demand. Fig. 3 illustrates
the relationship between consumption and temperature, indicating that
consumption is high during the cold and low during the warm months. The
consumption also displays short-term trends corresponding to transition
periods between the heating and summer seasons, when the temperature
fluctuates rapidly.

((a)) Change points detected in 2013 data.

((b)) Change points detected in 2015 data.

((c)) Change points detected in 2018 data.

Figure 4: Detected change points by PELT algorithm with Low settings in
selected years.

Fig. 4 shows the change points detected by the PELT algorithm (Low settings)
for three selected years. The detected change points partition the data into
segments with distinct levels or trends of gas consumption. The segments
overlap partially, with better agreement during the summer and fall months,
when consumption increases. The spring months have more frequent levels and
trend changes, resulting in lower segment overlap.

Dataset preprocessing. The original data is transformed into a suitable form
for machine learning algorithms and divided into sets of pairs of input and
output vectors. An input-output tuple is created for each forecast origin
(i.e., a point in the time series from which the forecast is done, for
example, midnight). The input vector includes past data up to the forecast
origin, and the output vector includes natural gas consumption values for each
hour of the following day. The input vector consists of the so-called lagged
features - values obtained prior to the forecast origin. In our experiments,
these are Consumption (endogenous variable) and Temperature, Price, Wind
Speed, and Humidity (exogenous variables) of the original dataset. We used a
fixed window of 72 hours for these lagged variables. On top of that, the input
vector contains the Temperature Forecast 222Forecasted by YR.no
(https://www.yr.no) for the next 24 hours and usual date time variables such
as Day of week, Month or if the day is a holiday. The forecast origin is at
midnight, the forecast horizon is 24 hours long; thus, the output vector
consists of the Consumption variable values for the next 24 hours, and each
forecasted value corresponds to one hour of the next day.

Algorithms used in the forecasting pipeline. The methodology proposed in
Section 3 is generally defined without the need for specific change point
detection and forecasting algorithms. For the experiments, we have used a PELT
algorithm for the change point detection phase, as it is probably the most
known state-of-the-art algorithm in this domain (see 2 for more details) and
Hoeffding Tree as a forecasting model due to the fact that it supports
incremental learning out of the box and tree-based algorithms performed very
well on the dataset used in classical offline learning scenarios [51]. The
Hoeffding Tree Regressor with $7$ instances (one week of the input data) a
leaf should observe between split attempts (the so-called grace period), decay
of the model selector of $0.2$ and threshold ($\tau$) below which a split is
forced to break ties of $0.5$ is used as a forecasting model. The parameter
values are based on the hyper-parameter tuning procedure employing the first
year of the data. As the defined multistep forecasting task requires multiple
output values, the multimodel direct forecasting approach is used. There is a
sequence of 24 models; each value of the output vector is forecasted by its
corresponding model. PELT algorithm with $L2$ segment model, $168$ hours of
minimum segment length, and a subsample rate of $24$ hours is used for the
change point detection phase of the process.

Experimental protocol. Because we employ data stream processing, thus only the
current data instance is available to the model at each training step. Models
are trained incrementally daily. The interleaved-test-then-train evaluation
[34] was used in all experiments. It works with a data stream of instances,
each instance is first used for the inference (testing phase), then it is used
for the model training. allowing for the gradual refinement of model accuracy.
This strategy allows a gradual refinement model’s accuracy. Following this
sequential evaluation process, the model consistently faces unseen samples.
This approach offers the benefit of eliminating the need for a separate
holdout set for testing, ensuring optimal utilization of the available data
[10].

The following experiments were carried out:

  * •

Single Model Collection Approach (Baseline). Only one model collection is used
for the whole data stream. The model collection consists of 24 forecasting
models, each model being used for one hour of the day.

  * •

Quarter-Divided Model Collection. Each quarter of the year has its model
collection. The collection choice depends on the current quarter of the year.

  * •

Pure Change Point-Divided Model Collection. The year is divided into multiple
segments by the change points in gas consumption detected by the PELT
algorithm. Each of the segments has its own collection of models. The change
points are detected using only the first year of the data (2013), and the same
change point locations are reused in the next years. When the change point
occurs, the model collections are switched.

  * •

Mixed Change Point-Divided Model Collection. The approach is the same as the
pure one at its core; however, there is a 7-day period before and after each
change point. Two strategies for forecast ensembling in these periods are
tested. During these 14-day long periods, both model collections are trained
on the data. The first strategy uses weighted forecast averaging. The Mean
Absolute Error (MAE) is computed for both models, and it is used to calculate
the weighted average of the two models forecast in the next forecast step. The
model with a higher error in the previous step has a lower weight in the
average and vice versa. The second strategy uses model switching. The MAE is
computed for both models as well, but only the forecast produced by the model
with the lower error in the previous step is used. Outside these periods, the
process works in the same way as in the Pure Change Point-Divided Model
Collection case.

The Pure and Mixed Change Point-Divided Model Collection experiments were
carried out with three different PELT algorithm settings, which detected 4
(Low settings), 7 (Medium settings), and 13 (High settings) change points; to
be able to assess the effect of increasing the number of detected change
points (i.e. number of model collections) on the forecast error. The
experimental setting that we designed for validating the proposed methodology
is depicted in Figure 5.

Figure 5: Designed experimental pipeline overview with high level of
abstraction.

Result evaluation. Let $G_{t}$ be the ground truth of natural gas consumption
and the variable $F_{t}$ the forecast value, both at time step $t$. We have
used the standard set of commonly used error metrics, each summarizing
different information, [40], MAE, MSE and SMAPE defined in expression (4), (5)
and (6), respectively.

$MAE=\frac{\sum_{t=1}^{n}|G_{t}-F_{t}|}{n},$ (4)
$MSE=\frac{\sum_{t=1}^{n}(G_{t}-F_{t})^{2}}{n},$ (5)
$SMAPE=\frac{100}{n}\sum_{t=1}^{n}\frac{\mid
G_{t}-F_{t}\mid}{\frac{1}{2}\left(\mid G_{t}\mid+\mid F_{t}\mid\right)}.$ (6)

The results are evaluated for the years 2014 to 2020. The first year 2013 was
intentionally left out of the evaluation process because the data were used in
the change point detection phase for the Pure and Mixed Change Point-Divided
Model Collection experiments; thus, it could be taken as an information leak
and this phenomenon would distort the comparison with other approaches in this
year.

Implementation and reproducibility. The experiments were carried out in Python
(v. 3.11) using the Ruptures library (v. 1.1.7) for the PELT algorithm and the
River library (v. 0.15.0) for the Hoeffding Tree Regressor. Pandas (v. 1.5.3)
and Numpy (v. 1.24.2) libraries were utilized for dataset manipulation and
preprocessing. The codebase with the method and experiment implementation is
available in the git repository333https://github.com/rasvob/Hoeffding-Trees-
with-CPD-Multistep-Forecasing

##  5 Results

Two baseline and three models based on the proposed methodology were
experimentally evaluated. Both baseline models omit the change point detection
integration, as the first uses only one model collection for the data stream,
and the second aggregates the model collections using year quarters. Models
based on the proposed methodology were evaluated using three levels (low,
medium, and high) of change point density. For each model, a Mean Absolute
Error (MAE), Mean Square Error (MSE), and Symmetric Mean Absolute Percentage
Error (SMAPE) were calculated. The aggregated results are summarized in Table
3 and Figure 6, a detailed view of the results that provides insight into the
model accuracy differences is included in Figure 7 and Table 2.

###  5.1 Threats to validity

In the realm of empirical research in the domain of natural gas consumption
forecasting, the identification and mitigation of validity threats play a
crucial role in ensuring the reliability and applicability of predictive
models. Our goal is to create accurate forecasts capable of informing real-
world decision making, and it becomes imperative to recognize potential
challenges that could compromise the validity of our results. The thoughtful
consideration of potential threats becomes the cornerstone of building robust,
accurate, and actionable forecasting systems. We explore specific threats to
validity, both internal and external, in the context of our forecasting task
and outline strategies to mitigate these challenges. We consider the following
threats to be the most severe in terms of our research.

####  5.1.1 Data selection bias

Connection to the research. Data selection bias occurs when data collection is
collected from a biased and non-representative population. In our context,
data selection bias could manifest itself if the data set predominantly
represents certain seasons, leading to skewed patterns in natural gas
consumption. For example, if the data set is predominantly from colder
seasons, the model may not generalize well to warmer seasons, affecting the
reliability of the forecasting system.

Mitigation strategy. To address data selection bias, we ensured that the
seasons are equally represented; the same applies to any medium- or long-term
trend, using a dataset that contains data from 2013 to 2020. This approach
aims to enhance the model’s adaptability to various consumption patterns and
ensuring better generalization across different scenarios.

####  5.1.2 Methodological bias

Connection to the research. Methodological bias refers to bias introduced by
researchers in the methodological design of the study. Both machine learning
and change point detection algorithms may be sensitive to hyper-parameter
settings. The selection of the number of used change points and the way to
deal with them may introduce potential sources of methodological bias because
it may severely affect the results obtained. Moreover, from the point of view
of the experiment design, there is a risk of selection bias in the evaluation
metrics and selection of the train-test set as well.

Mitigation strategy. Sensitivity analyzes were performed to evaluate the
impact of various methodological choices on the performance of the model.
Several different configurations for the number of change points used were
tested and multiple approaches to deal with them were used during the
experiments. The results were analyzed in detail and the different settings
and approaches were compared with each other and also with respect to
different time periods of the year. Cherrypicking evaluation metrics that
highlight the model’s strengths while downplaying its weaknesses was avoided
by reporting multiple widely used error metrics, such as MAE, MSE, or SMAPE.
The interleaved test-then-train protocol was used during the training and
evaluation of the models to assess generalization performance. This ensures
that the model performance is not overly tailored to the peculiarities of the
selected parts of the data. This approach was designed to minimize
methodological bias and ensure that the methodologies chosen produced robust
and reliable results.

####  5.1.3 Confounding features threat

Connection to the research. Confounding features threat arise when external
factors (exogenous variables) that could affect predictions are not considered
in the methodological design. External factors, such as weather, especially
temperature and forecast for the next day, or date-time features can influence
natural gas consumption. Failure to account for these confounding features and
rely solely on the endogenous variable during the methodological design could
lead to inaccurate predictions.

Mitigation strategy. To address the threat of confounding features, we have
conducted extensive research in this domain in our work [51] to identify
potential external factors that impact gas consumption. Relevant exogenous
variables were incorporated into the model.

###  5.2 Methodology and results in context

As it was described in Section 2, many existing studies are based on
conventional datasets [55, 54]. In contrast, the proposed research uses a
unique and underexplored real-world dataset [51] in a CL context, introducing
a new perspective to the field of natural gas forecasting. Although the
scarcity of prior research on this specific dataset contributes to the
originality of our work, it also poses distinct challenges in the comparison
of related work results from the forecast accuracy point of view. Due to this
fact, we have decided to put our methodology and experiment results in
contrast with other related methods from the perspective of model
adaptability, feature engineering, approach to analysis of results, and model
accuracy.

####  5.2.1 Forecasting model adaptability

Research context. The prevailing trend in related research within the realm of
natural gas consumption, or energies in general, forecasting predominantly
focuses on static models with limited adaptation capabilities [55]. Moreover,
these models are typically used for single-point forecasts; thus, they provide
forecasts as scalars for a single point in time [8].

Proposed methods contrasts. Unlike related approaches that focus on static
models and offer forecasts for a single point in time, our research takes a
different approach. We use CL principles by treating natural gas consumption
as an ongoing data stream. This means that the proposed models learn and adapt
to new data in real time. Note that the proposed method is focused on vector
forecasts with a forecast horizon beyond just one time unit ahead, thus it is
capable of predicting the whole forecast horizon at once; we have focused on a
24 hours long forecast horizon in our experiments as this is the common
forecast horizon length in this domain.

####  5.2.2 Feature selection possibilities

Research context. Related research primarily centers on leveraging statistical
methods and neural networks for natural gas consumption forecasting. However,
we observe that statistical models often rely on univariate time series, such
as Autoregressive integrated moving average (ARIMA) [27], which limits their
ability to capture intricate relationships within multivariate data.
Similarly, neural networks, while powerful, present limitations for modeling
time series in contexts when the model should adapt according to changes of
the input data distribution. Typically, longer input sequences are required,
offer limited opportunities for feature engineering through domain knowledge,
and generally focus on offline learning approaches [60]; therefore, models
often rely on the availability of the entire dataset during training. In
particular, conventional practice in the energy domain involves implementing a
single forecasting model for the entire dataset.

Proposed methods contrasts. In contrast to the prevalent emphasis on
statistical methods and neural networks in natural gas consumption
forecasting, our research prioritize tree-based models for vector forecasts.
Although these models are conventionally designed for scalar output, the
methodology proposes a model collection setup which enables the models to be
used for tasks which require vector output. Unlike statistical models and
neural networks, tree-based models present robust opportunities for feature
engineering, allowing us to make use of multivariate data more effectively. A
previous study showed that tree-based models achieve better accuracy compared
to statistical methods [51], however, it is important to note that the
research was focused on feature engineering in an offline learning context.
The approach presented in this article capitalizes on the recurrent behavior
presented in the consumption of natural gas.

####  5.2.3 Results analysis

Research context. The prevailing trend in related research predominantly
centers around the analysis of overall forecasting accuracy, where models are
typically trained only once. Although this approach provides valuable insights
into the general performance of forecasting models, it often overlooks time-
related patterns within the time series data. Additionally, results analysis
in the field tends to focus on classical properties, examining only
statistical aspects of error distribution throughout the day of the week or
month, and these analyses often operate under the assumption of a static
forecasting model.

Proposed methods contrasts. While we cover the standard analyses like overall
forecasting accuracy and classical error distribution, our research stands out
by giving a strong focus to how our models learn online. We examine how the
accuracy of the forecast changes over time, examining how the number of
detected change points impacts the predictions. We are especially interested
in how our forecasts differ in detail during periods near these change points.
This approach allows us to understand how well our models adapt to the
evolving nature of natural gas consumption patterns.

####  5.2.4 Forecast accuracy

Research context. Given the rather novel dataset used and the prevailing focus
of related energy forecasting research on offline learning and single-value-
ahead forecasts, direct numerical comparisons with existing results could be
misleading. We acknowledge the difficulty in numerical comparisons and have
implemented specific measures to address this concern, ensuring the robustness
of our results. It is important to note that from our previous research [51],
we can assert that the Symmetric Mean Absolute Percentage Error (SMAPE) for
24-hour ahead forecasts for this dataset generally falls within the range of
$7\%$ to $15\%$ for models trained offline. This variation depends on the
method employed and its complexity.

Proposed methods contrasts. To navigate the complexities arising from the
novelty of our dataset and the lack of continual/online learning research
within this domain, we opted to establish two baseline models, SMCA and QDMDC.
By defining these baselines ourselves, we ensured a reliable benchmark for
comparison against more intricate approaches. In particular, there has been no
prior research specifically focusing on continual or online learning for this
dataset. Using proposed approaches that incorporate change point detection, we
successfully reduced the Symmetric Mean Absolute Percentage Error (SMAPE) by
approximately 5 to 6.5 % compared to the baseline models. Importantly, this
improvement was achieved with minimal additional computational overhead, which
is particularly evident when compared to the QDMDC approach. This nuanced
comparative analysis underscores the effectiveness of our methodology in
enhancing forecast accuracy while maintaining computational efficiency.

##  6 Lessons learned

We have grouped conclusions according to the research questions formulated in
Section 4 to facilitate a systematic evaluation of the results obtained.

  * •

RQ1: How does the performance of a single-model continuous learning approach
(SMCA) compare to a multimodel approach aggregated by year quarter (QDMDC) in
the context of real-world data?

The SMCA approach exhibited a marginally lower SMAPE than the QDMDC approach,
a baseline that segments the data by year quarters (Table 3). Analysis of Fig.
6 shows that this discrepancy in SMAPE was mainly due to the lower SMAPE of
SMCA in 2014, after the first two years of training. With more training data,
the models performed similarly with negligible differences.

Additionally, the QDMDC approach yielded a higher SMAPE but slightly lower MSE
compared to the SMCA approach (Table 3). However, the performance difference
was not significant. Consequently, the straightforward SMCA approach proved
preferable to the QDMDC approach, which naively segmented the data by year
quarters without capitalizing on inherent cyclical patterns in natural gas
consumption.

  * •

RQ2: May we benefit from the model aggregation scheme based on change points
detected by the PELT algorithm (PCPDMC) compared to the quarter aggregation?

The change point divided model aggregation scheme (PCPDMC) outperformed both
baselines by all three metrics. The simplest variant (Pure) achieved the best
performance in almost all PELT settings, except for the low setting, where it
was comparable to the weighted average forecast ensemble variant (PCPDMC-WA).
However, the error of PCPDMC-WA increased with the number of change points.
Table 3 and Figures 6(a) and 6(c) show the data and the SMAPE evolution over
time.

  * •

RQ3: Does the number of detected change points affect the accuracy of the
forecast?

The PELT settings had a significant effect on the accuracy of the forecast.
The results indicated that fewer change points were preferable, as shown in
Table 3. More change points captured short-term changes in the data, often
leading to overfitting of the model and increasing error. The Low PELT setting
yielded a lower forecast error for all collection selection schemes compared
to the baselines, which was not the case for other setups. The High PELT
setting produced the highest forecast error for all approaches. This effect
can be observed in Figures 7(a) to 7(d). The figures provide a comparison of
Pure Change Point-Divided Model Collection (PCPDMC) approach with Low PELT
setting with both Mixed Change Point-Divided Model Collection approaches
(MCPDMC-WA and MCPDMC-SW) but with High PELT settings for the last forecasted
year 2020. Figures 7(a) and 7(b) show that both MCPDMC approaches with a
higher number of change points have a bias towards forecasting values higher
than the ground truth compared to the PCPDMC approach together with a higher
error variance, especially during the period from the end of March to the
first half of April. To see the effect more clearly, Figure 7(d) and Table 2
show the aggregated errors by month; we may see that the median of the
Symmetric Absolute Percentage Error in the majority of months is lower for the
PCPDMC approach compared to both MCPDMC approaches, and the biggest difference
occurred during the mentioned period. A detailed view of this period is shown
in Figure 7(c). We may see that the underperformance of the MCPDMC approaches
is mainly caused by the model performance after April 7th as the models
employing the MCPDMC approach with a higher number of change points adapt to
the slowly decreasing trend slower than the model based on the PCPDMC approach
with a lower number of change points.

((a)) SMAPE yearly aggregated progress during test-then-train experiment
protocol for PELT Settings = Low.

((b)) SMAPE yearly aggregated progress during test-then-train experiment
protocol for PELT Settings = Medium.

((c)) SMAPE yearly aggregated progress during test-then-train experiment
protocol for PELT Settings = High.

Figure 6: Progress of yearly aggregated SMAPE for all conducted experiments,
divided by the change point detection algorithm setting, proposed models, and
baseline comparison, is included for each setting. The SMAPE error measure was
computed for the forecasted data from January 1, 2014, to December 31, 2020.
SMCA = Single Model Collection Approach; QDMDC = Quarter-Divided Model
Collection; PCPDMC = Pure Change Point-Divided Model Collection; MCPDMC =
Mixed Change Point-Divided Model Collection, WA and SW suffix distinct between
WAVG and SWITCH procedures.

((a)) Ground truth natural gas consumption compared with forecasted values
using selected approaches for 2020.

((b)) Symmetric Absolute Percentage Error (SAPE) plot of the selected
forecasting approaches for 2020.

((c)) Ground truth natural gas consumption compared with forecasted values
using selected approaches for the selected part of the year 2020 in which a
multiple change points occur and significant forecasting accuracy difference
among the methods is present.

((d)) Symmetric Absolute Percentage Error (SAPE) of the selected forecasting
approaches aggregated by months for 2020.

Figure 7: Detailed look at results of PCPDMC (Low PELT settings) and MCPDMC-
WA, MCPDMC-SW (High PELT settings for both) approaches for the 2020 data. The
red dashed and black dotted vertical lines present in subfigures (a) to (c)
represent the change point locations detected with Low, respectively High,
PELT algorithm settings. Table 2: Symmetric Absolute Percentage Error (SAPE)
medians of the selected forecasting approaches aggregated by months for 2020.
The lowest median error for the month is in bold.

Month/Approach PCPDMC (Low) MCPDMC-WA (High) MCPDMC-SW (High) January 5.53
10.14 8.61 February 6.10 6.96 7.27 March 7.78 14.29 14.22 April 14.74 30.66
25.76 May 14.69 15.27 15.27 June 6.91 6.91 7.533 July 8.20 8.34 8.34 August
7.33 7.99 7.99 September 9.98 9.42 8.42 October 10.55 9.57 10.27 November 8.20
7.98 8.79 December 5.66 8.14 7.48

  * •

RQ4: Is there an advantage in employing forecast ensembling (MCPDMC-WA) or
model switching (MCPDMC-SW) in proximity to change points, with the main
consideration being the Symmetric Mean Absolute Percentage Error (SMAPE)
metric?

The performance of the schemas for Mixed Change Point-Divided Model Collection
(MCPDMC-WA and MCPDMC-SW) depended on the density of change points (i.e.,
number of segments). For Low PELT settings, the WAVG procedure performed
similarly to the PCPDMC approach and slightly reduced the forecast error for
all metrics (Table 3 and Figure 6(a)). However, with more change points, the
error reduction was only observed in the first few years of the data, as shown
in Figs. 6(b) and 6(c). After 2015, the PCPDMC approach had sub 13% SMAPE
while both WAVG and SWITCH procedures had SMAPE over 15 % and failed to beat
the baselines. The PCPDMC approach also showed a decreasing trend in SMAPE
during incremental learning, similar to the baseline models, but with a lower
SMAPE for all PELT settings. Therefore, the PCPDMC approach was more suitable
for CL tasks than the MCPDMC approach, as it was less sensitive to the number
of detected change points and achieved better results with a simpler
procedure. Furthermore, the PCPDMC approach had the lowest final SMAPE (year
2020) compared to all other approaches for each PELT setting, as seen in Figs.
6(a) to 6(c).

Table 3: Summarized results for all conducted experiments. The MAE, MSE and SMAPE measures were computed for the forecasted data from January 1, 2014, to December 31, 2020. SMCA = Single Model Collection Approach; QDMDC = Quarter-Divided Model Collection; PCPDMC = Pure Change Point-Divided Model Collection; MCPDMC = Mixed Change Point-Divided Model Collection, WA and SW suffix distinct between WAVG and SWITCH procedures. Setting/Metric | Approach | MAE | MSE | SMAPE
---|---|---|---|---
Baseline | SMCA | 1.170e+04 | 3.143e+08 | 12.94
QDMDC | 1.179e+04 | 3.109e+08 | 13.14
Low | PCPDMC | 1.110e+04 | 2.783e+08 | 12.32
MCPDMC-WA | 1.108e+04 | 2.770e+08 | 12.29
MCPDMC-SW | 1.124e+04 | 2.856e+08 | 12.48
Medium | PCPDMC | 1.181e+04 | 3.211e+08 | 12.60
MCPDMC-WA | 1.196e+04 | 3.233e+08 | 12.78
MCPDMC-SW | 1.189e+04 | 3.223e+08 | 12.81
High | PCPDMC | 1.265e+04 | 3.925e+08 | 13.09
MCPDMC-WA | 1.557e+04 | 6.368e+08 | 15.12
MCPDMC-SW | 1.483e+04 | 5.711e+08 | 14.72

##  7 Conclusions and future works

This article proposed and evaluated two novel approaches to forecasting
natural gas consumption based on change point detection and model aggregation.
We designed a general methodology for multistep forecasting of multivariate
time series with CL capabilities using data stream processing. The methodology
consists of several steps, such as dataset preprocessing, change point
detection, model collection setup, model collection selection schema, and
model training. We compared the performance of the pure and mixed change point
divided model collection approaches with two baseline approaches that do not
use change point detection using a real-world dataset from the energy domain.
We used three metrics (MSE, MAE and SMAPE) and three settings of the PELT
algorithm to measure the forecasting error and the effect of the number of
change points. Our experiments showed that the pure change point divided model
collection approach outperformed all other approaches by all metrics and for
all settings. We also found that fewer change points resulted in a lower
forecasting error and that the pure approach was more robust and suitable for
CL tasks than the mixed approach. Our results demonstrate the potential of
using change point detection and model aggregation to forecast natural gas
consumption in a dynamic and complex environment. We believe that this is an
important and promising direction for future research, as it could improve the
accuracy and efficiency of forecasting systems and enable them to adapt to
changing conditions. Some possible future work includes exploring different
types of models and selection schemes, applying the methodology to other
domains and datasets, and incorporating uncertainty estimation and anomaly
detection into the forecasting pipeline.

## Acknowledgment

This work was supported by the CEUS-UNISONO programme, which has received
funding from the National Science Center, Poland under grant agreement No.
2020/02/Y/ST6/00037, and the GACR-Czech Science Foundation project No.
21-33574K "Lifelong Machine Learning on Data Streams".

## References

  * Fab [2021] , 2021.  IEEE.  URL: "https://doi.org/10.1109%2Fbigdata52589.2021.9671962", doi:"10.1109/bigdata52589.2021.9671962".
  * Abdulsalam et al. [2007] Abdulsalam, H., Skillicorn, D., Martin, P., 2007.  Streaming random forests, pp. 225 – 232.  doi:10.1109/IDEAS.2007.4318108.
  * Adams and Mackay [2007] Adams, R.P., Mackay, D.J.C., 2007.  Bayesian online changepoint detection.  arXiv: Machine Learning .
  * Aljundi et al. [2019] Aljundi, R., Belilovsky, E., Tuytelaars, T., Charlin, L., Caccia, M., Lin, M., Page-Caccia, L., 2019.  Online continual learning with maximal interfered retrieval, in: Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., Garnett, R. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.  URL: https://proceedings.neurips.cc/paper_files/paper/2019/file/15825aee15eb335cc13f9b559f166ee8-Paper.pdf.
  * Aminikhanghahi and Cook [2017] Aminikhanghahi, S., Cook, D.J., 2017.  A survey of methods for time series change point detection.  Knowledge and information systems 51, 339–367.
  * Bai and Li [2016] Bai, Y., Li, C., 2016.  Daily natural gas consumption forecasting based on a structure-calibrated support vector regression approach.  Energy and Buildings 127, 571 – 579.  URL: http://www.sciencedirect.com/science/article/pii/S0378778816305096, doi:https://doi.org/10.1016/j.enbuild.2016.06.020.
  * Balestra and Nerlove [1966] Balestra, P., Nerlove, M., 1966.  Pooling cross section and time series data in the estimation of a dynamic model: The demand for natural gas.  Econometrica 34, 585–612.  URL: http://www.jstor.org/stable/1909771.
  * Bento et al. [2022] Bento, P.M., Pombo, J.A., Mariano, S.J., Calado, M.R., 2022.  Short-term price forecasting in the iberian electricity market: Sensitivity assessment of the exogenous variables influence , 1–7doi:10.1109/EEEIC/ICPSEurope54979.2022.9854716.
  * Bifet et al. [2012] Bifet, A., Frank, E., Holmes, G., Pfahringer, B., 2012.  Ensembles of restricted hoeffding trees.  ACM Trans. Intell. Syst. Technol. 3.  URL: https://doi.org/10.1145/2089094.2089106, doi:10.1145/2089094.2089106.
  * Bifet et al. [2018] Bifet, A., Gavaldà, R., Holmes, G., Pfahringer, B., 2018.  Machine Learning for Data Streams with Practical Examples in MOA.  MIT Press, Cambridge, MA.  URL: https://moa.cms.waikato.ac.nz/book-html/.
  * Boran [2015] Boran, F.E., 2015.  Forecasting natural gas consumption in turkey using grey prediction.  Energy Sources, Part B: Economics, Planning, and Policy 10, 208–213.  URL: https://doi.org/10.1080/15567249.2014.893040, doi:10.1080/15567249.2014.893040, arXiv:https://doi.org/10.1080/15567249.2014.893040.
  * Boschini et al. [2023] Boschini, M., Bonicelli, L., Buzzega, P., Porrello, A., Calderara, S., 2023.  Class-incremental continual learning into the extended der-verse.  IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 5497–5512.  doi:10.1109/TPAMI.2022.3206549.
  * Breiman et al. [1984] Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J., 1984.  Classification and regression trees.
  * Buzzega et al. [2020] Buzzega, P., Boschini, M., Porrello, A., Abati, D., CALDERARA, S., 2020.  Dark experience for general continual learning: a strong, simple baseline, in: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 15920–15930.  URL: https://proceedings.neurips.cc/paper_files/paper/2020/file/b704ea2c39778f07c617f6b7ce480e9e-Paper.pdf.
  * Caccia et al. [2022] Caccia, L., Aljundi, R., Asadi, N., Tuytelaars, T., Pineau, J., Belilovsky, E., 2022.  New insights on reducing abrupt representation change in online continual learning, in: International Conference on Learning Representations.  URL: https://openreview.net/forum?id=N8MaByOzUfb.
  * Cal and Woźniak [2013] Cal, P., Woźniak, M., 2013.  Parallel hoeffding decision tree for streaming data, in: Omatu, S., Neves, J., Rodriguez, J.M.C., Paz Santana, J.F., Gonzalez, S.R. (Eds.), Distributed Computing and Artificial Intelligence, Springer International Publishing, Cham. pp. 27–35.
  * Chaudhry et al. [2019] Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P.K., Torr, P.H.S., Ranzato, M., 2019.  Continual learning with tiny episodic memories.  CoRR abs/1902.10486.  URL: http://arxiv.org/abs/1902.10486, arXiv:1902.10486.
  * Chen et al. [2018] Chen, Z., Liu, B., Brachman, R., Stone, P., Rossi, F., 2018.  Lifelong Machine Learning.  2nd ed., Morgan & Claypool Publishers.
  * Cossu et al. [2021] Cossu, A., Carta, A., Lomonaco, V., Bacciu, D., 2021.  Continual learning for recurrent neural networks: An empirical evaluation.  Neural Networks 143, 607–627.  URL: https://www.sciencedirect.com/science/article/pii/S0893608021002847, doi:https://doi.org/10.1016/j.neunet.2021.07.021.
  * Creti et al. [2004] Creti, A., Villeneuve, B., et al., 2004.  Long-term contracts and take-or-pay clauses in natural gas markets.  Energy Studies Review 13, 75–94.
  * Domingos and Hulten [2000] Domingos, P., Hulten, G., 2000.  Mining high-speed data streams, in: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, New York, NY, USA. p. 71–80.  URL: https://doi.org/10.1145/347090.347107, doi:10.1145/347090.347107.
  * Ermis et al. [2022] Ermis, B., Zappella, G., Wistuba, M., Rawal, A., Archambeau, C., 2022.  Memory efficient continual learning with transformers, in: Oh, A.H., Agarwal, A., Belgrave, D., Cho, K. (Eds.), Advances in Neural Information Processing Systems.  URL: https://openreview.net/forum?id=U07d1Y-x2E.
  * French [1999] French, R.M., 1999.  Catastrophic forgetting in connectionist networks.  Trends in Cognitive Sciences 3, 128–135.  URL: https://www.sciencedirect.com/science/article/pii/S1364661399012942, doi:https://doi.org/10.1016/S1364-6613(99)01294-2.
  * González et al. [2022] González, G.G., Casas, P., Fernández, A., Gómez, G., 2022.  Steps towards continual learning in multivariate time-series anomaly detection using variational autoencoders, in: Proceedings of the 22nd ACM Internet Measurement Conference, Association for Computing Machinery, New York, NY, USA. p. 774–775.  URL: https://doi.org/10.1145/3517745.3563033, doi:10.1145/3517745.3563033.
  * Gupta et al. [2021] Gupta, V., Narwariya, J., Malhotra, P., Vig, L., Shroff, G., 2021.  Continual learning for multivariate time series tasks with variable input dimensions, in: 2021 IEEE International Conference on Data Mining (ICDM), pp. 161–170.  doi:10.1109/ICDM51629.2021.00026.
  * Haynes et al. [2014] Haynes, K., Eckley, I., Fearnhead, P., 2014.  Efficient penalty search for multiple changepoint problems .
  * Hošovský et al. [2021] Hošovský, A., Piteľ, J., Adámek, M., Mižáková, J., Židek, K., 2021.  Comparative study of week-ahead forecasting of daily gas consumption in buildings using regression arma/sarma and genetic-algorithm-optimized regression wavelet neural network models.  Journal of Building Engineering 34, 101955.  URL: https://www.sciencedirect.com/science/article/pii/S2352710220335877, doi:https://doi.org/10.1016/j.jobe.2020.101955.
  * Hussain et al. [2022] Hussain, A., Memon, J.A., Murshed, M., Md Shabbir Alam, Muhammad Rahman, U.H., 2022.  A time series forecasting analysis of overall and sector-based natural gas demand: a developing south asian economy case.  Environmental Science and Pollution Research 29.  doi:https://doi.org/10.1007/s11356-022-20861-3.
  * Jackson et al. [2005] Jackson, B., Scargle, J., Barnes, D., Arabhi, S., Alt, A., Gioumousis, P., Gwin, E., Sangtrakulcharoen, P., Tan, L., Tsai, T.T., 2005.  An algorithm for optimal partitioning of data on an interval.  IEEE Signal Processing Letters 12, 105–108.  doi:10.1109/LSP.2001.838216.
  * Khotanzad et al. [2000] Khotanzad, A., Elragal, H., Lu, T.., 2000.  Combination of artificial neural-network forecasters for prediction of natural gas consumption.  IEEE Transactions on Neural Networks 11, 464–473.
  * Killick et al. [2012] Killick, R., Fearnhead, P., Eckley, I.A., 2012.  Optimal detection of changepoints with a linear computational cost.  Journal of the American Statistical Association 107, 1590–1598.  URL: https://doi.org/10.1080%2F01621459.2012.737745, doi:"10.1080/01621459.2012.737745".
  * Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., Hadsell, R., 2017.  Overcoming catastrophic forgetting in neural networks.  Proceedings of the National Academy of Sciences 114, 3521–3526.  URL: https://www.pnas.org/doi/abs/10.1073/pnas.1611835114, doi:10.1073/pnas.1611835114, arXiv:https://www.pnas.org/doi/pdf/10.1073/pnas.1611835114.
  * Knoblauch et al. [2020] Knoblauch, J., Husain, H., Diethe, T., 2020.  Optimal continual learning has perfect memory and is np-hard, in: ICML 2020.  URL: https://www.amazon.science/publications/optimal-continual-learning-has-perfect-memory-and-is-np-hard.
  * Krawczyk et al. [2017] Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Woźniak, M., 2017.  Ensemble learning for data stream analysis: A survey.  Information Fusion 37, 132–156.  URL: https://www.sciencedirect.com/science/article/pii/S1566253516302329, doi:https://doi.org/10.1016/j.inffus.2017.02.004.
  * Lung-Yut-Fong et al. [2015] Lung-Yut-Fong, A., Lévy-Leduc, C., Cappé, O., 2015.  Homogeneity and change-point detection tests for multivariate data using rank statistics.  Journal de la société française de statistique 156, 133–162.
  * Manapragada et al. [2018] Manapragada, C., Webb, G.I., Salehi, M., 2018.  Extremely fast decision tree, in: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Association for Computing Machinery, New York, NY, USA. p. 1953–1962.  URL: https://doi.org/10.1145/3219819.3220005, doi:10.1145/3219819.3220005.
  * Masten and Crocker [1985] Masten, S.E., Crocker, K.J., 1985.  Efficient adaptation in long-term contracts: Take-or-pay provisions for natural gas.  The American Economic Review 75, 1083–1093.
  * Medina et al. [1986] Medina, J.M., McKenzie, G.A., Daniel, B.M., 1986.  Take or litigate: Enforcing the plain meaning of the take-or-pay clause in natural gas contracts.  Ark. L. Rev. 40, 185.
  * Parisi et al. [2019] Parisi, G.I., Kemker, R., Part, J.L., Kanan, C., Wermter, S., 2019.  Continual lifelong learning with neural networks: A review.  Neural Networks 113, 54–71.  URL: https://www.sciencedirect.com/science/article/pii/S0893608019300231, doi:https://doi.org/10.1016/j.neunet.2019.01.012.
  * Petropoulos et al. [2022] Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M.Z., Barrow, D.K., Ben Taieb, S., Bergmeir, C., Bessa, R.J., Bijak, J., Boylan, J.E., Browell, J., Carnevale, C., Castle, J.L., Cirillo, P., Clements, M.P., Cordeiro, C., Cyrino Oliveira, F.L., De Baets, S., Dokumentov, A., Ellison, J., Fiszeder, P., Franses, P.H., Frazier, D.T., Gilliland, M., Gönül, M.S., Goodwin, P., Grossi, L., Grushka-Cockayne, Y., Guidolin, M., Guidolin, M., Gunter, U., Guo, X., Guseo, R., Harvey, N., Hendry, D.F., Hollyman, R., Januschowski, T., Jeon, J., Jose, V.R.R., Kang, Y., Koehler, A.B., Kolassa, S., Kourentzes, N., Leva, S., Li, F., Litsiou, K., Makridakis, S., Martin, G.M., Martinez, A.B., Meeran, S., Modis, T., Nikolopoulos, K., Önkal, D., Paccagnini, A., Panagiotelis, A., Panapakidis, I., Pavía, J.M., Pedio, M., Pedregal, D.J., Pinson, P., Ramos, P., Rapach, D.E., Reade, J.J., Rostami-Tabar, B., Rubaszek, M., Sermpinis, G., Shang, H.L., Spiliotis, E., Syntetos, A.A., Talagala, P.D., Talagala, T.S., Tashman, L., Thomakos, D., Thorarinsdottir, T., Todini, E., Trapero Arenas, J.R., Wang, X., Winkler, R.L., Yusupova, A., Ziel, F., 2022.  Forecasting: theory and practice.  International Journal of Forecasting 38, 705–871.  URL: https://www.sciencedirect.com/science/article/pii/S0169207021001758, doi:https://doi.org/10.1016/j.ijforecast.2021.11.001.
  * Quinlan [1986] Quinlan, J.R., 1986.  Induction of decision trees.  Machine Learning 1, 81–106.  URL: https://doi.org/10.1007/BF00116251, doi:10.1007/BF00116251.
  * Quinlan [1993] Quinlan, J.R., 1993.  C4.5: Programs for Machine Learning.  Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
  * Rahman and Smith [2017] Rahman, A., Smith, A.D., 2017.  Predicting fuel consumption for commercial buildings with machine learning algorithms.  Energy and Buildings 152, 341 – 358.  doi:https://doi.org/10.1016/j.enbuild.2017.07.017.
  * Rusu et al. [2016] Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., Hadsell, R., 2016.  Progressive neural networks.  CoRR abs/1606.04671.  URL: http://arxiv.org/abs/1606.04671, arXiv:1606.04671.
  * Scott and Knott [1974] Scott, A.J., Knott, M., 1974.  A cluster analysis method for grouping means in the analysis of variance.  Biometrics 30, 507–512.  URL: http://www.jstor.org/stable/2529204.
  * Shin and Woo [2022] Shin, S.Y., Woo, H.G., 2022.  Energy consumption forecasting in korea using machine learning algorithms.  Energies 15.  URL: https://www.mdpi.com/1996-1073/15/13/4880, doi:10.3390/en15134880.
  * Soldo [2012] Soldo, B., 2012.  Forecasting natural gas consumption.  Applied Energy 92, 26 – 37.  doi:https://doi.org/10.1016/j.apenergy.2011.11.003.
  * Soldo et al. [2014] Soldo, B., Potočnik, P., Šimunović, G., Šarić, T., Govekar, E., 2014.  Improving the residential natural gas consumption forecasting models by using solar radiation.  Energy and Buildings 69, 498 – 506.  URL: http://www.sciencedirect.com/science/article/pii/S0378778813007299, doi:https://doi.org/10.1016/j.enbuild.2013.11.032.
  * Su et al. [2019] Su, H., Zio, E., Zhang, J., Xu, M., Li, X., Zhang, Z., 2019.  A hybrid hourly natural gas demand forecasting method based on the integration of wavelet transform and enhanced deep-rnn model.  Energy 178, 585 – 597.  doi:https://doi.org/10.1016/j.energy.2019.04.167.
  * Suykens et al. [1996] Suykens, J., Lemmerling, P., Favoreel, W., De Moor, B., Crepel, M., Briol, P., 1996.  Modelling the belgian gas consumption using neural networks.  Neural processing letters 4, 157–166.
  * Svoboda et al. [2021] Svoboda, R., Kotik, V., Platos, J., 2021.  Short-term natural gas consumption forecasting from long-term data collection.  Energy 218, 119430.  URL: http://www.sciencedirect.com/science/article/pii/S0360544220325378, doi:https://doi.org/10.1016/j.energy.2020.119430.
  * Szoplik [2015] Szoplik, J., 2015.  Forecasting of natural gas consumption with artificial neural networks.  Energy 85, 208 – 220.  URL: http://www.sciencedirect.com/science/article/pii/S036054421500393X, doi:https://doi.org/10.1016/j.energy.2015.03.084.
  * Tamba et al. [2018] Tamba, J.G., Essiane, S.N., Sapnken, E.F., Koffi, F.D., Nsouandélé, J.L., Soldo, B., Njomo, D., 2018.  Forecasting Natural Gas: A Literature Survey.  International Journal of Energy Economics and Policy 8, 216–249.  URL: https://ideas.repec.org/a/eco/journ2/2018-03-28.html.
  * Tascikaraoglu and Uzunoglu [2014] Tascikaraoglu, A., Uzunoglu, M., 2014.  A review of combined approaches for prediction of short-term wind speed and power.  Renewable and Sustainable Energy Reviews 34, 243–254.  URL: https://www.sciencedirect.com/science/article/pii/S1364032114001944, doi:https://doi.org/10.1016/j.rser.2014.03.033.
  * Taşpınar et al. [2013] Taşpınar, F., Çelebi, N., Tutkun, N., 2013.  Forecasting of daily natural gas consumption on regional basis in turkey using various computational methods.  Energy and Buildings 56, 23 – 31.  URL: http://www.sciencedirect.com/science/article/pii/S0378778812005324, doi:https://doi.org/10.1016/j.enbuild.2012.10.023.
  * Tong et al. [2023] Tong, M., Qin, F., Dong, J., 2023.  Natural gas consumption forecasting using an optimized grey bernoulli model: The case of the world’s top three natural gas consumers.  Engineering Applications of Artificial Intelligence 122, 106005.  URL: https://www.sciencedirect.com/science/article/pii/S0952197623001896, doi:https://doi.org/10.1016/j.engappai.2023.106005.
  * Truong et al. [2018] Truong, C., Oudre, L., Vayatis, N., 2018.  A review of change point detection methods.  CoRR abs/1801.00718.  URL: http://arxiv.org/abs/1801.00718, arXiv:1801.00718.
  * Viet and Mańdziuk [2005] Viet, N.H., Mańdziuk, J., 2005.  Neural and fuzzy neural networks in prediction of natural gas consumption.  Neural Parallel & Scientific Comp. 13, 265–286.
  * Vondráček et al. [2008] Vondráček, J., Pelikán, E., Konár, O., Čermáková, J., Eben, K., Malý, M., Brabec, M., 2008.  A statistical model for the estimation of natural gas consumption.  Applied Energy 85, 362 – 370.  URL: http://www.sciencedirect.com/science/article/pii/S0306261907001183, doi:https://doi.org/10.1016/j.apenergy.2007.07.004.
  * Woo et al. [2023] Woo, G., Liu, C., Sahoo, D., Kumar, A., Hoi, S., 2023.  Learning deep time-index models for time series forecasting.  arXiv:2207.06046.
  * Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S., 2017.  Continual learning through synaptic intelligence.  Proceedings of machine learning research 70, 3987–3995.
  * Zhang et al. [2022] Zhang, Y., Pfahringer, B., Frank, E., Bifet, A., Lim, N.J.S., Jia, Y., 2022.  A simple but strong baseline for online continual learning: Repeated augmented rehearsal, in: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 14771–14783.  URL: https://proceedings.neurips.cc/paper_files/paper/2022/file/5ebbbac62b968254093023f1c95015d3-Paper-Conference.pdf.