Opportunistic Air Quality Monitoring and Forecasting with Expandable
  Graph Neural Networks
Air Quality Monitoring and Forecasting has been a popular research topic in
recent years. Recently, data-driven approaches for air quality forecasting have
garnered significant attention, owing to the availability of well-established
data collection facilities in urban areas. Fixed infrastructures, typically
deployed by national institutes or tech giants, often fall short in meeting the
requirements of diverse personalized scenarios, e.g., forecasting in areas
without any existing infrastructure. Consequently, smaller institutes or
companies with limited budgets are compelled to seek tailored solutions by
introducing more flexible infrastructures for data collection. In this paper,
we propose an expandable graph attention network (EGAT) model, which digests
data collected from existing and newly-added infrastructures, with different
spatial structures. Additionally, our proposal can be embedded into any air
quality forecasting models, to apply to the scenarios with evolving spatial
structures. The proposal is validated over real air quality data from
PurpleAir.
Jingwei Zuo, Wenbin Li, Michele Baldo, Hakim Hacid
# Opportunistic Air Quality Monitoring and Forecasting with Expandable Graph Neural Networks

###### Abstract

Air Quality Monitoring and Forecasting has been a popular research topic in recent years. Recently, data-driven approaches for air quality forecasting have garnered significant attention, owing to the availability of well-established data collection facilities in urban areas. Fixed infrastructures, typically deployed by national institutes or tech giants, often fall short in meeting the requirements of diverse personalized scenarios, e.g., forecasting in areas without any existing infrastructure. Consequently, smaller institutes or companies with limited budgets are compelled to seek tailored solutions by introducing more flexible infrastructures for data collection. In this paper, we propose an expandable graph attention network (EGAT) model, which digests data collected from existing and newly-added infrastructures, with different spatial structures. Additionally, our proposal can be embedded into any air quality forecasting models, to apply to the scenarios with evolving spatial structures. The proposal is validated over real air quality data from PurpleAir.

 Air Quality Forecasting, Opportunistic Forecasting, Graph Neural Networks, Urban Computing

## I Introduction

Air quality forecasting using data-driven models has gained significant attention in recent years, thanks to the proliferation of data collection infrastructures such as sensor stations and advancements of telecommunication technologies. These infrastructures are typically managed by national institutes (e.g., AirParif1, EPA2) or large companies (e.g., PurpleAir3) that specialize in air quality monitoring or forecasting services and products. Leveraging existing data collection infrastructures proves beneficial for initial research exploration or validating product prototypes. However, reliance on fixed infrastructures presents practical constraints when customization is required for specific tasks. For instance, certain monitoring areas may be inadequately covered or completely absent from the existing infrastructures, or the density of coverage may not be sufficient. This issue particularly affects small or mid-sized industrial and academic players who face budget limitations that prevent them from investing in their own infrastructure from scratch, but have specific customization needs.

Footnote 1: [https://www.airparif.asso.fr/](https://www.airparif.asso.fr/)

Footnote 2: [https://www.epa.gov/air-quality](https://www.epa.gov/air-quality)

Footnote 3: [https://www2.purpleair.com/](https://www2.purpleair.com/)

In addition to data collection, air quality forecasting models trained solely with data from public fixed infrastructures may not perform well for users' specific scenarios, such as forecasting at a higher spatial resolution. Deploying additional sensors as a cost-effective solution can enrich the data and improve forecasting performance without the need to build infrastructures from scratch. Subsequently, this targeted solution leads us to consider the practical question: _how we can make use of the data collected from existing infrastructures, when integrating new sensor infrastructures?_

As depicted in Figure 1, the topological sensor network may change as the urban infrastructure evolves, resulting in varying network structures of air quality sensors. The data collected from the network \(G_{\tau}\) needs to be augmented with enriched data from newly installed sensors \(\Delta G_{\tau^{\prime}}\) and \(\Delta G_{\tau^{\prime\prime}}\). Training a model solely on recent data with \(G_{\tau^{\prime\prime}}\) would overlook valuable information contained in the historical data with \(G_{\tau}\) and \(G_{\tau^{\prime}}\).

In this paper, we propose an expandable graph attention network (EGAT) that effectively integrates data with various graph structures. This approach is versatile and can be seamlessly embedded into any existing air quality forecasting model. Furthermore, it applies to scenarios where sensors are not installed, enabling accurate forecasting in such areas. We summarize our approach's main advantages as follows:

* **Less is more:** With fewer installed sensors, we can directly predict the air quality of other unknown area where sensors are not installed and achieve comparable performance to models relying on extensive data collection infrastructures with more sensors.
* **Continual learning with self-adaptation:** The proposed model enables continuous learning from newly collected data with expanded sensor networks, demonstrating self-adaptability to different topological sensor networks.
* **Embeddable module with scalability:** The proposed module can be seamlessly integrated into any air quality forecasting model, enhancing its ability to forecast in real

Fig. 1: Expanded sensor networks and the related \(PM_{2.5}\) data at different time. The data was collected with \(PurpleairAPI\)[1].
world scenarios.

The rest of this paper starts with a review of the most related work. Then, we formulate the problems of the paper. Later, we present in detail our proposal, which is followed by the experiments on real-life datasets and the conclusion.

## II Related Work

### _Air Quality Forecasting_

Data-driven models for air quality forecasting has gained a huge popularity recently. Recent work [2, 3] studies graph-based representations of the air quality data by considering the sensor network as a graph structure, which extracts decent structural features between sensor data from a topological view. The air quality forecasting can be then formulated as a spatio-temporal forecasting problem.

Works like DCRNN [4], STGCN [5] and Graph WaveNet [6], have shown promising results in traffic forecasting tasks. These models can be adapted to air quality forecasting tasks owing to the shared spatio-temporal features present in the data. However, in practice, the above-mentioned models often overlook the evolving nature of sensor networks as more data collection infrastructures are incrementally built. Consequently, these models require re-training from scratch on the most recent data that reflects the evolved sensor network. It may result in the loss of valuable information contained in outdated data collected from different network configurations.

### _Expandable Graph Neural Networks_

In the field of graph learning, several works, such as ContinualGNN [7] and ER-GNN [8], have incorporated the concept of Continual Learning to capture the evolving patterns within graph nodes. While these approaches are valuable, it is important to consider spatio-temporal features in air quality forecasting tasks. Designed for traffic forecasting, TrafficStream [9] considers evolving patterns on both temporal and spatial axes; ST-GFSL [10] introduces a meta-learning model for cross-city spatio-temporal knowledge transfer. However, these works primarily focus on shared (meta-)knowledge between nodes, and give less attention to expandable graph structures. Basically, spectral-based graph neural networks (GNNs) face challenges when scaling to graphs with different structures due to the complexity of reconstructing the Laplacian matrix. To address this issue, our paper explores the use of spatial-based GNNs, such as Graph Attention Networks (GAT) [11], for expandable graph learning in air quality forecasting tasks.

## III Problem Formulation

**Definition 1**.: (Air Quality Forecasting). Given an air quality sensor network \(G=\{\mathcal{V},\mathcal{E}\}\), where \(\mathcal{V}=\{v_{1},...,v_{N}\}\) is a set of \(N\) sensor nodes/stations and \(\mathcal{E}=\{e_{1},...,e_{E}\}\) is a set of \(E\) edges connecting the nodes, the air quality data \(\{AQI_{t}\}_{t=1}^{T}\) and meteorological data \(\{M_{t}\}_{t=1}^{T}\) are collected over the \(N\) stations, where \(T\) is current timestamp. We aim to build a model \(f\) to predict the \(AQI\) over the next \(T_{p}\) timestamps.

To simplify, we denote input data as \(\mathcal{X}\)= \(\{AQI_{t},M_{t}\}_{t=1}^{T}\) = \(\{x_{t}\}_{t=1}^{T}\in\mathbb{R}^{N\times F\times T}\). Each node contains \(F\) features representing \(PM_{2.5}\), \(PM_{10}\), humidity, temperature, etc. As \(PM_{2.5}\) is _most reported and most difficult-to-predict_[12], we take \(PM_{2.5}\) as the AQI prediction target \(\mathcal{Y}\)=\(\{y_{t}\}_{t=T+1}^{T+p}\in\mathbb{R}^{N\times T_{p}}\).

**Definition 2**.: (Expanded Sensor Network). Given a sensor network at \(\tau\): \(G_{\tau}\) = \(\{\mathcal{V}_{\tau},\mathcal{E}_{\tau}\}\) with \(N_{\tau}\) sensors, the network at \(\tau^{\prime}\): \(G_{\tau^{\prime}}\)=\(G_{\tau}\)+\(\Delta G_{\tau}\) = \(\{V_{\tau^{\prime}},E_{\tau^{\prime}}\}\) expands \(G_{\tau}\) to \(N_{\tau^{\prime}}\) sensors.

We aim to build a model \(f\), which is firstly trained over a dataset \(\{\mathcal{X}_{\tau}\}\) on a sensor network \(G_{\tau}\) = \(\{\mathcal{V}_{\tau},\mathcal{E}_{\tau}\}\), and can be incrementally trained over \(\{\mathcal{X}_{\tau^{\prime}}\}\) on an expanded network \(G_{\tau^{\prime}}\). For inference, given a sequence \(\mathcal{X}\in\mathbb{R}^{N_{\tau^{\prime}}\times F\times T}\) and a sensor network \(G_{\tau^{\prime}}\), the model \(f\) can predict the \(AQI\) for the next \(T_{p}\) time steps \(\mathcal{Y}\)=\(\{y_{t}\}_{t=T+1}^{T+p}\in\mathbb{R}^{N\times T_{p}}\), where \(N_{\tau^{\prime}}\geq N_{\tau}\).

## IV Our proposals

In this paper, we adopt Graph WaveNet [6] as the backbone model, which consists of \(l\) Spatio-Temporal (ST) Blocks. However, our proposed EGAT can be integrated to any spatio-temporal models with adaptations on graph network layers. We employ Temporal Convolution Network (TCN) to encode the temporal dynamics of the AQIs. Specifically, as shown in Figure 2, we designed an Expandable Graph Attention Network (EGAT) to learn from the data with evolving graph structures. The output forecasting layer takes skip connections on the output of the final ST Block and the hidden states after each TCN module for final predictions.

### _Temporal Dynamics with Temporal Convolution Network_

Compared to RNN-based approaches, Temporal Convolution Network (TCN) [6] allows handling long-range sequences in a parallel manner, which is critical in industrial scenarios considering the model efficiency.

Given an input air quality sequence embedding \(H\)= \(f_{linear}(\mathcal{X})\in\mathbb{R}^{N\times d\times T}\), a filter \(\mathcal{F}\in\mathbb{R}^{1\times\mathrm{K}}\), \(\mathrm{K}\) is the temporal filter size, \(\mathrm{K}=2\) by default. The dilated causal convolution operation of \(H\) with \(\mathcal{F}\) at time \(t\) is represented as:

\[H\star\mathcal{F}(t)=\sum_{s=0}^{\mathrm{K}}\mathcal{F}(s)H(t-\textbf{d}\times s )\in\mathbb{R}^{N\times d\times T^{\prime}} \tag{1}\]

where \(\star\) is the convolution operator, **d** is the dilation factor, \(d\) is the embedding size, \(T^{\prime}\) is the generated sequence length. We define the output of a gated TCN layer as:

\[\textbf{h}=tanh(W_{\mathcal{F}^{1}}\star H)\odot\sigma(W_{\mathcal{F}^{2}} \star H)\in\mathbb{R}^{N\times d\times T^{\prime}} \tag{2}\]

where \(W_{\mathcal{F}^{1}}\), \(W_{\mathcal{F}^{2}}\) are learnable parameters, \(\odot\) is the element-wise multiplication operator, \(\sigma(\cdot)\) denotes Sigmoid function.

### _Expandable Graph Attention Networks (EGATs)_

Graph attention network (GAT) [11], as a weighted message-passing process, models neighboring nodes' relationships via their inherent feature similarities. Given a set of air pollution features at time \(t\): \(\textbf{h}(t)\) = \(\{h_{1},h_{2},...,h_{N}\},h_{i}\in\mathbb{R}^{N\times d}\) as input of a graph attention layer, following [11], we define the attention score between node \(i\), \(j\) as:

\[\alpha_{ij}=\frac{\exp\left(\mathrm{a}\left(Wh_{i},Wh_{j}\right)\right)}{\sum_ {k\in\mathcal{N}_{i}}\exp\left(\mathrm{a}\left(Wh_{i},Wh_{k}\right)\right)} \tag{3}\]
where \(W\in\mathbb{R}^{d\times d^{\prime}}\) is a weight matrix, \(\mathrm{a}\) is the attentional mechanism as mentioned in [11]: \(\mathbb{R}^{d^{\prime}}\times\mathbb{R}^{d^{\prime}}\rightarrow\mathbb{R}\), and \(\mathcal{N}_{i}\) is a set of neighbor nodes of \(v_{i}\). A _multi-head attention_ with a nonlinearity \(\sigma\) is employed to obtain abundant spatial representation of \(v_{i}\) with features from its neighbor nodes \(\mathcal{N}_{i}\):

\[h_{i}^{\prime}=\sigma\left(\frac{1}{K}\sum_{k=1}^{K}\sum_{j\in\mathcal{N}_{i}} \alpha_{ij}W^{k}h_{j}\right) \tag{4}\]

Therefore, the GAT layer in \(i\)-th ST Block can be defined as:

\[H_{i+1}=\sigma\left(\frac{1}{K}\sum_{k=1}^{K}\mathcal{A}\mathbf{h}_{i}W^{k}\right) \tag{5}\]

where \(\mathcal{A}\)=\(\{\alpha_{ij}\}\in\mathbb{R}^{N\times N}\), \(H_{i+1}\in\mathbb{R}^{N\times d^{\prime}\times T}\), \(W^{k}\in\mathbb{R}^{d\times d^{\prime}}\).

When expanding the graph with new sensor nodes, we scale up the GAT layers on new nodes while conserving the information learned over the old ones. Basically, new nodes can be considered during both model's training and inference.

#### Iv-B1 Expandable Graph Network Training

We consider that the sensor network expands with the newly built infrastructures. The model learned from \(G_{\tau}\) can be updated with recent data over \(G_{\tau^{\prime}}\) without re-training the model from scratch.

From Equation 5, with new embeddings \(\mathbf{h}_{\tau^{\prime}}\)\(\in\)\(\mathbb{R}^{N_{\tau^{\prime}}\times d\times T}\), the weight matrix \(W^{k}\) stays unchanged; only the adjacency matrix requires updates: \(\mathcal{A}_{\tau}\in\mathbb{R}^{N_{\tau}\times N_{\tau}}\rightarrow\mathcal{A }_{\tau^{\prime}}\in\mathbb{R}^{N_{\tau^{\prime}}\times N_{\tau^{\prime}}}\). We re-define \(\mathcal{N}_{i}\)=\(\{\mathcal{N}_{i,\tau},\mathcal{N}_{i,\tau^{\prime}}\}\) as the \(k\) nearest neighbors of \(v_{i}\), where \(\mathcal{N}_{i,\tau}\) denotes neighbors from existing nodes, \(\mathcal{N}_{i,\tau^{\prime}}\) indicates those from newly added nodes. Given a set of new sensors \(\Delta\mathcal{V}_{\tau}\), we obtain new edge connections \(\Delta\mathcal{E}_{\tau}\)=\(\{\mathcal{N}_{i}\}_{i=1}^{\Delta\Delta}\), where \(\Delta N\)=\(N_{\tau^{\prime}}-N_{\tau}\), with \(\mathcal{O}(N_{\tau^{\prime}}\Delta N)\) time for distance computations. According to Equation 3, the attentional mechanism will apply to \(\Delta\mathcal{E}_{\tau}\) with \(\mathcal{O}(\Delta Nk)\) time. Therefore, the attention score between node \(i,j\) can be re-defined as:

\[\alpha_{ij}=\frac{\exp\left(\mathrm{a}\left(Wh_{i},Wh_{j}\right)\right)}{\sum \limits_{k\in\mathcal{N}_{i,\tau}}\exp\left(\mathrm{a}\left(Wh_{i},Wh_{k} \right)\right)+\sum\limits_{k\in\mathcal{N}_{i,\tau^{\prime}}}\exp\left( \mathrm{a}\left(Wh_{i},Wh_{k}\right)\right)} \tag{6}\]

In this manner, we can update the graph layer, i.e., \(\mathcal{A}_{\tau^{\prime}}\) incrementally by considering cached attention scores over \(\mathcal{E}_{\tau}\), reducing the time complexity to \(\mathcal{O}(N_{\tau^{\prime}}\Delta N+\Delta Nk)\). This is much faster than rebuilding the entire graph layer (\(\mathcal{O}(N_{\tau^{\prime}}^{2})\)).

#### Iv-B2 Expandable Graph Network Inference

When no sensors are installed in (unseen) areas, _Spatial Smoothing_ can be performed on the unseen node \(v_{i}\). Based on its spatial location, we incorporate predictions from its neighbor nodes:

\[Y_{i}=\sum_{j\in\mathcal{N}_{i}}a_{ij}Y_{j},N_{i}=\{v_{j}|dist(v_{i},v_{j})<\varepsilon\} \tag{7}\]

where \(\mathcal{N}_{i}\) is the first-order neighbors of \(v_{i}\) (excluding \(v_{i}\), as the data on \(v_{i}\) is unavailable), \(a_{ij}=1-\frac{dist(v_{i},v_{j})}{\sum_{k\in\mathcal{N}_{i}}dist(v_{i},v_{k})}\) is the inverse Euclidean Distance (ED) between \(v_{i}\) and \(v_{j}\), \(\varepsilon\) is a threshold which decides the neighboring sensor nodes.

We propose a robust _Spatial Representation Smoothing_ technique that considers richer spatial relationships, in the embedding space, between unseen and existing nodes. Given an unseen node \(v_{i}\), its embedding \(h_{i}\) can be defined as follows:

\[h_{i}=\sigma\left(\frac{1}{K}\sum_{k=1}^{K}\sum_{j\in\mathcal{N}_{i}}a_{ij}W^{ k}h_{j}\right) \tag{8}\]

where \(a_{ij}\) is the inverse ED between \(v_{i}\) and \(v_{j}\), \(W^{k}\) is the learned weights in each attention head as shown in Equation 4.

### _Output Forecasting Layer_

For final predictions, we take skip connections as shown in [6] on the final ST Block's output and hidden states after each TCN. The concatenated output features are defined as:

\[O=(\mathbf{h}_{0}W^{0}+b^{0})\|...\|(\mathbf{h}_{l-1}W^{l-1}+bl-1)\ \|(\mathcal{H}_{l}W^{l}+b^{l}) \tag{9}\]

where \(O\in\mathbb{R}^{N\times(l+1)d}\), \(W^{i}_{s}\), \(b^{i}_{s}\) are learnable parameters for the convolution layers. Two fully-connected layers are added to project the concatenated features into the desired dimension:

\[\hat{\mathcal{Y}}=(ReLU(OW^{l}_{fc}+b^{1}_{fc}))W^{2}_{fc}+b^{2}_{fc}\in \mathbb{R}^{N\times T_{p}} \tag{10}\]

where \(W^{1}_{fc}\), \(W^{2}_{fc}\), \(b^{1}_{fc}\), \(b^{2}_{fc}\) are learnable parameters. We use mean absolute error (MAE) [6] as loss function for training.

## V Experiments

In this section, we demonstrate the effectiveness of EGAT with real-life air quality datasets. The experiments were designed to answer the following questions:

**Q1**: _Continal learning with self-adaptation:_ How well can our model make use of the ancient data with different graph structures, to improve the model's performance?

Fig. 2: Global system architecture of EGAT
* _Flexible Inference on unknown areas:_ How well is our model at predicting air quality in areas without any sensors installed? i.e., no available data over these areas.

### _Experimental Settings_

#### V-A1 Dataset description

We base our experiments on real air quality data [13] collected via PurpleAir API [1], which contains the AOIs and meteorological data in San Francisco (within \(10\,km^{2}\)) between 2021-10-01 and 2023-05-15. The datasets are split to training, validation, test sets with _7:1:2_. Table I shows more details of the collected datasets. For PurpleAirSF-1H, we adopt the last 12-hour data to predict the AQI (i.e., PM2.5) for the next 12 hours. For PurpleAir-6H, we consider the last 72 hours to predict the next 72 hours.

#### V-A2 Execution and Parameter Settings

We take Graph WaveNet as the backbone model. However, our proposal can be integrated to any air quality forecasting models. All the tests are done on a single Tesla A100 GPU of 40 Go memory. The forecasting accuracy of all tested models is evaluated by three metrics [2]: mean absolute error (MAE), root-mean-square error (RMSE) and mean absolute percentage error (MAPE).

#### V-A3 Baselines

We compare EGAT with various model variants and with Graph WaveNet [6]:

* **GraphWaveNet** (GWN) [6]: Trained on expanded graph data, as it is non-adaptable to different graph structures.
* **EGAT-Rec**: EGAT trained on data with expanded graph;
* **EGAT-FI-SS**: EGAT trained on data over ancient graph, Flexible Inference (FI) with _Spatial Smoothing_ is applied;
* **EGAT-FI-SRS**: EGAT trained on ancient data, FI with _Spatial Representation Smoothing_ is employed;
* **EGAT**: EGAT trained on both ancient and recent data.

### _Experimental Results_

Table II and Table III reports the average errors (12/72H) regarding the expanding node ratio and expanding time ratio determined by the deployment. **Bold** values indicate the best results, while underlined values represent the second-best.

EGAT consistently outperforms other models in continual learning with different node ratios and time radios, owning to its ability to leverage rich data from various graph structures. While GWN performs better than EGAT-Rec, this can be attributed to the k-order diffusion process in GCN. Even so, EGAT surpasses GWN by incorporating ancient graph data, further validating our proposal in graph adaptations (**Q1**).

When forecasting in unknown areas, EGAT-FI-SS provides approximate AQIs through _Spatial Smoothing_. However, its performance deteriorates with a high number of expanded nodes due to spatial sparsity. EGAT-FI-SRS performs better than EGAT-FI-SS and sometimes even better than GWN and comparable to EGAT, validating the viability of _Spatial Representation Smoothing_ for unknown areas' prediction (**Q2**).

## VI Perspectives and Conclusion

In this paper, we propose an Expandable Graph Attention Network (EGAT) for Air Quality monitoring and forecasting. It incorporates historical and recent graph data, which prevents industrial players with budget limitations from investing in their own infrastructures from scratch. EGAT also allows predicting air quality in areas without installed sensors. Future work includes comparing additional expandable graph learning models and exploring transfer learning and node alignment techniques to reduce re-training effort in industrial scenarios.
