Spatiotemporal forecasting has become an increasingly important prediction task in machine learning and statistics due to its vast applications, such as climate modeling, traffic prediction, video caching predictions, and so on. While numerous studies have been conducted, most existing works assume that the data from different sources or across different locations are equally reliable. Due to cost, accessibility, or other factors, it is inevitable that the data quality could vary, which introduces significant biases into the model and leads to unreliable prediction results. The problem could be exacerbated in black-box prediction models, such as deep neural networks. In this paper, we propose a novel solution that can automatically infer data quality levels of different sources through local variations of spatiotemporal signals without explicit labels. Furthermore, we integrate the estimate of data quality level with graph convolutional networks to exploit their efficient structures. We evaluate our proposed method on forecasting temperatures in Los Angeles. Recent advances in sensor and satellite technology have facilitated the collection of large spatiotemporal datasets. As the amount of spatiotemporal data increases, many have proposed representing this data as time-varying graph signals in various domains, such as sensor networks BID21 BID29 , climate analysis BID2 BID17 , traffic control systems BID15 , and biology BID18 BID26 .While existing work have exploited both spatial structures and temporal signals, most of them assume that each signal source in a spatial structure is equally reliable over time. However , a large amount of data comes from heterogeneous sensors or equipment leading to various levels of noise BID22 BID27 . Moreover , the noises of each source can vary over time due to movement of the sensors or abrupt malfunctions. This problem raises significantly challenges to train and apply complex black box machine learning models, such as deep neural networks, because even a small perturbation in data can deceive the models and lead to unexpected behaviors BID5 BID13 . Therefore, it is extremely important to consider data quality explicitly when designing machine learning models.The definitions of data quality can be varied -high quality data is generally referred to as fitness for intended uses in operations, decision making and planning BID19 . In this paper , we narrow down the definition as a penalizing quantity for high local variations. We consider a learning problem of spatiotemporal signals that are represented by time-varying graph signals for different data qualities. Given a graph G = (V, E, W) and observations X ∈ R N ×M ×T where N, M, T are the number of vertices, the types of signals, and the length of time-varying signals, respectively. We define the concept of data quality levels at each vertex as latent variables, which are connected through a graph using a local variation of the vertex. The local variation at each vertex depends on the local spatial structure and neighboring signals. Our definition of data quality can be easily incorporated into any existing machine learning models through a regularizer in their objective functions. In this paper, we develop data quality long short-term memory (DQ-LSTM) neural networks for spatiotemporal forecasting. DQ-LSTM effectively exploits spatial structures of data quality levels at each vertex through graph convolution, which examines neighboring signals at a set of K-hop neighboring vertices, and captures the temporal dependencies of each time series through LSTMs. We demonstrate that data quality is an essential factor for improving the predictive performance of neural networks via experiments on urban heat island prediction in Los Angeles.Related work A series of work have been conducted on addessing the issues of data qualities and heterogeneous data sources. BID23 is the first theoretical work that proposes a mixture model for captureing two types of labels in supervised learning. One type of the labels is considered as high quality labels from an expensive source (domain experts) while another type is from errorprone crowdsourcing. Since the reliability or quality of the labels is different, it is not desired to consider them equally. The authors proposed a learning algorithm that can utilize the error-prone labels to reduce the cost required for the expert labeling. BID27 address issues from strong and weak labelers by developing an active learning algorithm minimizing the number of label requests. BID22 focus on the data of variable quality resulting from heterogeneous sources. The authors define the concept of heterogeneity of data and develop a method of adjusting the learning rate based on the heterogeneity. Different from existing works, our proposed framework differentiates heterogeneous sources based on neighborhood signals without any explicit labels.Another set of work related to our study is learning and processing graph signals or features. Spectral graph theory BID3 BID24 BID21 has been developed as a main study to understand two aspects of graph signals: structures and signals. Under this theory many models have been introduced to exploit convolutional neural networks (CNNs) which provide an efficient architecture to extract localized patterns from regular grids, such as images BID14 . BID1 learns convolutional parameters based on the spectrum of the graph Laplacian. Later, BID8 extends the spectral aspect of CNNs on graphs into largescale learning problemsDefferrard et al. FORMULA0 proposes a spectral formulation for fast localized filtering with efficient pooling. Furthermore, BID12 re-formularizes existing ideas into layer-wise neural networks that can be tuned through backpropagation rule with a first-order approximation of spectral filters introduced in BID7 . Built on these work, we propose a graph convolutional layer that maps spatiotemporal features into a data quality level.Outline We review graph signal processing to define the local variation and a data quality level (DQL) with graph convolutional networks in Section 2. In Section 3, we provide how the data quality levels are exploited with recurrent neural networks to differentiate reliability of observations on vertices. Also, we construct a forecasting model, DQ-LSTM. Our main result is presented in Section 4 with other baselines. In Section 5 we discuss its properties and interpret the data reliability inferred from our model. In this work, we study the problem of data quality for spatiotemporal data analysis. While existing works assume that all signals are equally reliable over time, we argue that it is important to differentiate data quality because the signals come from heterogeneous sources. We proposed a novel formulation that automatically infers data quality levels of different sources and developed a specific formulation, namely DQ-LSTM, based on graph convolution for spatiotemporal forecasting. We demonstrate the effectiveness of DQ-LSTM on inferring data quality and improving prediction performance on a real-world climate dataset. For future work, we are interested in further refining the definitions of data quality and examining rigorous evaluation metrics.
We propose a method that infers the time-varying data quality level for spatiotemporal forecasting without explicitly assigned labels. Introduces a new definition of data quality that relies on the notion of local variation defined in (Zhou and Scholkopf) and extends it to multiple heterogenous data sources. This work proposed a new way to evaluate the quality of different data sources with the time-vary graph model, with the quality level used as a regularization term in the objective function