# GA-SmaAt-GNet: Generative Adversarial Small Attention GNet for Extreme
Precipitation Nowcasting

Eloy Reulen  Siamak Mehrkanoon<EMAIL_ADDRESS>Department of Information
and Computing Sciences, Utrecht University, Utrecht, The Netherlands

###### Abstract

In recent years, data-driven modeling approaches have gained considerable
traction in various meteorological applications, particularly in the realm of
weather forecasting. However, these approaches often encounter challenges when
dealing with extreme weather conditions. In light of this, we propose GA-
SmaAt-GNet, a novel generative adversarial architecture that makes use of two
methodologies aimed at enhancing the performance of deep learning models for
extreme precipitation nowcasting. Firstly, it uses a novel SmaAt-GNet built
upon the successful SmaAt-UNet architecture as generator. This network
incorporates precipitation masks (binarized precipitation maps) as an
additional data source, leveraging valuable information for improved
predictions. Additionally, GA-SmaAt-GNet utilizes an attention-augmented
discriminator inspired by the well-established Pix2Pix architecture.
Furthermore, we assess the performance of GA-SmaAt-GNet using real-life
precipitation dataset from the Netherlands. Our experimental results reveal a
notable improvement in both overall performance and for extreme precipitation
events. Furthermore, we conduct uncertainty analysis on the proposed GA-SmaAt-
GNet model as well as on the precipitation dataset, providing additional
insights into the predictive capabilities of the model. Finally, we offer
further insights into the predictions of our proposed model using Grad-CAM.
This visual explanation technique generates activation heatmaps, illustrating
areas of the input that are more activated for various parts of the network.

###### keywords:

Extreme Precipitation Nowcasting , UNet , GAN , Attention , Deep Learning

##  1 Introduction

Precipitation nowcasting [1], also referred to as short-term precipitation
forecasting, has long been a focal point of meteorological research.
Especially, the ability to accurately predict short-term extreme precipitation
events holds particular significance, as these weather phenomena have a
substantial impact on both human life and economy. Notably, the Netherlands
has experienced a noteworthy increase in precipitation over the past century,
as highlighted in the findings of studies such as [2, 3]. Furthermore, the
frequency of extreme precipitation events in the Netherlands has been on the
rise, as demonstrated by [4, 3]. In light of these developments, precise
nowcasts for extreme precipitation events are of great importance for timely
warnings and precautionary measures. Traditionally, meteorologists have relied
upon Numerical Weather Prediction (NWP) models for generating weather
forecasts [5]. NWP models employ intricate mathematical equations to simulate
various atmospheric properties. This involves solving a set of complex partial
differential equations governing the physical state of the atmosphere. While
NWP models can deliver accurate forecasts, their computational demands make
them less suitable for nowcasting tasks which require short-term forecasts,
typically ranging from minutes to a maximum of six hours. In recent years,
Convolutional Neural Networks (CNNs), a class of deep learning models
particularly well-suited for image and spatial data analysis, have emerged as
a promising area of research for tackling the precipitation nowcasting
problem. CNNs have demonstrated remarkable success in various fields,
including computer vision [6, 7, 8, 9], natural language processing [10, 11],
and, increasingly, meteorology [12, 13, 9]. Their ability to extract spatial
patterns and features from radar or satellite imagery has made them valuable
tools in the quest for more accurate precipitation forecasts, even for extreme
weather events [14]. However, it is essential to acknowledge that CNNs, while
showing promise, often encounter significant challenges when predicting
extreme precipitation events [15]. These events exhibit complex and non-linear
behaviors that can strain the capabilities of traditional deep learning
models. As such, ongoing research in this field is not only focused on
leveraging CNNs but also on enhancing their performance in predicting extreme
weather events through innovative architectures [14], data augmentation
techniques [16], and the integration of additional meteorological variables
[17]. By addressing these challenges, meteorologists aim to harness the full
potential of deep learning techniques in advancing the accuracy and
reliability for extreme precipitation nowcasting.

In this paper, we present GA-SmaAt-GNet, a novel generative adversarial
architecture designed to enhance the performance of the widely adopted SmaAt-
UNet architecture [18] in the context of extreme precipitation nowcasting. Our
approach leverages a novel SmaAt-GNet architecture as the generator and
incorporates an attention-augmented discriminator to achieve improved
performance. Additionally, we provide a look into uncertainty of the models as
well as the used precipitation data. Furthermore, we utilize Grad-CAM, a
visual explanation technique, to generate heatmaps of the activations of
different parts of our proposed network to gain further insight into its
predictions. It is important to note that the precise definition of extreme
precipitation can vary between different regions and research studies. For the
purposes of this paper, when we refer to extreme precipitation in the
Netherlands, we are specifically referring to precipitation depths exceeding
20 mm accumulated within a one-hour timeframe, as most drainage systems in the
Netherlands are not able to process more than this amount of rain per hour
[19].

This paper is organized as follows: we first give a brief overview of related
literature on weather forecasting in Section 2. We then introduce our proposed
models in Section 3, while Section 4 describes the used dataset and conducted
experiments. The results of the experiments are discussed in Section 5 and
finally we make some concluding remarks in Section 6.

##  2 Related work

In recent years, deep learning techniques have found applications in various
weather forecasting tasks [9]. For instance, Shi et al. introduced ConvLSTM,
which expanded upon the LSTM architecture by incorporating convolutional
structures in the input-to-state and state-to-state transitions [12]. Their
approach demonstrated improved performance when compared to traditional
optical flow-based methods for precipitation nowcasting. Building upon the
success of ConvLSTM, a subsequent study introduced TrajGRU [13]. TrajGRU
extended ConvLSTM by employing a GRU (Gated Recurrent Unit) instead of an
LSTM, a modification that enables the model to learn location-specific
patterns more effectively.

More recently, U-Net-based architectures have gained traction in the field of
weather forecasting. Originally designed for biomedical image segmentation
[20], the U-Net architecture has been repurposed effectively for various
applications, including weather forecasting. This architecture consists of an
encoder network, which progressively reduces the spatial resolution of the
input image, and a decoder network that upsamples the feature maps back to the
original image dimensions. These two networks are interconnected via skip
connections, enabling the decoder to access high-resolution features from the
encoder. While initially intended for biomedical image segmentation, the U-Net
architecture has found successful application in various domains, including
weather forecasting [21, 18, 22]. Trebing et al. [18] extended the U-Net
architecture by incorporating attention modules and depthwise-separable
convolutions, successfully deploying it for precipitation nowcasting in the
Netherlands and cloud cover nowcasting in France. The prediction performance
of their SmaAt-UNet architecture is on par with the traditional U-Net, but
with a notable reduction in the number of trainable parameters.

Han et al. conducted a comprehensive study, as documented in [23], to assess
the performance of various models in the radar precipitation nowcasting task.
Among the models they examined were ConvLSTM and U-Net architectures. Their
evaluation encompassed a period spanning from 2018 to 2020, as well as a
specific case study focusing on a summer heavy rainfall event. The results of
their investigation revealed a consistent trend: U-Net-based architectures
consistently outperformed other models in terms of the critical success index
(CSI) in both the long-term period and the context of the summer heavy
rainfall event. Furthermore, their findings indicated that U-Net architectures
achieved their best performance with shorter input sequences (60 minutes)
compared to longer sequences (120 minutes). Additionally, they highlighted the
significant influence of the most recent input frame on the accuracy of
predictions.

Recent research has also demonstrated the potential for enhancing forecast
accuracy by integrating different types of data sources. For instance, Ko et
al. [24] illustrated that combining radar-based precipitation data with
ground-based weather data yields improved forecast performance. Additionally,
Kaparakis et al. introduced WF-UNet [25], highlighting the benefits of
incorporating wind radar data into deep learning models for more accurate
precipitation forecasts. The approach by Kaparakis et al. involved a two-
stream architecture, with each stream featuring a distinct U-Net. One stream
takes precipitation maps as input, while the other processes wind speed maps.
The outputs from both streams are combined, and a 1x1 3D convolution is
applied, followed by linear activation to generate the final prediction. To
evaluate the effectiveness of their approach, they compared it with four other
models, including the persistence model, core UNet [20],
AsymmetricInceptionRes3DDR-UNet [26], and Broad-UNet [27], which did not
incorporate wind data. Their findings conclusively demonstrated that WF-UNet
outperformed all other models, highlighting the significant improvement in
prediction performance achieved through the inclusion of wind data.

Generative adversarial models (GANs) also offer significant potential for
enhancing precipitation forecasts, particularly for higher rain intensities
and for generating clearer predictions. Ravuri et al. introduced a novel
approach called the deep generative model of radar (DGMR) [14], which
demonstrated improved performance in these aspects compared to the
conventional UNet framework. DGMR leverages a generator that utilizes
convolutional gated recurrent units (GRUs) for enhanced modeling.
Additionally, DGMR uses two discriminators, each focusing on distinct aspects
of image quality assessment. One discriminator evaluates the temporal
consistency, ensuring that predictions maintain a coherent progression over
time. The other discriminator examines spatial consistency, ensuring that the
generated images align with spatial patterns accurately. This approach
contributes to more accurate and less blurry precipitation forecasts,
particularly for higher rain intensities.

For any warning system, understanding the confidence level of predictions is
crucial. Unlike Bayesian models, deep learning tools lack a solid mathematical
foundation for assessing model uncertainty. Foldesi et al. [28] investigated
this issue by evaluating various methods for quantifying uncertainty in deep
learning models, particularly when dealing with time-series data. Their
findings highlighted two effective approaches: ensemble methods [29] and
Dropout[30]/DropConnect[31] techniques. Ensemble methods involve training
multiple copies of the same model architecture and combining their
predictions. Lakshminarayanan et al. [31] demonstrated that ensembles offer
reliable uncertainty quantification properties, making them a valuable choice.
On the other hand, the dropout approach generates a posterior distribution
during testing by running the model for a set number of epochs ($k$) for each
target precipitation map. An uncertainty map is then derived by calculating
the variance across all $k$ precipitation maps. This method has been
effectively employed in studies such as [32] and [21]. DropConnect is
comparable to Dropout, with the distinction that it sets weights to zero
rather than activations. These methods provide valuable tools for assessing
uncertainty in deep learning models, a critical aspect for the reliability of
warning systems.

##  3 Methods

We propose the GA-SmaAt-GNet architecture, a generative adversarial model that
builds upon the SmaAt-UNet architecture [18]. The SmaAt-UNet architecture, an
extension of the UNet model, enhances the original design by introducing
Convolutional Block Attention Modules (CBAM) [33] in the encoder to emphasize
important features and depthwise-separable convolutions to reduce parameter
count. These enhancements make SmaAt-UNet particularly well-suited for time
series prediction tasks, facilitating precise value predictions for each pixel
over time. The model’s efficiency and attention-driven feature selection
contribute to its improved performance. In what follows, we will first
introduce our proposed generator architecture, SmaAt-GNet. Then we will
explain the loss function and our proposed discriminator architecture.

###  3.1 Proposed GA-SmaAt-GNet

We first introduce SmaAt-GNet (see Fig. 1(b)), which will serve as the
generator of GA-SmaAt-GNet shown in Fig. 1. SmaAt-GNet builds upon SmaAt-UNet
by incorporating a secondary stream encoder. This encoder facilitates the
integration of an extra data source, specifically binary precipitation masks
(refer to Section 4.2 for a comprehensive explanation), into the model. This
additional encoder has the same architecture as the original SmaAt-UNet
encoder. The CBAM layer results at each network level of the second encoder
stream are concatenated with the CBAM layer results from the original encoder
at the same network levels. These concatenated results are then connected to
the decoder through skip connections. This design ensures that each network
level can leverage features from both data sources, while also enabling
attention-based feature selection within each encoder. By sharing the same
decoder, the network can uncover relationships between features from both
streams, while the features themselves are learned independently. We also
include two dropout layers with a dropout probability of 0.5 after the first
two bilinear upsampling layers in the decoder to prevent overfitting and they
also allow us to approximate the model’s uncertainty as described in Section
5.2.1.

Our proposed GA-SmaAt-GNet (see Fig. 1) is a conditional Generative
Adversarial Network (cGAN) that merges the principles of SmaAt-GNet and the
attributes of GANs to produce outputs that closely mimic real data by
employing a discriminator to encourage the generator to produce real looking
outputs. A Conditional Generative Adversarial Network is a kind of GAN that
incorporates conditional information during the training process. In a
standard GAN, the generator learns to generate data without any specific
control over the characteristics of the generated output. In contrast, a cGAN
introduces conditional information, typically in the form of additional input
data that guides the generation process. This additional information can be in
the form of labels, class information, or in our case, precipitation maps. By
providing this conditional information to the generator and discriminator
during training, cGANs can be more directed in generating specific outputs.

In general, cGANs optimize performance through an adversarial loss function,
defined as follows:

$\displaystyle\mathcal{L}_{\text{cGAN}}(G,D)=$
$\displaystyle\frac{1}{n\ell}\sum_{i=1}^{n}\sum_{j=1}^{\ell}\log[D(x_{i},y_{i})]_{j}+$
(1) $\displaystyle\log(1-[D(x_{i},G(x_{i},m_{i})]_{j}),$

where $x_{i}$ represents the input sequence, $y_{i}$ is the ground truth,
$m_{i}$ are the precipitation masks, $n$ is the number of samples,
$[D(\cdot,\cdot)]_{j}$ indicates the $j$th element of the $4\times 4$ matrix
$[D(\cdot,\cdot)]$. The generator ($G$) seeks to minimize
$\mathcal{L}_{\text{cGAN}}$, while the discriminator ($D$) aims to maximize
it. Previous methodologies have demonstrated the advantage of combining the
GAN objective with a regularization term, such as L1 or L2 distance [34, 35].
In our case, we opt for L2 regularization over L1 regularization, given its
greater sensitivity for extreme cases due to its squaring operation, which
aligns with our focus.

$\displaystyle\mathcal{L}_{\text{L2}}(G)=\frac{1}{n\kappa}\sum_{i=1}^{n}\sum_{j=1}^{\kappa}[y_{i}-G(x_{i},m_{i})]^{2}_{j},$
(2)

where $y_{i}$ is the ground truth and $[G(\cdot,\cdot)]_{j}$ indicates the
$j$th element of the $12\times 64\times 64$ tensor $[G(\cdot,\cdot)]$. The
final objective function then becomes:

$\displaystyle\min_{G}\max_{D}\mathcal{L}_{\text{GA-SmaAt-GNet}}(G,D)=$
$\displaystyle\mathcal{L}_{\text{cGAN}}(G,D)+$ (3)
$\displaystyle\lambda\mathcal{L}_{\text{L2}}(G),$

where $\lambda$ is a hyperparameter used to balance the GAN loss and
regularization term.

It is crucial to note that we apply the regularization term exclusively when
optimizing the generator, while for the discriminator, the default
$\mathcal{L}_{\text{cGAN}}$ from Eq. (1) is optimized.

The dropout layers in the SmaAt-GNet architecture prevent the generator from
becoming a simple deterministic mapping function. An alternative method to
introduce randomness is by concatenating an image with random noise in the
model. However, as noted in [34], a model may simply learn to ignore this
noise.

Our proposed discriminator architecture is shown in Fig. 1(c) and is an
attention-augmented version of the Pix2Pix’s PatchGAN discriminator [34]. The
PatchGAN architecture consists of several convolutional layers that process
the input image in a hierarchical manner. Instead of classifying the entire
image as real or fake, PatchGAN focuses on classifying small, overlapping
patches of the input image. It also uses convolutions with a stride of two to
gradually reduce the spatial resolution of the feature maps, helping it to
capture information at different scales. We extend the PatchGAN architecture
by adding an additional convolution before each convolution in the default
PatchGAN architecture to prevent underfitting of the discriminator caused by a
lack of model complexity. These added convolutions have a stride of one so
that the spatial resolution does not decrease to quickly. Furthermore, after
each convolution, a Convolutional Block Attention Module (CBAM) layer is added
to focus on relevant parts of the features generated by the preceding
convolutional layer.

(a) General overview of the GA-SmaAt-GNet architecture

(b) Generator: SmaAt-GNet

(c) Attention-Augmented Discriminator

Figure 1: General overview of proposed (a) GA-SmaAt-GNet architecture with
mask generation, (b) SmaAt-GNet used as generator, and (c) the Attention-
Augmented Discriminator architecture.

###  3.2 Training

We conduct a comparative evaluation of our proposed GA-SmaAt-GNet and SmaAt-
GNet against the SmaAt-UNet model [18] and the persistence model. The
persistence model uses the last input image of the input sequence repeated 12
times as its prediction.

Training is performed on the precipitation dataset specified in Section 4.1.
The maximum epoch limit is set to 200, although in practice, this limit was
never reached. To enhance training efficiency and prevent overfitting, we
employed the same early stopping criterion as in [18], where training halts if
there is no improvement on the validation set for more than 15 epochs.

For all models, we initialized the learning rate at 0.001, following the lines
of [18]. To prevent overfitting of GA-SmaAt-GNet’s discriminator, we adjusted
the backpropagation frequency of the discriminator to once every two
iterations. Additionally, a learning rate scheduler was implemented, reducing
the learning rate to a tenth of its size when no improvement occurred on the
validation set for four consecutive epochs. GA-SmaAt-GNet featured separate
learning rate schedulers for the discriminator and generator, monitoring their
respective losses on the validation set. The Adam optimizer is utilized as
optimizer [36] with a batch size of 32 and Mean Squared Error (MSE) served as
the loss function for SmaAt-UNet and SmaAt-GNet. For GA-SmaAt-GNet, the loss
function in Eq. (3) is used with an empirically found $\lambda$-value of
$10^{6}$. All models are trained on an Apple M2 Ultra with 128GB of memory.
The implementation of the models is available on
GitHub111https://github.com/EloyReulen/GA-SmaAt-GNet where there is also more
information about pre-trained models, which are available upon request.

###  3.3 Evaluation

MSE serves as the metric for assessing the performance of all models in terms
of regression and is calculated as follows:

MSE
$\displaystyle=\frac{1}{n\kappa}\sum^{n}_{i=1}\sum^{\kappa}_{j=1}[y_{i}-\hat{y}_{i}]^{2}_{j},$
(4)

where $n$ represents the number of samples, $y_{i}$ is the ground truth and
$\hat{y}_{i}$ is the model’s prediction. Here, $y_{i}$ and $\hat{y}_{i}$ are
tensors of size $12\times 64\times 64$ and $[y_{i}-\hat{y}_{i}]^{2}_{j}$
indicates the $j$th element of the tensor $[y_{i}-\hat{y}_{i}]^{2}$. In
addition to MSE, we evaluate model performance using various binary
classification metrics. To this end, precipitation maps are transformed into
binary representations by denormalizing and summing up 12 consecutive frames,
where 12 is the number of output frames of the models. Subsequently, values
exceeding specific thresholds (0.5, 10, and 20 mm/h) are set to one, while
others are set to zero. This enables performance assessment for both higher
and lower precipitation intensities. The thresholds of 0.5 mm/h (used to
distinguish between rain and no rain) aligns with [18, 12]. From these binary
maps, true positives (TP) (prediction=1, target=1), false positives (FP)
(prediction=1, target=0), true negatives (TN) (prediction=0, target=0), and
false negatives (FN) (prediction=0, target=1) are calculated. Binary
classification metrics used include F1 score, Critical Success Index (CSI),
Heidke Skill Score (HSS), and Matthews Correlation Coefficient (MCC).

The F1 score is a metric commonly used in binary classification tasks to
evaluate the model’s performance by considering both precision and recall.
Precision is the ratio of true positive predictions to the total predicted
positives, while recall is the ratio of true positive predictions to the total
actual positives. The F1 score is the harmonic mean of precision and recall,
providing a balance between the two:

$\begin{split}\text{Precision}&=\frac{TP}{TP+FP},\\\
\text{Recall}&=\frac{TP}{TP+FN},\\\ \text{F1
score}&=\frac{2\times\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}},\end{split}$
(5)

and it is particularly valuable in scenarios where there is an uneven
distribution between classes, and both false positives and false negatives
carry significance.

The Critical Success Index,

$\text{CSI}=\frac{TP}{TP+FN+FP},$ (6)

is fundamental in evaluating the performance of detection and classification
systems and is often used in meteorology. It provides a comprehensive
evaluation, considering both correct and incorrect classifications.

Heidke Skill Score,

$\begin{split}&\text{HSS}=\\\ &\frac{TP\times TN-FP\times
FN}{(TP+FN)(FN+TN)+(TP+FP)(FP+TN)},\end{split}$ (7)

is also a valuable metric for evaluating the skill of binary classification
models, providing a more nuanced assessment than simple accuracy. It is
especially useful in scenarios where both positive and negative outcomes are
important, and it helps to understand the model’s performance beyond what
could be achieved by chance alone, as a value smaller than 0 for this metric
indicates that it performs worse than a random guessing model.

Additionally, the Matthews Correlation Coefficient,

$\begin{split}&\text{MCC}=\\\ &\frac{TP\times TN-FP\times
FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}},\end{split}$ (8)

is a metric that is especially useful in situations where class imbalances are
present. As it offers a balanced assessment of a model’s performance,
considering both correct and incorrect predictions.

##  4 Experiments

###  4.1 Dataset

Figure 2: Preparation of the raw precipitation data. All pixels where there is
no data available are set to zero and a center 64 by 64 cutout is taken from
the center of the land mass of the Netherlands.

We train and test the models on 25 years of precipitation data from the Royal
Netherlands Meteorological Institute (Koninklijk Nederlands Meteorologisch
Instituut, KNMI). The dataset is a climatological radar rainfall dataset of 5
minute precipitation depths at a 2.4-km grid, which have been adjusted with
validated and complete rain gauge data from both KNMI rain gauge networks. The
first network is an automated rain gauge network of 32 stations and the second
network is a manual rain gauge network of about 325 stations. The collected
dataset contains data from 1998 till 2022. Of all 25 years of data, years
1998-2016 are used for the training set and years 2017-2022 are used for the
test set.

Each value in the precipitation map is the precipitation dept in hundredths of
millimeters (so that the values are integers). To obtain precipitation values
in mm/h pixels must be multiplied by 0.01 and 12 subsequent frames must be
summed up.

The raw images in the dataset have a dimension of 256 by 256 pixels that
contain precipitation data for a geographical area bounded by a north latitude
of 53.7, an east longitude of 7.4, a south latitude of 50.6 and a west
longitude of 3.2, which includes the entire Netherlands. Only pixels that
specify an area on the land mass of the Netherlands contain data. For this
reason, all pixels with no data are set to 0 and a center cutout of 64 by 64
pixels is taken from the land mass of the Netherlands as shown in Fig. 2. The
data is normalized by dividing both the values in the training and test set by
the highest value found in the training set. Extremely high values can occur
in the data. These are mostly non-meteorological echos. Non-meteorological
echos occur when radar bundles are redirected back to the surface by large
temperature and humidity gradients. To filter out most non-meteorological
echos all pixels with a year sum larger than 1300 mm or 24 hour sum larger
than 174 mm can be discarded.

The input for the models is a sequence of 12 precipitation maps which
corresponds to one hour of precipitation data. This input length allows for a
fair comparison with SmaAt-UNet [18] which also used 12 precipitation maps as
input. The output consists of the next 12 precipitation maps. As described in
[18] many of the precipitation maps contain little to no rain data which can
cause the models to become biased towards predicting zero values. To prevent
this from happening, only sequences of which the output frames contain more
than 50% rainy pixels are selected similar to the approach by [18]. This 50%
is calculated only on the pixels that contain precipitation data. The sample
sizes of the preprocessed dataset and the original dataset are compared in
Table 1. The preprocessed dataset is available upon request, for more details
see the GitHub page of this paper.

Table 1: Comparison of the dataset sizes of the preprocessed dataset and the original dataset Required rain pixels | Train | Test | Subset
---|---|---|---
0% (original) | 1986452 | 630864 | 100%
50% | 51021 | 15293 | 2.53%
Figure 3: An example of the precipitation mask generation process. For mask
generation 12 consecutive radar frames are summed up to obtain the accumulated
precipitation of one hour. 25 masks are generated from this accumulated
precipitation, one for each integer threshold between 1 and 25 mm/h.

###  4.2 Precipitation masks

In the case of the SmaAt-GNet model, in addition to precipitation maps, a
collection of binary masks is used as input. These masks are made by
binarizing the accumulated rainfall over a one-hour period, as illustrated in
Fig. 3. Each pixel in the mask receives a value of 1 if the accumulated
rainfall surpasses the specified threshold, and 0 otherwise. Each mask aligns
with a distinct precipitation threshold in mm/h, ranging from 1 to 25 mm/h,
resulting in a set of 25 masks for a 1 hour sequence. This augmentation
supplies supplementary information to the models, pinpointing areas of
specific precipitation intensities.

##  5 Results and discussion

Following the training procedure detailed in Section 3.2, the model with the
lowest validation error (MSE) for each architecture was chosen for performance
evaluation on the precipitation test set. For each test case, the models are
run 10 times with unfixed dropout layers. The mean of these 10 results serves
as the prediction. This process makes it also possible to obtain uncertainty
for the models and will be discussed in more detail in Section 5.2.1.
Additionally, all models are compared against the SmaAt-UNet model and the
persistence model which serves as the baseline.

###  5.1 Precipitation nowcasting evaluation

The obtained results on the precipitation test set are presented in Table 2
and Fig. 4. It is important to note that these results are computed after
denormalization, similar to those outlined by [18]. The results in Table 2
reveal that all models perform significantly better than the persistence model
in terms of Mean Squared Error (MSE). Furthermore, we observe that the
introduced SmaAt-GNet architecture leads to a slight reduction in MSE on the
test set when compared to SmaAt-UNet. However, the lowest MSE of all models is
observed for GA-SmaAt-GNet.

In addition to the overall MSE on the test set, we computed MSE per predicted
time step, as illustrated in Fig. 4. It is evident that all models exhibit
similar behavior as time progresses, i.e., MSE increases for predictions
further into the future. However, when comparing SmaAt-GNet to SmaAt-UNet, it
becomes clear that the overall performance gain of SmaAt-GNet can primarily be
attributed to the first half-hour of the prediction.

Table 2: MSE, and several binary metrics calculated on the test set with
different thresholds. A $\uparrow$ means that higher values for this metric
are better while a $\downarrow$ means that lower values are better for this
metric.

Threshold | Model | MSE $\downarrow$ | F1 score $\uparrow$ | CSI $\uparrow$ | HSS $\uparrow$ | MCC $\uparrow$
---|---|---|---|---|---|---
$\geq$ 0.5 mm/h | Persistence | 0.02297 | 0.79121 | 0.65454 | 0.30640 | 0.62084
SmaAt-UNet | 0.01038 | 0.88785 | 0.79832 | 0.38372 | 0.76859
SmaAt-GNet | 0.01021 | 0.89108 | 0.80356 | 0.38464 | 0.77305
GA-SmaAt-GNet | 0.00990 | 0.88470 | 0.79324 | 0.37375 | 0.75842
$\geq$ 10 mm/h | Persistence | - | 0.14543 | 0.07842 | 0.07179 | 0.16843
SmaAt-UNet | - | 0.24236 | 0.13789 | 0.12093 | 0.28717
SmaAt-GNet | - | 0.26871 | 0.15521 | 0.13408 | 0.30433
GA-SmaAt-GNet | - | 0.30762 | 0.18177 | 0.15352 | 0.33612
$\geq$ 20 mm/h | Persistence | - | 0.01944 | 0.00982 | 0.00969 | 0.04146
SmaAt-UNet | - | 0.00082 | 0.00041 | 0.00041 | 0.00280
SmaAt-GNet | - | 0.05475 | 0.02815 | 0.02737 | 0.09524
GA-SmaAt-GNet | - | 0.08264 | 0.04310 | 0.04131 | 0.10266

Note: A - indicates that the value is the same as for the previous threshold.
This is the case for MSE as it is not calculated on binarized data.

Figure 4: MSE per time step calculated on the precipitation test set. The
persistence model is not shown to improve visibility.

(a)

(b)

(c)

(d)

Figure 5: Binary metrics per season for a threshold of 10 mm/h calculated on
the test set. Given a specific metric, a $\uparrow$ indicates that higher
values for that metric are better.

Furthermore, we computed various binary metrics, as detailed in Section 3.3,
to assess model performance across different precipitation intensities. The
results presented in Table 2 reveal that different models perform better for
different thresholds.

For a threshold of 0.5 mm/h, SmaAt-GNet demonstrates superior performance in
all binary metrics, closely followed by SmaAt-UNet and GA-SmaAt-GNet. GA-
SmaAt-GNet performs similar to the SmaAt-UNet in terms of F1 and CSI and
slightly worse in terms of HSS and MCC. However all models, significantly
outperform the persistence model at this threshold.

At a heavier precipitation intensity with a threshold of 10 mm/h, the
persistence model performs exceptionally poor for all binary metrics. SmaAt-
GNet demonstrates an improvement over SmaAt-UNet for all metrics at this
threshold. GA-SmaAt-GNet exhibits superior performance compared to all other
models, outperforming them for all metrics.

In the scenario of extreme precipitation, defined by an accumulated rainfall
exceeding 20 mm in an hour, one can observe that SmaAt-UNet performs worse
than the persistence model for all metrics. GA-SmaAt-GNet demonstrates
superior performance for all binary metrics. In addition, we note that SmaAt-
GNet significantly outperforms the persistence model and SmaAt-UNet in terms
all binary metrics, however it is not able to reach the level of performance
of the GA-SmaAt-GNet.

Figure 6: Example of precipitation nowcasting on a case with more than 20 mm
of rain in one hour using the examined models.

Furthermore, we compute binary metrics per season to provide insights into
model performance across different seasons. Fig. 5 illustrates the binary
metrics per season for a precipitation intensity of 10 mm/h. This threshold
was chosen as no significant differences were observed for threshold of 0.5
mm/h, and for a threshold of 20 mm/h some seasons experienced a shortage of
positive cases, making it less suitable for meaningful comparisons. We note
that across all metrics, the tested models tend to perform less effectively in
winter compared to other the other seasons. This can possibly be attributed to
the scarcity of positive heavy precipitation cases in the training set during
winter. Moreover, heavy precipitation in winter tends to have a different
nature, being characterized by prolonged and widespread events whereas heavy
summer precipitation is often local and short-lived [3]. When comparing SmaAt-
UNet and SmaAt-GNet, we note an interesting observation. SmaAt-UNet exhibits
slightly superior performance in spring and autumn for all metrics, while
SmaAt-GNet notably outperforms in summer. This is particularly interesting as
summer is the season marked by the most intense rain showers [3]. However, the
standout performer in Fig. 5 is GA-SmaAt-GNet which significantly outperforms
all models for most metrics in all seasons with the exception of winter.

Figure 7: Epistemic uncertainty quantification results per time step
calculated on the precipitation test set. The mean epistemic uncertainties
over all time steps are indicated by the dotted lines. The persistence model
is not shown as it has no uncertainty.

(a) Locations of the North Sea, Markermeer and Ijselmeer.

(b) 10 min ahead

(c) 30 minutes ahead

(d) 50 minutes ahead

Figure 8: Epistemic uncertainty of the GA-SmaAt-GNet model for 10, 30 and 50
minutes ahead. From left to right the images are the ground truth, model
prediction and epistemic uncertainty respectively. The map in (a) indicates
areas of relevance for our analysis of epistemic uncertainty. Note that the
values in this figure are not denormalized.

We also provide a visual comparison of the models on a specific case from the
test set where the accumulated rainfall exceeded 20 mm. The model outputs for
this case are depicted in Fig. 6. As anticipated, when compared with the
ground truth, all models exhibit decreased accuracy as the predicted frame
extends further into the future. Upon closer examination of the last two
frames predicted by SmaAt-UNet, we note the following observation: it
overestimates the rain intensity in the bottom left corner more than the other
models. Additionally, in the prediction, the high intensity rain cluster
spreads out over time and is almost unnoticeable in the prediction at the 60
minute mark, while in the ground truth the intensity stays the roughly same.

Moving to SmaAt-GNet’s predictions, the spread of prediction is less
pronounced than that of SmaAt-UNet, but it fails to fully capture the upward
movement of the high-intensity rain cluster evident in the ground truth.

The standout performer in capturing this movement is GA-SmaAt-GNet. The
predictions of GA-SmaAt-GNet for 40, 50, and 60 minutes ahead showcase the
high-intensity rain cluster moving significantly farther north than
predictions from other models, aligning more closely with the ground truth.
Additionally, for the first half hour, the prediction of the intensity of the
high intensity rain cluster is sharper and the intensity resembles that of the
ground truth more closely. Furthermore, the shape of the entire precipitation
field is more similar to the ground truth than the other model, which is most
noticeable for 50 and 60 minutes ahead.

###  5.2 Uncertainty analysis

(a) Mean epistemic uncertainty per season

(b) Mean aleatoric uncertainty per season

(c) Mean precipitation per season

Figure 9: The obtained mean epistemic uncertainty, mean aleatoric uncertainty
and mean precipitation per season calculated on the test set. Note that here
the values are not denormalized.

In Bayesian modeling the consideration of uncertainty is critical and it is
present in two primary forms [37]. First, aleatoric uncertainty which is the
inherent noise present in observations, such as sensor noise. This form of
uncertainty persists even when the volume of data is increased. Aleatoric
uncertainty is further classified into homoscedastic uncertainty, which
remains constant across different inputs, and heteroscedastic uncertainty,
where the degree of uncertainty depends on the inputs to the model.
Heteroscedastic aleatoric uncertainty holds particular significance in
computer vision applications.

Secondly, epistemic uncertainty, often referred to as model uncertainty,
addresses the uncertainty embedded in the model parameters. Unlike aleatoric
uncertainty, epistemic uncertainty has the potential to decrease when
expanding the dataset.

####  5.2.1 Epistemic uncertainty

We approximate epistemic uncertainty by employing test-time dropout (TTD), a
method used for estimating Bayesian inference, as demonstrated by [21, 32].
With TTD, dropout layers are not fixed at test time like they normally would
be. This intentional randomness during testing provides insights into the
model’s uncertainty. We perform TTD for $k=10$ epochs, generating a posterior
distribution which results in 10 sets of 12 precipitation maps. Subsequently,
we calculate uncertainty maps by determining the variance across the 10
precipitation maps for each of the 12 predicted timesteps, resulting in 12
uncertainty maps, each corresponding to a predicted timestep. If, instead of
variance, we calculate the mean, we obtain the model’s prediction.
Additionally, if we take the mean of all uncertainty maps generated by the
model we get an overall measure of uncertainty on the entire test set.

Upon examining the temporal evolution of epistemic uncertainty in Fig. 7, it
becomes evident that, across all models, epistemic uncertainty tends to rise
as predictions project further into the future. Notably, a point of interest
emerges when comparing GA-SmaAt-GNet and SmaAt-UNet: despite GA-SmaAt-GNet
starting with higher certainty at 5 minutes ahead, its uncertainty escalates
more rapidly than that of SmaAt-UNet. However, in the time span from 30 to 45
minutes ahead, GA-SmaAt-GNet’s epistemic uncertainty experiences a slight
reduction, bringing it closer to that of SmaAt-UNet.

Besides evaluating the epistemic uncertainty for all models on the test set,
we also visually inspect the epistemic uncertainty for a particular case, with
a specific emphasis on the proposed GA-SmaAt-GNet model. Analyzing the
epistemic uncertainty in Fig. 8, we observe that the model tends to exhibit
higher uncertainty in regions where it predicts more intense rainfall.
Additionally, the area of uncertainty expands as the predictions become more
spread out, particularly noticeable in the transition from 30 to 50 minutes
ahead. For predictions further into the future, particularly evident at 50
minutes ahead in Fig. 8(d), the model displays slightly higher uncertainty
around the edges of water. In specific areas of the top half of the image,
where the uncertainty is generally low, a faint line of increased uncertainty
appears along the northern coast and the edges of the Ijselmeer and Markermeer
(see Fig. 8(a)). This phenomenon could be attributed to the absence of data
above water in the dataset. Consequently, rain may abruptly appear and
disappear at the water’s edges, potentially explaining the model’s uncertainty
in these specific regions.

The mean epistemic uncertainties for all models on the test are indicated by
the dotted lines in Fig. 7. The results reveal that SmaAt-UNet is the most
certain of its predictions closely followed by GA-SmaAt-GNet. SmaAt-GNet,
conversely, showcases the highest uncertainty in its predictions.

Figure 10: Examples of aleatoric uncertainty of the data obtained with the
SmaAt-GNet model. The images on the left are the actual observed precipitation
and the image on the right is the predicted aleatoric uncertainty. Note that
the values in this figure are not denormalized.

(a) Input, ground truth and prediction

(b) Encoders

(c) Decoder

Figure 11: Activation heatmaps obtained using Grad-CAM on all pixels predicted
with rain after binarization. The one hour accumulated precipitation of the
input, ground truth and model prediction are shown in (a). The activation
heatmaps of the encoders can be found in (b) and the activation heatmaps of
the decoder can be found in (c). Each row represents a level (depth) of the
encoder or decoder of the GA-SmaAt-GNet generator. For the encoders we show
activation heatmaps for the double depthwise-seperable convolution (DSC)
blocks and CBAM layers and for the decoder we show the activation heatmaps of
the DSC blocks.

Furthermore, we examine the epistemic uncertainty across seasons using the GA-
SmaAt-GNet model, as illustrated in Fig. 9. An interesting finding comes to
light, indicating that epistemic uncertainty reaches its highest point during
the summer season. This coincides with the observation that mean precipitation
is at its peak during this season, as depicted in Fig. 9(c). The increased
occurrence of sudden, heavy rain showers during summer, as noted in [3], is
likely a contributing factor to the heightened epistemic uncertainty. Notably,
it is intriguing to observe that, despite higher precipitation levels in
winter, epistemic uncertainty tends to be lower compared to spring. This
phenomenon may be attributed to the specific precipitation patterns
characterizing winter, involving longer-lasting, lower-intensity rainfall
events on a larger scale, as highlighted in [3]. Consequently, these patterns
suggest that epistemic uncertainty is more influenced by precipitation
intensity than the overall amount, supporting our observations derived from
the illustrative examples showcased in Fig. 8.

####  5.2.2 Aleatoric uncertainty

To obtain heteroscedastic aleatoric uncertainty we require a neural network
that maps the input to both an output and a measure of aleatoric uncertainty
given by predicted variance, $\hat{\sigma}_{i}^{2}$, as described by Kendall
et al. [37]. The predicted variance $\hat{\sigma}_{i}^{2}$ is then implicitly
learned from the loss function. Instead of learning $\hat{\sigma}_{i}^{2}$
directly, the log variance $s_{i}:=\log{\hat{\sigma}_{i}^{2}}$ is learned to
improve training stability by avoiding a potential division by 0. The loss
function that is used to obtain aleatoric uncertainty can be defined as:

$\displaystyle a_{i}$
$\displaystyle=\frac{1}{2}\exp(-s_{i})(y_{i}-\hat{y}_{i})^{2}+\frac{1}{2}s_{i},$
(9) $\displaystyle\mathcal{L}_{\text{Aleatoric,MSE}}$
$\displaystyle=\frac{1}{n\kappa}\sum_{i=1}^{n}\sum_{j=1}^{\kappa}\left[a_{i}\right]_{j},$

where $n$ represents the number of samples, $y_{i}$ is the ground truth and
$\hat{y}_{i}$ is the prediction. Here, $a_{i}$, $s_{i}$, $y_{i}$ and
$\hat{y}_{i}$ are all tensors of size $12\times 64\times 64$ and $[a_{i}]_{j}$
indicates the $j$th element of $[a_{i}]$. The predicted aleatoric uncertainty
is then obtained using $\hat{\sigma}_{i}^{2}=\exp(s_{i})$.

To acquire aleatoric uncertainty, we make a slight modification to the SmaAt-
GNet architecture. Specifically, we introduce a modification to include the
prediction of $s_{i}$. We therefore add an extra 1x1 convolution at the end of
the network so that the model has 2 identically shaped outputs, one for the
actual predictions and one for $s_{i}$. This means that each pixel of the
output has a corresponding value which indicates the uncertainty for that
pixel.

We investigate aleatoric uncertainty through visual examples showcased in Fig.
10. Across all examples, a clear pattern emerges, revealing heightened
uncertainty surrounding regions with higher precipitation intensity. Despite
the possible reduction of epistemic uncertainty with a larger dataset, the
elevated aleatoric uncertainty persists in these high-intensity regions. This
increased aleatoric uncertainty poses additional challenges when predicting
cases characterized by high-intensity precipitation compared to cases with
lower intensity precipitation.

Additionally, we explore aleatoric uncertainty across seasons, as illustrated
in Fig. 9. Similar to our observation for epistemic uncertainty, aleatoric
uncertainty is also highest in summer, where mean precipitation is highest.
Furthermore, in winter, mean precipitation is higher compared to spring, yet
aleatoric uncertainty is lower, aligning with the observation we made for
epistemic uncertainty. Thus, the patterns here also suggest that aleatoric
uncertainty is more influenced by precipitation intensity than the overall
amount, which aligns with our findings from the examples shown in Fig. 10.

###  5.3 Activation heatmaps

Explaining the outcomes of deep learning models is essential, and following
the approach in [22], we use Grad-CAM [38] to generate activation heatmaps.
These heatmaps highlight the input regions influencing high values in the
output, offering valuable insights into the decision-making process of the
model. Originally designed for visual explanations in classification networks,
Grad-CAM calculates the gradient of the class score with respect to feature
map activations to determine the importance weights of neurons. It then
combines these weights linearly with the forward activation maps, followed by
ReLU, to produce the final outcome. To generate activation heatmaps for our
nowcasting task, we convert it into an image segmentation task by binarizing
the output using a threshold of 0.5 mm/h in the same way we did for our binary
evaluation metrics.

In Fig. 11, we present the activation heatmaps of the double depthwise-
separable convolution (DSC) blocks and Convolutional Block Attention Modules
(CBAMs) at different levels (depths) of the GA-SmaAt-GNet network for a heavy
precipitation case.

Examining the encoder activation heatmaps in Fig. 11(b), we notice distinct
patterns. The precipitation map encoder’s heatmaps exhibit higher activation
at depths 0 and 1 in regions corresponding to elevated precipitation levels in
the ground truth. In contrast, the precipitation mask encoder’s heatmaps at
depth 0 show increased activation in areas with higher precipitation in the
model input, while at depth 1, it shows increased activation around the high
precipitation area in the input. As we move to depths 2, 3, and 4, the
activation areas for both encoders become more abstract and less directly
linked to regions of higher precipitation, an observation also made by [22] at
lower levels of their architecture. When comparing both encoders at all
depths, we see that they are generally activated in different areas,
suggesting complementary contributions to the prediction. We can also observe
the effects of the CBAM layers if we compare the double DSC activation maps
and the CBAM activation maps. We can observe in which areas the CBAM layers
emphasize and de-emphasize the double DSC activations. As an example, consider
the precipitation map encoder at depth 1; after the application of the CBAM
layer, the region corresponding to higher precipitation in the model’s
prediction exhibits increased activation. Additionally, the influence of CBAM
layers is more pronounced in the precipitation mask encoder than in the
precipitation map encoder, evident from the greater differences between the
activation heatmaps of the DSC blocks and the CBAM layers in the precipitation
mask encoder.

Examining the activation heatmaps of the DSC blocks in the decoder (Fig.
11(c)), one can observe that at depths 3 and 2, the activation areas do not
align with regions of high precipitation in the model’s prediction. However,
at depth 1 and 0, areas with more precipitation in the model’s prediction
generally exhibit heightened activation. Notably, at depth 0, the transitions
in the activations of the double DSC block become more gradual compared to the
activations at depth 1, resembling the final prediction closely. This aligns
logically with its position as one of the concluding layers of the network.

##  6 Conclusion and future work

In this paper we introduced GA-SmaAt-GNet, a novel generative adversarial
model for extreme precipitation nowcasting. GA-SmaAt-GNet uses two
methodologies to more accurately nowcast extreme precipitation: the
integration of precipitation masks in a novel SmaAt-GNet architecture and a
generative adversarial approach with an attention-augmented discriminator,
which resulted in a substantial advancement for extreme precipitation
nowcasting. Furthermore, we demonstrated that both aleatoric and epistemic
uncertainty increase with higher precipitation intensities, emphasizing the
inherent challenges associated with the task of extreme precipitation
nowcasting. Finally, we utilized Grad-CAM to generate activation heatmaps for
various parts of our proposed GA-SmaAt-GNet, providing valuable insights into
the model’s predictions.

##  7 Bibliography

## References

  * [1] A. Hering, C. Morel, G. Galli, S. Sénési, P. Ambrosetti, M. Boscacci, Nowcasting thunderstorms in the alpine region using a radar based adaptive thresholding scheme, in: Proceedings of ERAD, Vol. 1, 2004.
  * [2] E. Daniels, G. Lenderink, R. Hutjes, A. Holtslag, Spatial precipitation patterns and trends in the netherlands during 1951–2009, International journal of climatology 34 (6) (2014) 1773–1784.
  * [3] J. Beersma, H. Hakvoort, R. Jilderda, A. Overeem, R. Versteeg, Neerslagstatistiek en -reeksen voor het waterbeheer 2019, https://www.stowa.nl/sites/default/files/assets/PUBLICATIES/Publicaties%202019/STOWA%202019-19%20neerslagstatistieken.pdf.
  * [4] J. M. Eden, S. F. Kew, O. Bellprat, G. Lenderink, I. Manola, H. Omrani, G. J. van Oldenborgh, Extreme precipitation in the netherlands: An event attribution case study, Weather and climate extremes 21 (2018) 90–101.
  * [5] P. Bauer, A. Thorpe, G. Brunet, The quiet revolution of numerical weather prediction, Nature 525 (7567) (2015) 47–55.
  * [6] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, Advances in neural information processing systems 25 (2012).
  * [7] K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
  * [8] K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014).
  * [9] S. Mehrkanoon, Deep shared representation learning for weather elements forecasting, Knowledge-Based Systems 179 (2019) 120–128.
  * [10] Y. Kim, Convolutional neural networks for sentence classification, arXiv preprint arXiv:1408.5882 (2014).
  * [11] X. Zhang, J. Zhao, Y. LeCun, Character-level convolutional networks for text classification, Advances in neural information processing systems 28 (2015).
  * [12] X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-K. Wong, W.-c. Woo, Convolutional lstm network: A machine learning approach for precipitation nowcasting, Advances in neural information processing systems 28 (2015).
  * [13] X. Shi, Z. Gao, L. Lausen, H. Wang, D.-Y. Yeung, W.-k. Wong, W.-c. Woo, Deep learning for precipitation nowcasting: A benchmark and a new model, Advances in neural information processing systems 30 (2017).
  * [14] S. Ravuri, K. Lenc, M. Willson, D. Kangin, R. Lam, P. Mirowski, M. Fitzsimons, M. Athanassiadou, S. Kashem, S. Madge, et al., Skilful precipitation nowcasting using deep generative models of radar, Nature 597 (7878) (2021) 672–677.
  * [15] G. Franch, D. Nerini, M. Pendesini, L. Coviello, G. Jurman, C. Furlanello, Precipitation nowcasting with orographic enhanced stacked generalization: Improving deep learning predictions on extreme events, Atmosphere 11 (3) (2020) 267.
  * [16] R. Wang, L. Su, W. K. Wong, A. K. Lau, J. C. Fung, Skillful radar-based heavy rainfall nowcasting using task-segmented generative adversarial network, IEEE Transactions on Geoscience and Remote Sensing (2023).
  * [17] Y. Zhang, M. Long, K. Chen, L. Xing, R. Jin, M. I. Jordan, J. Wang, Skilful nowcasting of extreme precipitation with nowcastnet, Nature 619 (7970) (2023) 526–532.
  * [18] K. Trebing, T. Stańczyk, S. Mehrkanoon, SmaAt-UNet: Precipitation nowcasting using a small attention-unet architecture, Pattern Recognition Letters 145 (2021) 178–186.
  * [19] KNMI, Zware regen, https://www.knmi.nl/kennis-en-datacentrum/waarschuwingen/zware-regen.
  * [20] O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer, 2015, pp. 234–241.
  * [21] Y. Yang, S. Mehrkanoon, AA-Transunet: Attention augmented transunet for nowcasting tasks, in: 2022 International Joint Conference on Neural Networks (IJCNN), IEEE, 2022, pp. 01–08.
  * [22] M. Renault, S. Mehrkanoon, Sar-unet: Small attention residual unet for explainable nowcasting tasks, in: 2023 International Joint Conference on Neural Networks (IJCNN), 2023, pp. 1–8.  doi:10.1109/IJCNN54540.2023.10191095.
  * [23] D. Han, J. Im, Y. Shin, J. Lee, Key factors for quantitative precipitation nowcasting using ground weather radar data based on deep learning, Geoscientific Model Development 16 (20) (2023) 5895–5914.
  * [24] J. Ko, K. Lee, H. Hwang, K. Shin, Deep-learning-based precipitation nowcasting with ground weather station data and radar data, in: 2022 IEEE International Conference on Data Mining Workshops (ICDMW), IEEE, 2022, pp. 1063–1070.
  * [25] C. Kaparakis, S. Mehrkanoon, WF-UNet: Weather data fusion using 3d-unet for precipitation nowcasting, Procedia Computer Science 222 (2023) 223–232.
  * [26] J. G. Fernández, I. A. Abdellaoui, S. Mehrkanoon, Deep coastal sea elements forecasting using unet-based models, Knowledge-Based Systems 252 (2022) 109445.
  * [27] J. G. Fernández, S. Mehrkanoon, Broad-UNet: Multi-scale feature learning for nowcasting tasks, Neural Networks 144 (2021) 419–427.
  * [28] L. Foldesi, M. Valdenegro-Toro, Comparison of uncertainty quantification with deep learning in time series regression, arXiv preprint arXiv:2211.06233 (2022).
  * [29] B. Lakshminarayanan, A. Pritzel, C. Blundell, Simple and scalable predictive uncertainty estimation using deep ensembles, Advances in neural information processing systems 30 (2017).
  * [30] Y. Gal, Z. Ghahramani, Dropout as a bayesian approximation: Representing model uncertainty in deep learning, in: international conference on machine learning, PMLR, 2016, pp. 1050–1059.
  * [31] A. Mobiny, P. Yuan, S. K. Moulik, N. Garg, C. C. Wu, H. Van Nguyen, Dropconnect is effective in modeling uncertainty of bayesian deep networks, Scientific reports 11 (1) (2021) 5458.
  * [32] P. Natekar, A. Kori, G. Krishnamurthi, Demystifying brain tumor segmentation networks: interpretability and uncertainty analysis, Frontiers in computational neuroscience 14 (2020) 6.
  * [33] S. Woo, J. Park, J.-Y. Lee, I. S. Kweon, Cbam: Convolutional block attention module, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 3–19.
  * [34] P. Isola, J.-Y. Zhu, T. Zhou, A. A. Efros, Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1125–1134.
  * [35] D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, A. A. Efros, Context encoders: Feature learning by inpainting, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2536–2544.
  * [36] D. P. Kingma, J. Ba, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014).
  * [37] A. Kendall, Y. Gal, What uncertainties do we need in bayesian deep learning for computer vision?, Advances in neural information processing systems 30 (2017).
  * [38] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, D. Batra, Grad-cam: Visual explanations from deep networks via gradient-based localization, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 618–626.