# Improved Flood Insights: Diffusion-Based SAR to EO Image Translation

Minseok Seo, Youngtack Oh, Doyi Kim, Dongmin Kang, Yeji Choi
SI Analytics
70, Yuseong-daero 1689beon-gil, Yuseong-gu, Daejeon, Republic of Korea
{minseok.seo, ytoh96, doyikim, dmkang<EMAIL_ADDRESS>
###### Abstract

Driven by rapid climate change, the frequency and intensity of flood events
are increasing. Electro-Optical (EO) satellite imagery is commonly utilized
for rapid response. However, its utilities in flood situations are hampered by
issues such as cloud cover and limitations during nighttime, making accurate
assessment of damage challenging. Several alternative flood detection
techniques utilizing Synthetic Aperture Radar (SAR) data have been proposed.
Despite the advantages of SAR over EO in the aforementioned situations, SAR
presents a distinct drawback: human analysts often struggle with data
interpretation. To tackle this issue, this paper introduces a novel framework,
Diffusion-Based SAR to EO Image Translation (DSE). The DSE framework converts
SAR images into EO images, thereby enhancing the interpretability of flood
insights for humans. Experimental results on the Sen1Floods11 and SEN12-FLOOD
datasets confirm that the DSE framework not only delivers enhanced visual
information but also improves performance across all tested flood segmentation
baselines.

##  1 Introduction

Under global warming conditions, the intensity and frequency of heavy
precipitation and associated flooding events have increased in most regions
[2, 35]. Flooding is one of the most prevalent natural disasters, and its
damage leads to catastrophic consequences, especially in low-income countries
[33]. It is important and urgent to decide where to deploy the necessary
resources to mitigate the damage and quickly recover from a crisis The
allocation of needed resources relies on precise information, which are
collected through both manual and remote means.

Herein, Electro-Optical (EO) satellites have provided a broad and
comprehensive view of the disaster-stricken region, surpassing the scope of
on-site surveys by humans.

Figure 1: Examples of EO images and unprocessed SAR images from a flood-
affected region. (a) depicts an EO image, (b) shows a SAR image, (c) presents
the result of flood area segmentation using SAR imagery, and (d) displays a
SynEO sample generated through the DSE framework.

One significant capability of EO satellites is the use of various optical
channels to capture target images with high spatial resolution.

Satellite-based indexes, such as the Normalized Difference Water Index (NDWI)
[25] and Modified Normalized Difference Water Index (MNDWI) [47], leverages
variances in specific channels to monitor the water bodies and delineate the
flood extent. However, in EO satellite observations, cloud cover has
obstructed the view of the region underneath, and often water and cloud
shadows are misclassified ( Fig. 1-(a)).

It is because the sensors of EO satellites cannot penetrate dense cloud cover,
and unfortunately, most flood events are due to heavy rains accompanied by
thick clouds. Thus, the EO satellites are not suitable for real-time flood
monitoring. As an alternative, approaches that employ Synthetic Aperture Radar
(SAR) data have been proposed [43]. SAR imaging holds the advantage of being
unaffected by cloud cover and can operate under nighttime conditions,
providing a flexible practice for disaster.

However, SAR images often contain impediments to interpretation, such as
speckle noise, as shown in Fig. 1-(b). Hence, although a model appropriately
estimates the inundated region as depicted in Fig. 1-(c), people cannot easily
affirm that model decisions are reliable without the aid of EO image ( Fig.
1-(a)). In response to this issue, we introduce the Diffusion-Based SAR to EO
Image Translation (DSE) framework, an innovative method to generate clean
Synthetic EO (SynEO) images from SAR inputs for enhanced flood monitoring and
mapping. Grounded in the Brownian Bridge Diffusion Model (BBDM) [20], our DSE
framework stands as a robust image-to-image translation model based on the
diffusion process. Furthermore, we incorporate a self-supervised denoising
method to enhance the clarity of generated images, thus improving the
interpretability and usability of SAR images. Concentrating on flood events,
our work is designed to support decision-makers in their rapid and effective
response to these natural disasters. We leverage the DSE framework to generate
EO-like images (SynEO) from SAR observations. As demonstrated in Fig. 1-(d),
this enhanced view of the affected areas, which includes the SAR image and its
corresponding SynEO, offers valuable insight for accurate flood monitoring. To
validate our DSE framework, we conducted assessments on the Sen1Floods11 [4]
dataset. To confirm whether our SynEO truly aids SAR experts, we conducted
experiments using the SEN12-FLOOD [32] dataset. Through our comprehensive
analysis, we found that compared to cloud-free EO images, our SynEO only
exhibited a negligible performance drop of approximately 1%.

Figure 2: Overview of the DSE framework. The DSE framework takes in the SAR
image, applies a self-supervised denoising method, and then carries out
diffusion-based SAR2EO image translation. Subsequently, the generated EO and
corresponding SAR images are reviewed by the analysts for the purpose of flood
mapping.

##  2 Related Work

###  2.1 EO Imagery for Disaster Management

Earth observation through satellite delivered a major breakthrough in disaster
risk management [7]. With the EO satellite images, various types of disasters
and hazards could be monitored and managed. For example, inland flooding [19,
17, 16], subsidence [6], landslides [12], earthquakes and volcanic eruptions
[39], or wildfires [5].

Among those disasters, flood is the single most hydrometeorological hazard
causing substantial losses. [4] For rapid and efficient monitoring of flooding
by EO image, there has been a wide range of methods proposed. Thesholding NDVI
(Normalized Difference Vegetation Index) or NDWI (Normalized Difference Water
Index) based models had been simple yet effective [16]. Hence they are still
utilized as a reference for models or human analysts [4]. Recently, machine
learning based supervised classification algorithms have been used, including
Random Forests [15], decision trees [1], support vector machine (SVM) [38] and
the perceptron model [26], neural networks [41]. Those classification-based
approaches achieve relatively higher accuracy than index thresholding methods
while it requires ground truth data to select appropriate training samples.

The major drawback of using EO images for disaster management is that it
cannot guarantee consecutive monitoring when the Area of Interest (AoI) is
covered by clouds or is in night time. [27]

###  2.2 SAR Imagery for Disaster Management

Compared to the EO-based system, observation through radar is another
important approach that accelerated the era of earth observation and disaster
management. SAR provides a stable monitoring environment even in the case of
cloud-covered or night-time situations. [16]

In a similar sense to EO imagery, SAR has been used to monitor various types
of disasters (e.g., flood [27], landslide [11], tsunami [9], etc.).
Especially, SAR has shown noticeable advantages against EO imagery in studying
water bodies [14]. Widely used water index in EO data has suffered one
drawback: bands associated with the near-infrared and short-saved infrared can
present a loss of resolution compared to the RGB ones.

With SAR imagery, there have been several well-known methods for flood
monitoring. Conventional method, similar with EO imagery, is threshold based
method [46]. Recently, machine learning based algorithms have been proposed
and shown comparable or better performance than threshold based method in
terms of generalization [3].

###  2.3 SAR to EO Image Translation

Despite of high performance of those methods, SAR based model has an intrinsic
weakness; the interpretation of the SAR images is extremely difficult because
of speckle noise and innately different data representation than EO image.
Inevitably, these interpretation difficulties require annotated data [44].
Recent automated disaster monitoring models are based on neural networks,
which means they are heavily data-driven. [28, 40, 29] Therefore, the scarcity
of data is disrupting the potential of SAR in many applications.

To alleviate this disadvantage of SAR datasets, visualization techniques for
SAR imagery have been studied. Most direct and simple methods are re-
normalization methods such as histogram equalization [22] and thresholding-
based methods [21].

Although it helps to recognize the overall structure of the scene, still
detailed features are not easily acquired. One straightforward solution is
denoising the speckle noise [8] while it shows its effectiveness in limited
environments. Hence, recent studies proposed SAR to EO image translation
method in the sense that EO imagery is most straightforward for humans [23].

Generative adversarial Networks (GAN) based Image translation has been
proposed [44, 34]. However, GAN-based models frequently suffer from mode
collapse problems that significantly degrade the quality of generated data.
This is the first application of flood disaster monitoring to our best
knowledge. We propose DSE framework which does not suffer from those issues.

###  2.4 Diffusion Models

For the new scheme of generative model, diffusion models has been recently
highlighted. [13] Diffusion models generate data by sequantial denoising
steps. Initially, diffusion models were treated as Markov chain with Gaussian
noise and recently we interpret it as a stochastic differential equation
(SDE). A remarkable property of this SDE is the existence of an ordinary
differential equation (ODE), dubbed the Probability Flow (PF) ODE by [42].

With the great success in image generation quality, there are recent trials to
use diffusion model as conditional generation model [45]. However, those trial
succeeded in limited application. To handle with the limitation of diffusion
models in image translation task, [20] exploited the formulation of Brownian
bridge to model the stochastic process of image translation in latent space.

In satellite imagery domain, also, diffusion models have been broadly applied
[30]. Our proposed method also get advantage of diffusion model’s generation
power to make SAR more interpretable. For the detailed description of our
proposed method, please refer to the following sections.

##  3 Method

In this section, BBDM [20], which is the basis of the DSE framework, is first
briefly described, and then the preprocessing, model, and function are
sequentially explained in detail. Please note that the reverse process of DSE
aligns perfectly with that of BBDM, so we won’t delve into the details of the
reverse process in this paper.

###  3.1 Brownian Bridge Diffusion Model (BBDM)

Given two datasets, $X_{A}$ and $X_{B}$, originating from domains $A$ and $B$
respectively, the purpose of image-to-image translation is to ascertain a
function that establishes a mapping from domain $A$ to domain $B$. While
numerous image-to-image translation methods based on conditional diffusion
models have been proposed, they are not intuitively suited for the task as its
translation process seamlessly converts a noise back into an image, not image
to image. Moreover they does not have a clear theoretical guarantee because of
their complex conditioning algorithm based on attention mechanism. BBDM,
however, provides a method for image-to-image translation grounded in the
Brownian diffusion process which avoid leveraging complex conditioning
algorithm.

Reffering to the original BBDM, we also conduct the process in the latent
space of VQGAN[10]. Following the convention, let $(x,y)$ denote the paired
training data from $X_{A}$ and $X_{B}$, each. For simplicity, we use $x$ and
$y$ to denote the corresponding latent features ($x:=L_{S}(x),y:=L_{E}(y)$).
The forward diffusion process of Brownian Bridge is defined as:

$\displaystyle q_{BB}(x_{t}|x_{0},y)$
$\displaystyle=\mathcal{N}(x_{t};(1-m_{t})x_{0}+m_{t}y,\delta_{t}I),$ (1)
$\displaystyle\ x_{0}$ $\displaystyle=x,m_{t}=\frac{t}{T}$ (2)

where $T$ is the total steps of the diffusion process, $\delta_{t}$ is the
variance.

The forward diffusion of the Brownian Bridge process provides only the
marginal distribution at each time step $t$, as shown by the transition
probability in (1). However, for training and inference, it is essential to
deduce the forward transition probability $q_{\text{BB}}(x_{t}|x_{t-1},y)$. In
the original BBDM, given an initial state $x_{0}$ and a destination state $y$,
the intermediate state $x_{t}$ can be computed in discrete form as follows:

$\displaystyle x_{t}$
$\displaystyle=(1-m_{t})x_{0}+m_{t}y+\sqrt{\delta_{t}}\epsilon_{t},$ (3)
$\displaystyle x_{t-1}$
$\displaystyle=(1-m_{t-1})x_{0}+m_{t}y+\sqrt{\delta_{t-1}}\epsilon_{t-1}$ (4)

here, $\epsilon_{t}$, $\epsilon_{t-1}\sim\mathcal{N}(0,I)$.

However, in the SAR2EO task, diversity isn’t as crucial as in the original
BBDM. Rather, the emphasis is on prediction that closely aligns with the
actual outcome. For instance, in the SAR2EO task, the goal is to generate
images that are akin to the actual EO image or resemble the distribution of
training EO images, instead of producing a variety of colors and textures like
generation. Consequently, we sample $\epsilon$ from the target distribution
rather than the standard normal distribution $\mathcal{N}(0,I)$. Moreover, in
the reverse process, we set the size of $\epsilon$ to $\epsilon\times 0.1$.
This adjustment brings the SAR2EO task closer to prediction.

###  3.2 Pre-processing

SAR images are intrinsically speckled due to the way they are generated, which
can be captured by the following mathematical model for multiplicative speckle
noise $N$:

$Y=XN,$ (5)

where $Y$ is the observed SAR intensity, $X$ is the speckle-free or clean
image, and $N$ is the speckle noise.

Generally, it is postulated that $N$ conforms to a Gamma distribution,
characterized by a mean of 1 and a variance of $1/L$, where $L$ represents the
number of ‘looks’ in the multi-look process. The probability density function
of this particular distribution can be formulated as follows:

$p(N)=\frac{1}{\Gamma(N)}L^{N}N^{L-1}e^{-LN},$ (6)

where $\Gamma(.)$ is the Gamma function. This formulation gives a more
comprehensive account of the characteristics of speckle noise in SAR images.
As the DSE framework applies a diffusion-based image-to-image translation
model, it is designed to predict both the SAR image and the noise added during
the forward process.

However, the model may struggle to distinguish between the speckle noise
inherent to SAR imaging, as depicted by Eq. 6, and additional noise introduced
during the forward process. Consequently, remnants of noise may still be
present after the DSE framework is applied, as shown in Fig. 3-(b).

Figure 3: Qualitative comparison between original SAR images and SAR images
denoised using the self-supervised method.

To address this issue, we de-noise the SAR image using a blind-spot based
self-supervised denoising method beforehand we leverage them in the image-to-
image translation process. The conventional blind-spot based approaches
inherently assume that the noise is independent of the clean image [18], a
condition not fulfilled by SAR images produced via Eq. 6. To mitigate this, we
applied the [49] method, a variant of blind-spot techniques utilizing diverse
kernels. Fig. 3-(b,d) presents the results of de-noising using the [49] method
and the SAR2EO generation results using the de-noised SAR image with the DSE
framework.

Method | Modality | Metric
---|---|---
Pre.(0/1) | Rec. (0/1) | F1 (0/1) | IoU (0/1) | ACC
U-Net [37] (ResNet34) | SAR | 0.9622/0.4267 | 0.9796/0.2871 | 0.9708/0.3409 | 0.9433/0.2146 | 0.9442
U-Net [37] (ResNet34) | EO | 0.9930/0.7833 |  0.9858/0.8642 |  0.9894/0.8162 |  0.9790/0.6928 | 0.9800
U-Net [37] (ResNet34) | SAR+EO | 0.9894/0.7673 | 0.9840/0.7954 | 0.9867/0.7685 | 0.9455/0.6291 | 0.9749
U-Net [37] (ResNet34) | SAR+SynEO | 0.9862/0.7085 | 0.9817/0.7352 | 0.9838/0.7041 | 0.9683/0.5556 | 0.9695
U-Net [37] (ResNet34) | SynEO |  0.9931/0.4697 | 0.9440/0.8723 | 0.9678/0.6070 | 0.9378/0.4377 | 0.9408

Table 1: Comparison of precision, recall, F1-Score, IoU, and accuracy of flood
segmentation results for different modalities for the SEN12-FLOOD Dataset. In
metric, 0 is the performance when detecting a background, and 1 is the
performance when detecting a flood. Also, in modalities, the + sign means that
each image is concatenated to a channel axis.

Method | PSNR | SSIM | LPIPS
---|---|---|---
Pix2PixHD [45] | 31.09 | 0.81 | 0.116
BBDM [20] | 29.20 | 0.74 | 0.124
DSE | 32.43 | 0.84 | 0.109
DSE+multi-temporal | 34.94 | 0.87 | 0.082

Table 2: Comparison results of the DSE framework with the commonly employed
SAR2EO baselines, pix2pixHD, using a test set derived from the SEN12-FLOOD
dataset, where missing or cloud-affected data points have been excluded.

###  3.3 SAR2EO Image Translation

Figure 2 provides a schematic overview of the DSE framework. DSE accepts VV
and VH channels, transforming them into a 3-channel image in the form of (VV,
VH, (VV+VH)/2.). Subsequently, the image undergoes self-supervised denoising
via the method described in Sec. 3.2 to minimize speckle noise. This processed
image is then fed into the diffusion model-based SAR2EO translation network to
generate a synthetic EO (SynEO) image. Ultimately, through this process, the
synthetic EO (SynEO) image, paired with the corresponding SAR image, is
presented to the SAR experts. This combined imagery provides additional
insights into flood conditions, aiding the evaluator in making a more informed
assessment of the flood extent.

##  4 Experiments

In this section, we conducted experiments on two flood datasets, Sen1Floods11
[4] and SEN12-FLO [31], to evaluate the effectiveness of the DSE framework.
The Sen1Floods11 dataset is characterized by a high presence of clouds, while
the SEN12-FLOOD dataset, a multi-temporal dataset, has fewer clouds. It’s
important to note that for both datasets, we conducted experiments after
removing clouds using the QA60 method [48].

#### Training & Test Datasets

The Sen1Floods11 dataset we used for SAR2EO is a dataset consisting of
Sentinel-1 and Sentinel-2 flood event imagery sourced from Google Earth Engine
[4]. The reference flood maps were generated applying specific thresholds. The
data was segmented into $512\times 512$ pixel chips, with a select 446 chips
manually labelled for validation. The remainder of the data was randomly split
into a 60-20-20 distribution for training, validation, and testing,
incorporating non-hand-labeled Sentinel-1 and Sentinel-2 data for weakly
supervised training. Note that our method does not generate clouds, so the
test set was selected with cloud-free images of the same volume.

The SEN12-FLOOD Dataset, which we used for our semantic segmentation and
qualitative comparison, is compiled from 336 time series featuring Sentinel 1
and Sentinel 2 images of regions that experienced significant flooding during
the winter of 2019. The data collection period spans from December 2018 to May
2019, with the observed areas primarily located in East Africa, South West
Africa, the Middle-East, and Australia. A sequence corresponds to multiple
512x512 tiles, each representing a crop from a specific acquisition

#### Evaluation metric

In our flood segmentation task, we evaluated model performance through F1,
precision, recall, and IoU metrics, while the image-to-image translation task
was assessed via PSNR, SSIM, and LPIPS metrics. Note that due to the lack of
color information in SAR images, PSNR may not provide fully meaningful
insights, thus necessitating the addition of the SSIM and LPIPS metric. Even
though the SAR2EO task is a predictive task with definite GT paired with
input, low PSNR or SSIM does not indicate the model’s output is simply wrong.
As SAR data aggregated significantly different wavelength range compared to
that of EO, there can be multiple (diverse) probable EO images corresponding
to one SAR image and vice versa. It leads to the necessity of utilizing LPIPS
as a supplementary indicator.

Figure 4: Results of applying the DSE framework on the SEN12-FLOOD dataset.
From left to right: SAR, EO, and Synthetic EO. (Note that SAR image is (VV,
VH, (VV+VH)/2.) 3-channel image, but it is visualized as a gray scale.)

#### Implementation details

The DSE framework comprises two main components: a pretrained VQGAN [10] model
and our newly proposed Diffusion-Based SAR to EO model. VQGAN typically excels
at reducing both computational load and inference time. In the interest of
constructing an efficient diffusion model, we adopted the VQGAN model, as
featured in the Latent Diffusion Model [36]. We set the number of time steps
for the Brownian Bridge to be 1000 during the training stage and employed 200
sampling steps during the inference stage, thus striking a balance between
sample quality and computational efficiency. It’s important to note that our
baseline code builds upon BBDM 111github.com/xuekt98/BBDM, and all
hyperparameters not explicitly mentioned align with those used in BBDM [20].

In addition, the semantic segmentation model we employed in our experiment was
sourced from repository 222github.com/qubvel/segmentation_models.pytorch, and
we utilized a fundamental U-Net based on ResNet34. All hyperparameters
strictly adhere to the default settings.

###  4.1 Quantitative Results

Figure 5: Comparison of flood detection by SAR experts using pairs of EO and
SAR images versus pairs of SynEO and SAR images. Please note that we provided
SAR experts with both three-channel SAR images (VV, VH, (VV+VH)/2) and one-
channel images simultaneously. The SAR images included in the figure have been
identified by SAR experts as being more conducive to their analyses.

#### Image-to-Image Translation

Table 2 provides the experimental findings from the SEN12-FLOOD dataset’s test
subset, with clouds and data voids excluded, showcasing the proficiency of the
DSE framework in synthesizing SAR2EO imagery. It’s crucial to note that the
temporal alignment between the multi-temporal SAR and EO data within the
SEN12-FLOOD dataset isn’t precise. Accordingly, we’ve matched the EO data from
the nearest date to the reference SAR imagery.

As displayed in Tab 2, among the compared baselines, DSE demonstrated superior
performance in terms of PSNR, SSIM, and LPIPS. Furthermore, the most favorable
performance was recorded when multi-temporal SAR (spatially registered and
temporally random) was utilized as an input. These results confirm that the
DSE framework operates efficiently under these conditions. Regrettably, our
experimented dataset did not provide topographic information, thus we were
unable to test the model under this scenario. Future work aims to improve
model accuracy by introducing topographic information as an additional
condition.

#### Flood Segmentation

Tab 1 displays the quantitative comparison of flood segmentation results
utilizing SAR, EO, and SynEO. As indicated in the table, the flood
segmentation solely based on SAR exhibits the lowest performance in terms of
Recall, Rec., F1 score and IoU. This is primarily due to the irregular pattern
for the flood area in our single-temporal SAR experimental setting, as opposed
to a multi-temporal SAR setting as in these referenced works [24]. It’s worth
noting that even if the intensities are identical in two different SAR images,
they could correspond to distinct objects.

The best result was achieved through segmentation using solely EO images. This
is because, for the sake of a fair experiment, we employed only cloud-free
images in the test set; hence, the EO image appears the clearest and is the
closest representation of the correct answer.

Although the performance was lower than the EO image for the cloudless test
set, the IoU when using SAR and SynEO was 0.555, which was almost 30 higher
than when only SAR was used. Note that EO video is unavailable at night and in
cloudy conditions. These experimental results suggest that the SynEO can
provide a level of information to the segmentation model that is comparable to
clear, cloud-free EO images.

#### Human analysis

Modality | IoU
---|---
SAR + EO | 0.5532
SAR + SynEO | 0.5464 ($\downarrow 0.68$)

Table 3: Comparison of flood detection by SAR experts using pairs of EO and
SAR images versus pairs of SynEO and SAR images. Note that this value
represents the average result of flood mapping performed by five SAR experts.

The outcomes presented in Tab 3 represent flood mapping results derived by a
SAR expert utilizing (EO,SAR) and (SynEO,SAR) pairs. As evident from the
table, simple EO images yield relatively low performance as they predominantly
rely on SAR data due to obstructions like clouds, cloud shadows, and fog. In
contrast, SynEO images display superior performance compared to (EO, SAR)
pairs as they offer additional information to the examiner devoid of
obstructions such as clouds, cloud shadows, and fog.

Our experimental outcomes demonstrate that SynEO images can effectively be
used as auxiliary data when EO images are impaired by conditions such as fog,
night, and clouds. Furthermore, the potential utility of SynEO data in
circumstances where matched EO data cannot be secured is suggested. It should
be highlighted, however, that in our application, SynEO images may not prove
particularly beneficial in situations where both clean SAR and corresponding
EO images are available. Given that our SynEO is based on SAR, it may
incorporate errors. Hence, in our application setting, SynEO was not used in
isolation, but rather served as a supplemental resource for SAR.

###  4.2 Qualitative Results

#### Image-to-Image Translation

Figure 5 showcases the utilization of the DSE framework on the SEN12-FLOOD
dataset. In the SEN12-FLOOD dataset, SAR and EO images are not always aligned
by date, so the closest EO images were selected for both training and
inference. As demonstrated in the figure, the DSE framework is capable of
generating synthetic EO images with remarkable clarity. Specifically, Fig.
5-(a) reveals that the flood area is accurately represented, and notably,
without the formation of clouds or cloud shadows. Further, as indicated in
Fig. 5-(c), while the original EO image suffers from data loss, hence the
absence of the top portion of the image, the synthetic EO (SynEO) images
derived from SAR data are not afflicted by such data loss. Finally, as
depicted in Fig. 5-(d), in contrast to the original EO image, the generated
image demonstrates the effective removal of haze.

#### Human analysis

Figure 5 provides a qualitative comparison of flood mapping results obtained
from (SynEO and SAR) and (EO, SAR) images, as analyzed by SAR experts. As
depicted in the figure, the use of EO images becomes challenging in conditions
such as clouds and fog. Furthermore, deriving flood maps from simple SAR
images is complex, due to the inherent nature of these images. Specifically,
SAR images may display dark areas over water bodies or extremely flat surfaces
where radar signals are reflected away from the sensor, and in regions of
dense vegetation where signals are scattered in various directions. As
depicted in the figure, SynEO can support SAR experts under these conditions
by supplementing the SAR image with additional information.

#### Variance map

Differentiating from existing models, the DSE framework emerges as the
pioneering diffusion model-based approach to SAR2EO. This feature allows it to
generate multiple varied samples. In practical terms, the objective of the
SAR2EO task is to create real-time SynEO images that align closely with SAR
images. However, given that SynEO images derive from SAR data rather than from
actual EO, it presents challenges in determining which sections to trust
during expert analysis. As demonstrated in Fig. 6, the DSE’s capacity to
produce an array of samples and a corresponding variance map, as seen in Fig.
6-(d), provides a unique advantage. This benefit allows interpreters to place
confidence in areas of consistency across the samples, focusing more on the
SAR over the SynEO in areas of inconsistency. We firmly believe that such
features embody considerable promise for the continued evolution of SAR2EO.

##  5 Limitation and Future Work

#### Limitation

In this study, we proposed a DSE framework designed to enhance the human
interpretability of SAR. However, the DSE framework requires the use of both
spatial (strict) and temporal (near-time) pairs of SAR and EO for image-to-
image translation. Despite not necessitating flood labeling, obtaining and
registering paired images can often prove challenging. Consequently, this can
result in significant costs, which constitutes a notable limitation of our
current work.

#### Future work

In our future work, we plan to create a more cost-effective DSE framework by
effectively applying unpaired image-to-image translation from SAR to EO.
Additionally, we aim to broaden its applicability to a diverse range of
disaster scenarios, including earthquakes and forest fires, in addition to
floods.

Figure 6: Various output samples generated by the DSE framework. The
diffusion-based image-to-image translation allows for the creation of multiple
samples, enabling uncertainty measurement through ensemble methods.

##  6 Conclusion

In this paper, we introduced a Diffusion-Based SAR to EO Image Translation
(DSE) framework, aiming to improve human analysis of floods. The DSE was
designed to address two central issues. First, exploiting EO imagery for flood
mapping frequently suffers from impracticability in cloud cover or night-time
while it is simply interpretable so that handy for labeling and analysis.
Second, the alternative deep learning-based SAR flood detection methodology,
which was proposed to remedy this problem, demands a substantial volume of
labeled flood data. In order to exploit advantages of EO and SAR
simultaneously, we proposed the DSE framework, a SAR-to-EO translation scheme
which effectively contributes to human analysis of flood. We validated our
algorithm on the Sen1Floods11 and SEN12FLO datasets and obtained quantitative
and qualitatively significant results. We hope that our research will be
widely used in flood detection.

## References

  * [1] Tri Dev Acharya, Dong Ha Lee, Intae Yang, and Jae Kang Lee.  Identification of water bodies in a landsat 8 oli image using a j48 decision tree.  Sensors (Basel, Switzerland), 16, 2016.
  * [2] Richard P Allan, Ed Hawkins, Nicolas Bellouin, and Bill Collins.  Ipcc, 2021: summary for policymakers.  2021\.
  * [3] Yifang Ban, Puzhao Zhang, Andrea Nascetti, Alexandre R Bevington, and Michael A Wulder.  Near real-time wildfire progression monitoring with sentinel-1 sar time series and deep learning.  Scientific reports, 10(1):1322, 2020.
  * [4] Derrick Bonafilia, Beth Tellman, Tyler Anderson, and Erica Issenberg.  Sen1floods11: A georeferenced dataset to train and test deep learning flood algorithms for sentinel-1.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 210–211, 2020.
  * [5] Yang Chen, Stijn Hantson, Niels Andela, Shane R. Coffield, Casey A. Graff, Douglas C. Morton, Lesley E. Ott, Efi Foufoula‐Georgiou, Padhraic Smyth, Michael L. Goulden, and James T. Randerson.  California wildfire spread derived using viirs satellite observations and an object-based tracking system.  Scientific Data, 9, 2022.
  * [6] Francesca Cigna and Deodato Tapete.  Satellite insar survey of structurally-controlled land subsidence due to groundwater exploitation in the aguascalientes valley, mexico.  Remote Sensing of Environment, 254:112254, 2021.
  * [7] Gonéri Le Cozannet, Matthieu Kervyn, S. Russo, Chinwe Ifejika Speranza, Pierric Ferrier, Michael Foumelis, Teodolina Lopez, and Hormoz Modaressi.  Space-based earth observations for disaster risk management.  Surveys in Geophysics, 41:1209 – 1235, 2020.
  * [8] Charles-Alban Deledalle, Loïc Denis, Florence Tupin, Andreas Reigber, and Marc Jäger.  Nl-sar: A unified nonlocal framework for resolution-preserving (pol)(in)sar denoising.  IEEE Transactions on Geoscience and Remote Sensing, 53:2021–2038, 2015.
  * [9] Yukio Endo, Bruno Adriano, Erick Mas, and Shunichi Koshimura.  New insights into multiclass damage classification of tsunami-induced building damage from sar images.  Remote Sensing, 10(12):2059, 2018.
  * [10] Patrick Esser, Robin Rombach, and Bjorn Ommer.  Taming transformers for high-resolution image synthesis.  In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021.
  * [11] André Florentino, Saúl Charapaqui, Cesar De La Jara, and Marco A. Milla.  Implementation of a ground based synthetic aperture radar (gb-sar) for landslide monitoring: system description and preliminary results.  2016 IEEE XXIII International Congress on Electronics, Electrical Engineering and Computing (INTERCON), pages 1–6, 2016.
  * [12] Omid Ghorbanzadeh, Hejar Shahabi, Alessandro Crivellari, Saeid Homayouni, Thomas Blaschke, and Pedram Ghamisi.  Landslide detection using deep learning and object-based image analysis.  Landslides, 19:929 – 939, 2022.
  * [13] Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  Advances in Neural Information Processing Systems, 33:6840–6851, 2020.
  * [14] Yaqi Ji, Josaphat Tetuko Sri Sumantyo, Ming Yam Chua, and Mirza Muhammad Waqar.  Earthquake/tsunami damage level mapping of urban areas using full polarimetric sar data.  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11:2296–2309, 2018.
  * [15] ByoungChul Ko, Hyeong Hun Kim, and Jae Yeal Nam.  Classification of potential water bodies using landsat 8 oli and a combination of two boosted random forest classifiers.  Sensors (Basel, Switzerland), 15:13763 – 13777, 2015.
  * [16] Georgios Kordelas, Ioannis Manakos, David G. Aragones, Ricardo Díaz-Delgado, and Javier Bustamante.  Fast and automatic data-driven thresholding for inundation mapping with sentinel-2 data.  Remote. Sens., 10:910, 2018.
  * [17] Georgios A. Kordelas, Ioannis Manakos, Gaëtan Lefebvre, and Brigitte Poulin.  Automatic Inundation Mapping Using Sentinel-2 Data Applicable to Both Camargue and Doñana Biosphere Reserves.  Remote Sensing, 11(19):2251, Jan. 2019.  Number: 19 Publisher: Multidisciplinary Digital Publishing Institute.
  * [18] Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila.  Noise2noise: Learning image restoration without clean data.  arXiv preprint arXiv:1803.04189, 2018.
  * [19] Tianjie Lei, Jiabao Wang, Xiangyu Li, Weiwei Wang, Changliang Shao, and Baoyin Liu.  Flood disaster monitoring and emergency assessment based on multi-source remote sensing observations.  Water, 2022.
  * [20] Bo Li, Kaitao Xue, Bin Liu, and Yunyu Lai.  Bbdm: Image-to-image translation with brownian bridge diffusion models.  2022\.
  * [21] Ying Li, Hongli Gong, David Dagan Feng, and Yanning Zhang.  An adaptive method of speckle reduction and feature enhancement for sar images based on curvelet transform and particle swarm optimization.  IEEE Transactions on Geoscience and Remote Sensing, 49:3105–3116, 2011.
  * [22] Ying Li, Jie Hu, and Yu Jia.  Automatic sar image enhancement based on nonsubsampled contourlet transform and memetic algorithm.  Neurocomputing, 134:70–78, 2014.
  * [23] Spencer Low, Oliver Nina, Angel D Sappa, Erik Blasch, and Nathan Inkawhich.  Multi-modal aerial view image challenge: Translation from synthetic aperture radar to electro-optical domain results-pbvs 2023.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 515–523, 2023.
  * [24] Gonzalo Mateo-Garcia, Joshua Veitch-Michaelis, Lewis Smith, Silviu Vlad Oprea, Guy Schumann, Yarin Gal, Atılım Güneş Baydin, and Dietmar Backes.  Towards global flood mapping onboard low cost satellites with machine learning.  Scientific reports, 11(1):1–12, 2021.
  * [25] Stuart K McFeeters.  The use of the normalized difference water index (ndwi) in the delineation of open water features.  International journal of remote sensing, 17(7):1425–1432, 1996\.
  * [26] Kshitij Mishra and P. Rama Chandra Prasad.  Automatic extraction of water bodies from landsat imagery using perceptron model.  2015\.
  * [27] Edoardo Nemni, Joseph Aylett-Bullock, Samir Belabbes, and Lars Bromley.  Fully convolutional neural network for rapid flood segmentation in synthetic aperture radar imagery.  Remote. Sens., 12:2532, 2020.
  * [28] Edoardo Nemni, Joseph Bullock, Samir Belabbes, and Lars Bromley.  Fully convolutional neural network for rapid flood segmentation in synthetic aperture radar imagery.  Remote Sensing, 12(16):2532, 2020.
  * [29] Hyeoncheol Noh, Jingi Ju, Minseok Seo, Jongchan Park, and Dong-Geol Choi.  Unsupervised change detection based on image reconstruction loss.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1352–1361, 2022.
  * [30] Malsha V Perera, Nithin Gopalakrishnan Nair, Wele Gedara Chaminda Bandara, and Vishal M Patel.  Sar despeckling using a denoising diffusion probabilistic model.  IEEE Geoscience and Remote Sensing Letters, 2023.
  * [31] Clément Rambour, Nicolas Audebert, Elise Koeniguer, Bertrand Le Saux, Michel Crucianu, and Mihai Datcu.  Sen12-flood : a sar and multispectral dataset for flood detection, 2020\.
  * [32] C Rambour, N Audebert, E Koeniguer, B Le Saux, M Crucianu, and M Datcu.  Sen12-flood: a sar and multispectral dataset for flood detection, 2020\.
  * [33] Jun Rentschler, Melda Salhab, and Bramka Arga Jafino.  Flood exposure and poverty in 188 countries.  Nature communications, 13(1):3527, 2022.
  * [34] Mario Fuentes Reyes, Stefan Auer, Nina Merkle, Corentin Henry, and Michael Schmitt.  Sar-to-optical image translation based on conditional generative adversarial networks - optimization, opportunities and limits.  Remote. Sens., 11:2067, 2019.
  * [35] Melissa M Rohde.  Floods and droughts are intensifying globally.  Nature Water, 1(3):226–227, 2023.
  * [36] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022.
  * [37] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pages 234–241. Springer, 2015.
  * [38] Gulcan Sarp and Mehmet Ozcelik.  Water body extraction and change detection using time series: A case study of lake burdur, turkey.  Journal of Taibah University for Science, 11:381 – 391, 2017.
  * [39] David J. Schneider, Alexa R. Van Eaton, and Kristi L. Wallace.  Satellite observations of the 2016–2017 eruption of bogoslof volcano: aviation and ash fallout hazard implications from a water-rich eruption.  Bulletin of Volcanology, 82:1–20, 2020.
  * [40] Minseok Seo, Hakjin Lee, Yongjin Jeon, and Junghoon Seo.  Self-pair: Synthesizing changes from single source for object change detection in remote sensing imagery.  In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6374–6383, 2023.
  * [41] Serhiy Skakun.  A neural network approach to flood mapping using satellite imagery.  Comput. Informatics, 29:1013–1024, 2012.
  * [42] Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations.  Jan 2021.
  * [43] Cheryl WJ Tay, Sang-Ho Yun, Shi Tong Chin, Alok Bhardwaj, Jungkyo Jung, and Emma M Hill.  Rapid flood and damage mapping using synthetic aperture radar in response to typhoon hagibis, japan.  Scientific data, 7(1):100, 2020.
  * [44] Lei Wang, Xin Xu, Yue Yu, Rui Yang, Rong Gui, Zhaozhuo Xu, and Fangling Pu.  Sar-to-optical image translation using supervised cycle-consistent adversarial networks.  IEEE Access, 7:129136–129149, 2019.
  * [45] Ting-Chun Wang, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro.  High-resolution image synthesis and semantic manipulation with conditional gans.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8798–8807, 2018.
  * [46] Lori White, Brian Brisco, Mohammed Dabboor, Andreas Schmitt, and Andrew Pratt.  A collection of sar methodologies for monitoring wetlands.  Remote. Sens., 7:7615–7645, 2015.
  * [47] Hanqiu Xu.  Modification of normalised difference water index (ndwi) to enhance open water features in remotely sensed imagery.  International journal of remote sensing, 27(14):3025–3033, 2006\.
  * [48] Jun Yu, Shenshen Du, Renjie Lu, Pengwei Li, Guochen Xie, Zhongpeng Cai, Keda Lu, Qing Ling, Cong Wang, Luyu Qiu, et al.  Sar2eo: A high-resolution image translation framework with denoising enhancement.  arXiv preprint arXiv:2304.04760, 2023.
  * [49] Dan Zhang, Fangfang Zhou, Yuwen Jiang, and Zhengming Fu.  Mm-bsn: Self-supervised image denoising for real-world with multi-mask based on blind-spot network.  In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4188–4197, 2023.