capbtabboxtable[][]

# Deep Metric Learning for Unsupervised Remote Sensing Change Detection

Wele Gedara Chaminda Bandara, and Vishal M. Patel The authors are with the
Department of Electrical and Computer Engineering, Johns Hopkins University,
Baltimore, MD, 21218 USA e-mail: {wbandar1<EMAIL_ADDRESS>
###### Abstract

Remote Sensing Change Detection (RS-CD) aims to detect relevant changes from
Multi-Temporal Remote Sensing Images (MT-RSIs), which aids in various RS
applications such as land cover, land use, human development analysis, and
disaster response. The performance of existing RS-CD methods is attributed to
training on large annotated datasets. Furthermore, most of these models are
less transferable in the sense that the trained model often performs very
poorly when there is a domain gap between training and test datasets. This
paper proposes an unsupervised CD method based on deep metric learning that
can deal with both of these issues. Given an MT-RSI, the proposed method
generates corresponding change probability map by iteratively optimizing an
unsupervised CD loss without training it on a large dataset. Our unsupervised
CD method consists of two interconnected deep networks, namely Deep-Change
Probability Generator (D-CPG) and Deep-Feature Extractor (D-FE). The D-CPG is
designed to predict change and no change probability maps for a given MT-RSI,
while D-FE is used to extract deep features of MT-RSI that will be further
used in the proposed unsupervised CD loss. We use transfer learning capability
to initialize the parameters of D-FE. We iteratively optimize the parameters
of D-CPG and D-FE for a given MT-RSI by minimizing the proposed unsupervised
“similarity-dissimilarity loss”. This loss is motivated by the principle of
metric learning where we simultaneously maximize the distance between change
pair-wise pixels while minimizing the distance between no-change pair-wise
pixels in bi-temporal image domain and their deep feature domain. The
experiments conducted on three CD datasets show that our unsupervised CD
method achieves significant improvements over the state-of-the-art supervised
and unsupervised CD methods. Code available at
https://github.com/wgcban/Metric-CD

##  I Introduction

Figure 1: How our deep metric learning-based CD method differs from the state-
of-the-art CD methods. (a) Supervised CD: In this case, MT-RSIs and the
corresponding change masks are available for training the CD model [53, 11,
32]. (b) Self-supervised CD: In this case, change masks are not available
during training, but a large unlabeled MT-RSI dataset is utilized to
synthesize changes [9, 39]. (c) Unsupervised CD: These methods start with a
sub-optimal pre-trained Feature Extractor (FE) that is trained for some other
task to obtain deep features, and then perform Change Vector Analysis (CVA)
followed by thresholding to obtain change maps [41, 21]. (d) Deep Metric
Learning for Unsupervised CD (ours): We iteratively optimize the parameters of
deep change probability generator (D-CPG) to obtain change and no-change
probability maps for a given MT-RSI according to the proposed unsupervised
“similarity-dissimilarity” loss for CD. The “similarity-dissimilarity” loss is
motivated by the popular principle of metric learning where we maximize the
distance between change pair-wise pixels and minimize the distance between no-
change pairwise pixels in bi-temporal image domain and deep feature domain.

Change detection (CD) aims to detect relevant changes from multi-temporal
remote sensing images (MT-RSIs) captured over the same geographic area at
distinct times. In recent years, CD has gained great attention in the research
community due to its numerous applications in urban planning, environmental
monitoring, disaster assessment, agriculture, forestry, etc. [47, 10]. Most of
these practical applications typically need to quickly obtain the relevant
change information from large MT-RSIs. In the context of CD, the irrelevant
changes such as vegetation color variations, building shadows, atmospheric
variations, lighting changes, and cloud covering limit the accuracy of change
maps.

Most CD techniques proposed in the literature are based on supervised training
where a model (i.e., classical or deep learning based model) is trained with a
large number of MT-RSIs and corresponding change labels (see Fig. 1-(a)).
However, in MT problems, it is expensive and often impossible to obtain a
large number of annotated training samples for modeling change and no change
classes [53, 11, 32, 4, 2]. This limited access to labeled training data has
motivated the researchers to explore self-supervised and unsupervised CD
methods.

The recently proposed self-supervised methods are often based on either self-
supervised pre-training [9, 2] or contrastive learning [39] (see Fig. 1-(b)).
In self-supervised pre-training a model is trained by utilizing the change
labels extracted directly from the MT-RSIs themselves. In other words, instead
of training a model for the CD task, self-supervised pre-training optimizes a
model for a pretext task in which the labels are extracted directly from the
MT-RSIs by alternating relative positions of patches. By doing that, the
network is pre-trained to recognize the changes in MT-RSIs with no annotated
change masks. In contrastive self-supervised CD, a pseudo-Siamese network is
trained to regress the output between its two branches, which is pre-trained
in a contrastive way on a large unlabeled MT-RSI dataset. However, both of
these self-supervised CD methods require a large unlabeled MT-RSI dataset for
optimizing the model parameters. In addition, the performance of the trained
model heavily depends on how realistic the changes they make while training
the model.

In contrast to self-supervised CD methods, unsupervised CD methods [41, 21] do
not often depend on a large MT-RSI dataset for training. They extract feature
representations of a given MT-RSI from a Feature Extractor (FE) which is
initialized from a network pre-trained for some other task (like
classification, segmentation, etc.) via transfer learning (see Fig. 1-(c)).
Next, Change Vector Analysis (CVA) [42] or Deep Feature Difference (D-FD) [55]
is performed on the extracted deep features to identify the change pixels.
However, the existing deep unsupervised CD methods are sensitive to pseudo
unwanted changes, because the feature extractor is not trained to produce deep
features which are invariant to color and seasonal transformations that
usually appear in MT-RSIs.

In order to address the aforementioned issues of state-of-the-art (SOTA) CD
methods, we propose a novel unsupervised deep metric learning-based strategy
for CD (see Fig. 1 \- (d)). Our CD method consists of two networks, namely
Deep-Change Probability Generator (D-CPG) and Deep-Feature Extractor (D-FE).
The parameters of D-CPG are randomly initialized while the parameters of D-FE
are initialized via transfer learning. The weights of both D-CPG and D-FE are
iteratively optimized for a given MT-RSI according to the proposed similarity-
dissimilarity loss which is motivated by the principal of metric learning
where we maximize the similarity between no-change pair-wise pixels while
minimizing the similarity between change pair-wise pixels in bi-temporal image
domain and their respective deep feature domain as shown in Fig. 2.
Furthermore, to force the deep features of the given MT-RSI from the D-FE to
be invariant to local colorimetric variations, we also define a self-
supervised auxiliary task in which we force the feature representations of a
given MT-RSI to be consistent under different random color transformations
through the auxiliary context consistency loss. In our experiments, we show
the significance of the proposed CD method by comparing it with SOTA
supervised, self-supervised, and unsupervised CD methods.

Figure 2: We start from random weights for D-CPG $\theta^{0}_{\text{D-CPG}}$,
and iteratively update the weights in order to minimize the proposed
$\mathcal{L}_{\text{CD}}$. At the $t$-th iteration, weights
$\theta^{t}_{\text{D-CPG}}$ are used to generate the change probability maps
$\mathbf{P}_{\text{c}}^{t}=f_{\theta_{\text{D-CPG}}}(\mathbf{I}_{d})$.
$\mathbf{P}_{\text{c}}^{t}$ is used to calculate $\mathcal{L}_{\text{CD}}$.
The gradient of the loss $\mathcal{L}_{\text{CD}}$ with respect to the weights
$\theta$ is then computed and used to update the parameters.

##  II Related Works

#### Classical CD methods.

Although there are many classical methods available for CD from RSIs, most of
them are based on the Difference Image (DI). A DI can be obtained by image
differentiation [49], image ratio [48], Principal Component Analysis (PCA)
[25], or Change Vector Analysis (CVA) [27]. Once the DI is calculated, the
binary change map is obtained by classifying the DI into two classes, namely
change class and no change class. This can be achieved by thresholding the DI
with an appropriate threshold [49], using a clustering algorithm (i.e.,
K-Means [8, 18], metric learning [18], etc.), or a supervised classification
[52]. However, the main drawback of these classical approaches is that the
resulting change map contains many pseudo-random changes due to isolated
radiometric changes, vegetation color variations, and color changes due to
seasonal effects [56]. Although supervised classification is the best way to
avoid such random changes, it requires expertly annotated pixel-level labels,
limiting the applicability of many conventional approaches.

#### Deep Learning-based CD methods.

Recently, a variety of methods based on Convolutional Neural Networks
(ConvNets) have also been proposed for CD from MT-RSIs due to their excellent
ability to learn low-level and high-level features automatically during
training [44, 22, 3]. Among these, most of them are fully supervised methods.
They often require more training samples and more training time to learn the
deep features of RSIs [15]. Early Fusion (EF) [53], and Siamese networks [11]
are the widely adopted architectures for supervised CD. However, collecting a
large number of training samples that require learning deep features is
difficult to achieve in many CD applications, especially with high-precision
and pixel-level annotations [22]. In recent years, the limited access to
labeled change information has increased the researcher’s attention to self-
supervised and unsupervised CD. Self-supervised pre-training (SSP-CD) [26],
Generative Adversarial Networks [39] (GAN-CD), and contrastive loss-based
pseudo-Siamese networks (CPS-CD) [9] are the widely used self-supervised CD
methods. Even though self-supervised learning encourages the network to learn
meaningful features, it requires a large unlabeled dataset that may not be
available for some CD applications. Most unsupervised CD methods are based on
transfer learning [36], where they extract deep features of MT-RSIs by
initializing network weights that are pre-trained for some other task (i.e.,
segmentation or classification) on real or RS images. Examples of
unsupervised-CD include Deep - Feature Difference (FeatureDiff) [41] and Deep
- Change Vector Analysis (D-CVA) [21]. Once the deep features are obtained,
FeatureDiff and D-CVA obtain the change map by computing the feature
difference or performing CVA followed by thresholding, respectively.

##  III Proposed Method

###  III-A Problem formulation

Consider two co-registered multi-spectral remote sensing images
$\mathbf{I}_{1}$ and $\mathbf{I}_{2}$ with $b$ bands, captured over the same
geographical area at distinct time instances $t_{1}$ and $t_{2}$,
respectively. Usually $\mathbf{I}_{1}$ and $\mathbf{I}_{2}$ are referred to as
the pre-change image and post-change image, respectively and
$\\{\mathbf{I}_{1},\mathbf{I}_{2}\\}$ is referred to as the multi-temporal
image. Our objective is to detect the changes from $\mathbf{I}_{1}$ and
$\mathbf{I}_{2}$ in an unsupervised way; i.e., we classify the set of all
pixels ($\bf\Omega$) into change ($\mathbf{\Omega}_{\textit{c}}$) and no-
change ($\mathbf{\Omega}_{\textit{nc}}$) pixels. If we compute the
dissimilarity between each pixel of $\mathbf{I}_{1}$ and $\mathbf{I}_{2}$
individually, we often end up obtaining either not-meaningful changes from a
context-based perspective or changes which are minute to be considered as a
change of interest because of inherent radiometric dissimilarities between
pre-change and post-change images. Such small isolated radiometric changes are
therefore not considered as a change.

###  III-B Overview of the proposed solution

The proposed unsupervised framework addresses the aforementioned issues of
change detection. As depicted in Fig. 3, we first perform polynomial color
correction on multi-temporal image $\\{\mathbf{I}_{1},\mathbf{I}_{2}\\}$ to
obtain pre-processed multi-temporal image
$\\{\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2}\\}$. Next, we
compute the difference image $\mathbf{I}_{d}$ and process it through a Deep -
Change Probability Generator (D-CPG) to generate the change probability map
$\mathbf{P}_{c}$. The weights of D-CPG are optimized by minimizing the
proposed similarity-dissimilarity loss in an unsupervised manner. However,
minimizing the similarity-dissimilarity loss in image-domain results in
detecting unwanted changes due to local radiometric shifts of pre-change and
post-change images. To make our change detection framework robust to such
isolated radiometric changes, we also utilize similarity-dissimilarity loss on
multi-level feature representations of
$\\{\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2}\\}$, obtained from a
Deep Feature Extractor (D-FE) that is pre-trained for some other task. In
order to constrain features from D-FE to be robust to isolated color shifts,
we apply data augmentations on
$\\{\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2}\\}$ and constrain
their deep-features to be consistent with deep-features of
$\\{\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2}\\}$ via the context-
consistency loss. We iteratively minimize the overall loss function
$\mathcal{L}_{\text{CD}}$ for unsupervised CD to generate optimal change
probability map $\mathbf{P}^{*}_{c}$ for a given multi-temporal image
$\\{\mathbf{I}_{1},\mathbf{I}_{2}\\}$ as shown in fig. 2.

###  III-C Proposed unsupervised CD framework

Figure 3: The proposed unsupervised deep metric learning framework for CD in
multi-temporal remote sensing images.

#### Image preprocessing and colorimetric mapping.

We adopt Polynomial Color Correction (PCC)[20] to reduce colorimetric changes
due to different sensor properties, seasonal variations, and lighting
conditions. It has been experimentally shown that PCC can significantly reduce
colorimetric errors [1, 17]. Using PCC, we map the pre-change image color
space to the post-change image color space using a polynomial mapping function
$\bf M$, usually found by Least Squares Regression (LSR)[17]. To do that, we
first take $N_{c}$ color samples (pixels) from each pre-change and post-change
image by downsampling the original images by a factor of $\beta$. We denote
the downsampled versions of pre-change and post-change images as
$\mathbf{i}_{1}$ and $\mathbf{i}_{2}$, respectively. The reason for applying
LSR on downsampled image pixels is to more accurately bring out the color
properties of the images than the actual changes present in multi-temporal
image. Also, this will significantly reduce the pre-processing time especially
when the satellite images have a very high spatial resolution. Consequently,
we compute the polynomial mapping matrix $\bf M$ that maps values of
$\rho(\mathbf{i}_{1})$ to the colors of the post-change image
$\mathbf{i}_{2}$, where $\rho(\cdot)$ is a polynomial kernel function that
maps the spectral vectors of the satellite image to a higher $d$-dimensional
space. See the supplementary document for an assessment of the different
kernel functions. The mapping matrix $\bf M$ is found by least squares
regression and has a closed form solution of the form:

$\mathbf{M}=(\rho(\mathbf{i}_{1})^{T}\rho(\mathbf{i}_{1}))^{-1}\rho(\mathbf{i}_{1})^{T}\mathbf{i}_{2}.$
(1)

Once $\bf M$ is computed, we obtain our final color transformed version of
pre-change and post-change images as follows:

$\widetilde{\mathbf{I}}_{1}=\mathbf{M}\rho(\mathbf{I}_{1})\text{ , and
}\widetilde{\mathbf{I}}_{2}=\mathbf{I}_{2}.$ (2)

#### Difference Image $\mathbf{I}_{d}$.

We compute the dissimilarity measure $\mathbf{I}_{d}$ between pre-change
$(\mathbf{\widetilde{I}}_{1})$ and post-change $(\mathbf{\widetilde{I}}_{2})$
images by utilizing the Mahalanobis distance [12] as follows:

$\mathbf{I}_{d}=\left\|\widetilde{\mathbf{I}}_{1}-\widetilde{\mathbf{I}}_{2}\right\|_{\mathbf{M}}=\sqrt{\left(\widetilde{\mathbf{I}}_{1}-\widetilde{\mathbf{I}}_{2}\right)^{T}\mathbf{S}^{-1}\left(\widetilde{\mathbf{I}}_{2}-\widetilde{\mathbf{I}}_{2}\right)},$
(3)

where $\bf S$ is the covariance matrix of size $b\times b$, and $T$ denotes
the matrix transpose. We use the Mahalanobis distance since it computes the
dissimilarity measure based on the distribution pattern of data points through
the covariance matrix $\mathbf{S}$, unlike $L_{1}$ or $L_{2}$ distances. The
conventional CD methods use the difference image $\mathbf{I}_{d}$ and
threshold it with an appropriate generic rule to obtain the binary change map
[38]. However, the latter case is sensitive to noise and local illumination
variations, and does not consider local consistency properties and object-
based perspective of the change mask.

#### Deep-Change Probability Generator (D-CPG).

To enforce the local consistency property of the change mask, we go one step
beyond the conventional CD methods where we take the difference image
$\mathbf{I}_{d}$ as input and process it through a Deep - Change Probability
Generator (D-CPG) network. The D-CPG estimates the probability of change for a
given pixel location $(i,j)-\mathbf{P}_{c}(i,j)$ by considering a local
neighborhood $\mathbf{N}(i,j)$ around $(i,j)$ in the difference image
$\mathbf{I}_{d}$. Mathematically, we can defined this process as:

$\mathbf{P}_{c}=f_{\theta_{\text{D-CPG}}}(\mathbf{I}_{d}),$ (4)

where $f_{\theta_{\text{D-CPG}}}(\cdot)$ is the parametric representation of
D-CPG. The size of the local-neighborhood $\mathbf{N}(i,j)$ is usually defined
by the size of the receptive field [29] of D-CPG, and can be appropriately
controlled by changing the network depth. We conduct experiments with
different ConvNet architectures for our D-CPG like hourglass network [33],
U-Net[40] with skip connections, and Residual Networks (ResNet) [19] with
different depth (i.e., 4, 8, 16, 32, and 64 residual blocks). The parameters
of the D-CPG are initialized randomly and optimized over each iteration for a
given MT-RSI during testing (see Fig. 2) by minimizing the proposed
unsupervised loss $\mathcal{L}_{\text{CD}}$ function which will be introduced
in the following section.

###  III-D Proposed unsupervised loss function for CD

The proposed unsupervised loss function for CD $\mathcal{L}_{\text{CD}}$
consists of three terms, namely image-domain and feature-domain similarity-
dissimilarity loss ($\mathcal{L}_{\text{img}}$ and
$\mathcal{L}_{\text{feat}}$) and context consistency loss
($\mathcal{L}_{\text{ctx}}$) as detailed below.

#### Image-domain loss for CD: $\mathcal{L}_{\text{img}}$.

We propose an unsupervised loss function for CD, motivated by the most popular
principle in metric learning: minimize the distance between similar datasets
and maximize the distance between dissimilar datasets [37, 51, 54]. We adopt
this notion to build our unsupervised loss for CD where we try to maximize the
distance between change pairwise pixels while minimizing the distance between
no-change pairwise pixels. We weight $\widetilde{\mathbf{I}}_{1}$ and
$\widetilde{\mathbf{I}}_{2}$ according to the change probability
$\mathbf{P}_{\text{c}}$ and no change probability $\mathbf{P}_{\text{nc}}$
$(=1-\mathbf{P}_{\text{c}})$ to identify change and no change pixels, and then
use the Mahalanobis distance to calculate the change loss
$\mathcal{L}_{\text{img-c}}(\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2},\mathbf{P}_{\text{c}})$
and no-change loss $\mathcal{L}_{\text{img-
nc}}(\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2},\mathbf{P}_{\text{nc}})$
as follows:

$\displaystyle\mathcal{L}_{\text{img}}$
$\displaystyle=\alpha_{\text{img}}\mathcal{L}_{\text{img-c}}(\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2},\mathbf{P}_{\text{c}})+\mathcal{L}_{\text{img-
nc}}(\widetilde{\mathbf{I}}_{1},\widetilde{\mathbf{I}}_{2},\mathbf{P}_{\text{nc}})$
(5)
$\displaystyle=-\alpha_{\text{img}}\left\|\mathbf{P}_{\text{c}}(\widetilde{\mathbf{I}}_{1}-\widetilde{\mathbf{I}}_{2})\right\|_{\mathbf{M}}+\left\|\mathbf{P}_{\text{nc}}(\widetilde{\mathbf{I}}_{1}-\widetilde{\mathbf{I}}_{2})\right\|_{\mathbf{M}},$

where we refer $\mathcal{L}_{\text{img}}$ to as image-domain change-nochange
loss for CD, and $\alpha_{\text{img}}$ is the regularization parameter for
balancing the trade-off between image-domain change and no-change loss terms.
By default we set $\alpha_{\text{img}}$ to 1.0. When
$\mathcal{L}_{\text{img}}$ is minimized, the no change loss
$\mathcal{L}_{\text{img-nc}}$ will lead to minimize the distance between no-
change pairwise pixels, and change loss $\mathcal{L}_{\text{img-c}}$ will lead
to maximize the distance between change pairwise pixels. Since
$\mathcal{L}_{\text{img}}$ is differentiable, it back-propagates through the
D-CPG and adjusts its parameters to generate better change map until it
converges.

#### Feature-domain loss for CD: $\mathcal{L}_{\text{feat}}$.

To impose context-based perspective of change maps in optimization, we adopt
transfer learning [36] capability for CD. Transfer learning enables us to
extract deep-features from a multi-layer deep feature extractor (D-FE) that is
pre-trained for some other task. Such deep feature representations can capture
spatial context of an image effectively. Therefore, to improve context-based
perspective of predicted change map, we extend our change-nochange loss
function from image-domain to feature-domain by utilizing the multi-layer
feature representations of multi-temporal image
$\\{\mathbf{I}_{1},\mathbf{I}_{2}\\}$ as follows:

$\displaystyle\mathcal{L}_{\text{feat}}$
$\displaystyle=\sum_{l=1}^{L}\left(\alpha_{\text{f}}^{l}\mathcal{L}_{\text{feat-c}}(\mathbf{F}_{1}^{l},\mathbf{F}_{2}^{l},d^{l}(\mathbf{P}_{\text{c}}))+\mathcal{L}_{\text{feat-
nc}}(\mathbf{F}_{1}^{l},\mathbf{F}_{2}^{l},d^{l}(\mathbf{P}_{\text{nc}}))\right)$
(6)
$\displaystyle=\sum_{l=1}^{L}\left(-\alpha_{\text{feat}}^{l}\left\|d^{l}(\mathbf{P}_{\text{c}})(\mathbf{F}_{1}^{l}-\mathbf{F}_{2}^{l})\right\|_{\mathbf{M}}+\left\|d^{l}(\mathbf{P}_{\text{nc}})(\mathbf{F}_{1}^{l}-\mathbf{F}_{2}^{l})\right\|_{\mathbf{M}}\right),$

where $l$ denotes the $l$-th layer of $L$-layered D-FE,
$\alpha_{\text{feat}}^{l}$ is the regularization parameter for the $l$-th
layer that is set to 1.0, $\mathcal{L}_{\text{feat-c}}(\cdot)$ is the feature-
domain change loss, $\mathcal{L}_{\text{feat-nc}}(\cdot)$ is the feature-
domain no-change loss, $d^{l}(\cdot)$ is the nearest-neighbor down-sampling of
$\\{\mathbf{P}_{c},\mathbf{P}_{nc}\\}$ to the spatial size of
$\mathbf{F}_{1}^{l}$ or $\mathbf{F}_{2}^{l}$, respectively.

#### Context consistency loss for CD: $\mathcal{L}_{\text{ctx}}$.

Although deep features can provide contextual information for an image to a
great extent, they are still sensitive to different illumination conditions,
different image sensors, and noise. To make our estimated change map
$\mathbf{P}_{c}$ to be further robust to such unwanted changes, we apply
random augmentations on the multi-temporal image
$\\{\mathbf{I}_{1},\mathbf{I}_{2}\\}$ at each iteration of testing, process
them though the deep feature extractor, and then constrain these features to
be consistent with their original feature representations. In particular, we
apply random color augmentations such as random brightness, contrast,
saturation and hue jittering, as well as random noise augmentation [45]. We
denote the augmented versions of pre-change and post-change images at $t$-th
iteration of optimization as $[\hat{\mathbf{I}}_{1}]_{t}$ and
$[\hat{\mathbf{I}}_{2}]_{t}$, and their respective features from the $l$-th
layer of D-FE as $[\hat{\mathbf{F}}^{l}_{1}]_{t}$ and
$[\hat{\mathbf{F}}^{l}_{2}]_{t}$, respectively. We then impose context
consistency between the original feature representations and the augmented
feature representations by context consistency loss $\mathcal{L}_{\text{ctx}}$
as follows:

$\mathcal{L}_{\text{ctx}}=\sum_{l=1}^{L}\left(\left\|\mathbf{F}_{1}^{l}-[\hat{\mathbf{F}}^{l}_{1}]_{t}\right\|_{1}+\left\|\mathbf{F}_{2}^{l}-[\hat{\mathbf{F}}^{l}_{2}]_{t}\right\|_{1}\right),$
(7)

where $\left\|\cdot\right\|_{1}$ denotes the $L_{1}$ norm.

#### Sparsity penalty: $\mathcal{L}_{sparse}$.

A trivial solution exists for the objective (6) where,
$\mathbf{P}_{c}=\mathbf{1}$. To avoid such degenerate solutions, we introduce
a sparsity penalty $\mathcal{L}_{sparse}$ in the form of $1/\sin(\cdot)$ that
discourages D-CPG predicting all-one (or all zero); specifically,

$\mathcal{L}_{sparse}=\sin\left(\frac{\pi}{hw}\sum_{i=1}^{h}\sum_{j=1}^{w}P_{c}(i,j)\right)^{-1},$
(8)

where $(h,w)$ is the (height, width) of the change probability map
$\mathbf{P}_{c}$.

The overall unsupervised loss function for CD $\mathcal{L}_{\text{CD}}$ that
we use to optimize D-CPG and D-FE is defined as follows:

$\mathcal{L}_{\text{CD}}=\mathcal{L}_{\text{img}}+\mathcal{L}_{\text{feat}}+\mathcal{L}_{\text{ctx}}+\mathcal{L}_{sparse}.$
(9)

By minimizing $\mathcal{L}_{\text{CD}}$ using an optimizer such as Adam [23]
over the parameters of D-CPG $\left(\theta_{\text{D-CPG}}\right)$ and D-FE
$\left(\theta_{\text{D-FE}}\right)$, we obtain optimum change probability map
$\mathbf{P}^{*}_{c}$ for a given bi-temporal image
$\\{\mathbf{I}_{1},\mathbf{I}_{2}\\}$, starting from the difference image
$\mathbf{I}_{d}$. Hence, the empirical information available to the change
detection process is the bi-temporal image
$\\{\mathbf{I}_{1},\mathbf{I_{2}}\\}$ and pre-trained weights of D-FE
$\theta_{\text{DFE}}^{0}$ from transfer learning, and hence the CD process is
completely unsupervised. This approach is schematically depicted in Fig. 2 and
Fig. 3.

Table I: The average quantitative CD results on the OSCD dataset [7], the
SZTAKI AirChange dataset [5, 6], and the QuickBird dataset [55]. Higher values
of OA, UA, Recall, F1, and AUC indicate good change detection performance.
Color convention: Best, 2-nd Best, 3-rd Best.

Method | OSCD Dataset [7] |  | SZTAKI AirChange Dataset [5, 6] |  | QuickBird Dataset [55]
---|---|---|---|---|---
OA |  UA |  Recall |  F1 |  AUC |  |  OA |  UA |  Recall |  F1 |  AUC |  |  OA |  UA |  Recall |  F1 |  AUC
Unsupervised CD methods: |  |  |  |  |  |  |  |  |  |
CVA [43] |  0.830 |  0.164 |  0.457 |  0.192 |  0.751 |  |  0.768 |  0.118 |  0.545 |  0.179 |  0.708 |  |  0.858 |  0.415 |  0.699 |  0.521 |  0.865
DPCA [13] |  0.929 |  0.238 |  0.238 |  0.189 |  0.823 |  |  0.861 |  0.167 |  0.377 |  0.191 |  0.704 |  |  0.893 |  0.512 |  0.508 |  0.510 |  0.807
ImageDiff [31] |  0.799 |  0.148 |  0.483 |  0.181 |  0.752 |  |  0.769 |  0.121 |  0.546 |  0.182 |  0.696 |  |  0.857 |  0.411 |  0.695 |  0.516 |  0.861
ImageRatio [28] |  0.810 |  0.113 |  0.402 |  0.149 |  0.697 |  |  0.863 |  0.067 |  0.136 |  0.042 |  0.604 |  |  0.829 |  0.352 |  0.667 |  0.461 |  0.807
ImageRegr [30] |  0.897 |  0.196 |  0.413 |  0.222 |  0.821 |  |  0.747 |  0.134 |  0.615 |  0.198 |  0.743 |  |  0.870 |  0.437 |  0.647 |  0.522 |  0.849
IRMAD [35] |  0.900 |  0.155 |  0.472 |  0.204 |  0.864 |  |  0.822 |  0.162 |  0.507 |  0.223 |  0.722 |  |  0.843 |  0.394 |  0.789 |  0.526 |  0.885
MAD [34] |  0.877 |  0.143 |  0.534 |  0.191 |  0.857 |  |  0.770 |  0.121 |  0.594 |  0.189 |  0.714 |  |  0.815 |  0.348 |  0.782 |  0.482 |  0.871
PCDA [14] |  0.855 |  0.192 |  0.456 |  0.210 |  0.805 |  |  0.758 |  0.119 |  0.598 |  0.183 |  0.737 |  |  0.862 |  0.425 |  0.728 |  0.537 |  0.889
DeepCVA [42] |  0.916 |  0.216 |  0.350 |  0.245 |  0.835 |  |  0.859 |  0.259 |  0.277 |  0.268 |  0.847 |  |  0.863 |  0.432 |  0.544 |  0.482 |  0.891
Supervised CD methods (supervised on the OSCD training set and tested on the
test set of the respective dataset):
UNet [50] |  0.946 |  0.466 |  0.176 |  0.186 |  0.776 |  |  0.936 |  0.187 |  0.098 |  0.088 |  0.718 |  |  0.844 |  0.344 |  0.355 |  0.349 |  0.839
SeCo-(Rand) [32] |  0.946 |  0.477 |  0.150 |  0.180 |  0.812 |  |  0.907 |  0.234 |  0.306 |  0.210 |  0.720 |  |  0.880 |  0.506 |  0.488 |  0.497 |  0.777
SeCo-(Pre) [32] |  0.928 |  0.412 |  0.182 |  0.195 |  0.747 |  |  0.860 |  0.198 |  0.363 |  0.199 |  0.691 |  |  0.905 |  0.654 |  0.588 |  0.619 |  0.867
Deep Metric Learning CD:
Ours |  0.958 |  0.383 |  0.480 |  0.325 |  0.937 |  |  0.948 |  0.325 |  0.256 |  0.286 |  0.913 |  |  0.908 |  0.486 |  0.697 |  0.573 |  0.928

##  IV Experimental Setup

#### Datasets.

We evaluate the performance of our proposed CD framework on three widely used
and publically available CD datasets, namely Onera Satellite Change Detection
Dataset (abbreviated as OCSD) [7], SZTAKI AirChange Benchmark set (abbreviated
as SZTAKI) [5, 6], and QuickBird dataset [55]. The OSCD dataset consists of 24
pairs of 13-band ($b$ = 13) multispectral images taken from the Sentinel-2
satellites [16], captured all over the world (such as Brazil, the US, Europe,
the Middle East, and Asia) between 2015 and 2018. Note that we present results
on the testing set of the OSCD dataset. In addition, the SZTAKI dataset
contains 13 aerial image pairs of size $952\times 640$ and the corresponding
binary change masks. The bi-temporal images are taken with a 23-years of time
difference. We use all 13 image pairs of the SZTAKI dataset for testing.
Furthermore, the QuickBird dataset consists of a three-band bi-temporal image
pair of size $1154\times 740$ which was captured in 2009 and 2014. In the
QuickBird dataset, seasonal changes are very prominent - making it difficult
to filter out relevant changes from unwanted ones compared to the other two
datasets.

#### Implementation details and reproducibility.

We implemented our unsupervised CD framework in Pytorch with an NVIDIA Quadro
RTX 8000 GPU. As we discussed in Sec. III, our CD framework does not involve a
training phase, unlike supervised or self-supervised CD approaches. Instead,
we iteratively optimize the parameters of the network for each MT-RSI
separately. After a fixed number of iterations (set to 80), we save the
resulting change probability map from the network (note that we do not save
the network parameters), and obtain a binary change map by thresholding it
with an appropriate probability (set to 0.5, but can manually vary for better
performance). We initialize multi-level D-FE from VGG-16 [46] pre-trained
weights on ImageNet [24]. Additional experiments on how CD performance varies
with different initialization methods can be found in the supplementary
document. For all the experiments, we used Adam optimizer with a learning rate
of 1e-5. Implementation code will be made publicly available after the review
process.

#### Performance metrics and visualization of change maps.

We adopt the overall accuracy (OA), user accuracy (UA) - often called as
precision, recall rate (Recall), F-1 measure (F1), and Area Under the Receiver
Operating Characteristic (ROC) curve (AUC) to quantitatively compare the CD
performance of different methods. Among these, AUC is the best way to
summarize a model’s ability to distinguish between change and no change
regions. A higher value of AUC indicates a good CD model. For the qualitative
comparisons, we highlight each pixel in the predicted change map with
different colors to denote true positives (TP), true negatives (TN), false
positives (FP), and false negatives (FN). Thus, a change map with more green
and white pixels, and fewer red and blue pixels indicates a good CD
performance.

Figure 4: Average ROC curves corresponding to different CD methods on (a) OSCD
[7], (b) SZTAKI [5, 6], and (c) QuickBird datasets [55]. Figure 5: Qualitative
results on lasvegas image in the OSCD dataset. (a) $\mathbf{I}_{1}$. (b)
$\mathbf{I}_{2}$. (c) Reference change mask. (d) ImageDiff [31]. (e)
ImageRatio [28]. (f) ImageRegr [30]. (g) CVA [43]. (h) DPCA [13]. (i) MAD
[34]. (j) IRMAD [35]. (k) PCDA [14]. (l) Deep-CVA [42]. (m) SeCo [32]: Self-
supervised pre-trained on Sentinel-2 [16] and supervised on the OSCD [7]
training set. (n) Deep Metric Learning CD (Ours).  Figure 6: Qualitative
results on the archive image in the SZTAKI dataset [5, 6]. (a)
$\mathbf{I}_{1}$. (b) $\mathbf{I}_{2}$. (c) Reference change mask. (d)
ImageDiff [31]. (e) ImageRatio [28]. (f) ImageRegr [30]. (g) CVA [43]. (h)
DPCA [13]. (i) MAD [34]. (j) IRMAD [35]. (k) PCDA [14]. (l) Deep-CVA [42]. (m)
SeCo [32]: Self-supervised pre-trained on Sentinel-2 [16] and supervised on
the OSCD [7] training set. (n) Deep Metric Learning CD (Ours). Figure 7:
Qualitative results on the QuickBird dataset [55]. (a) $\mathbf{I}_{1}$. (b)
$\mathbf{I}_{2}$. (c) Reference change mask. (d) ImageDiff [31]. (e)
ImageRatio [28]. (f) ImageRegr [30]. (g) CVA [43]. (h) DPCA [13]. (i) MAD
[34]. (j) IRMAD [35]. (k) PCDA [14]. (l) Deep-CVA [42]. (m) SeCo [32]: Self-
supervised pre-trained on Sentinel-2 [16] and supervised on the OSCD [7]
training set. (n) Deep Metric Learning CD (Ours).

##  V Results and Discussion

#### Quantitative results.

Table I presents average quantitative results corresponding to different CD
methods. From these results, we can make the following key observations: 1.
The proposed CD method outperforms the existing unsupervised approaches
(classical and deep) by a significant margin, especially in OA, F1 and AUC
metrics. 2. It even outperforms recent deep supervised approaches, which are
pre-trained in an unsupervised manner on a large RS (SeCo- (P) [32]) dataset
and then fine-tuned on the OSCD training set in a supervised manner. The
latter observation proves the point we made in the introduction, where the
supervised CD approaches are weak at generalizing under domain shifts, which
is common in the datasets we consider here. Since the proposed CD approach
addresses this problem by optimizing the network parameters for a given MT-RSI
separately to obtain the change probability map, it shows consistently higher
CD performance in OA, F1, and AUC metrics than the SOTA approaches.
Furthermore, we visualize the ROC curves corresponding to different CD methods
in Fig. 4. From these graphs, one can clearly see the significance of the
proposed CD approach compared to SOTA CD methods.

#### Qualitative results.

We present qualitative visualizations of the predicted change maps from
different CD algorithms on the OSCD, SZTAKI, and QuickBird datasets in Fig. 5,
Fig. 6, and Fig. 7, respectively. We can see that the proposed CD can capture
most of the relevant changes that are difficult to capture by the SOTA CD
methods (see the highlighted regions by bounding boxes) while minimizing false
positives and false negative predictions. Please see supplementary document
for additional qualitative results.

Figure 8:  The effect of PCC on rio image pair in the OSCD [7].

##  VI Ablation Study

#### Effect of PCC.

Fig. 8 shows an example of how PCC minimizes the global and local color
changes in MT-RSIs. After PCC, the pre-change image’s $(\mathbf{I}_{1})$ color
space is mapped to the post-change image’s $(\mathbf{I}_{2})$ color space -
making the difference image $(\mathbf{I}_{d})$ to be less sensitive to
colorimetric changes. Please see supplementary document for additional
examples.

Figure 9: Average ROC curves for different architectures of D-CPG on OSCD [7]. Network |  | AUC
---|---|---
U-Net |  | 0.847
Hourglass |  | 0.852
ResNet (N=4) |  | 0.890
ResNet (N=8) |  | 0.908
ResNet (N=16) |  | 0.922
ResNet (N=32) |  | 0.937
ResNet (N=64) |  | 0.925
Table II: Caption

#### Experiments with different architectures for D-CPG.

We experimented with different network architectures for D-CPG, and the
results are summarized in Fig. 9 and Tab. II. We observe a higher AUC when we
cascade 32 residual blocks (denoted as ResNet ($N=32$)) compared to Hourglass,
U-Net, and other variations of ResNet. Furthermore, we generally observe
faster convergence when we utilize ResNet architecture than U-Net and
Hourglass networks. Therefore, for all of our experiments, we utilized ResNet
with $N=32$.

Figure 10: How our CD method converges to the optimal change map
$\mathbf{P}^{*}_{\text{c}}$ during testing on the beirut image pair in OSCD
[7].

#### Convergence characteristic.

Fig. 10 shows an example of how the proposed approach converges to the optimal
change probability map $\mathbf{P}^{*}_{c}$ for a given bi-temporal image
pair.

Table III: Contribution from each loss term in $\mathcal{L}_{\text{CD}}$ on the CD performance for $l\in\\{1,2\\}$(averaging over the test-set of OSCD [7]). $\mathcal{L}_{\text{img}}$ | $\mathcal{L}_{\text{feat}}$ | $\mathcal{L}_{\text{ctx}}$ | OA | F1 | AUC
---|---|---|---|---|---
✓ | ✗ | ✗ | 0.911 | 0.222 | 0.821
✗ | ✓ | ✗ | 0.936 | 0.296 | 0.879
✗ | ✓ | ✓ | 0.953 | 0.319 | 0.926
✓ | ✓ | ✗ | 0.940 | 0.303 | 0.892
✓ | ✓ | ✓ | 0.958 | 0.325 | 0.937

#### Contribution from different loss terms.

Table III summarizes the contribution from each loss term -
$\mathcal{L}_{\text{img}}$, $\mathcal{L}_{\text{feat}}$, and
$\mathcal{L}_{\text{ctx}}$ on the CD performance. We can observe that
computing similarity-dissimilarity loss in the feature domain
($\mathcal{L}_{\text{feat}}$) improves the CD performance in OA, F1, and AUC
metrics significantly over the image domain loss ($\mathcal{L}_{\text{img}}$).
Furthermore, imposing context-consistency constraint on deep features
($\mathcal{L}_{\text{ctx}}$) on top of $\mathcal{L}_{\text{feat}}$ results in
a significant boost in the CD performance. Moreover, combining the three loss
terms - $\mathcal{L}_{\text{img}}$, $\mathcal{L}_{\text{feat}}$, and
$\mathcal{L}_{\text{ctx}}$ with appropriate regularization constants further
improves the CD performance.

Table IV: Effect of utilizing different scales of deep features from D-FE on the CD performance (averaging over test-set of OSCD [7]). Loss function configuration | OA | F1 | AUC
---|---|---|---
Baseline: $\mathcal{L}_{\text{img}}$ | 0.911 | 0.222 | 0.821
$\mathcal{L}_{\text{img}}$+$\mathcal{L}_{\text{feat}}^{l}+\mathcal{L}_{\text{ctx}}^{l}$; $l\in\\{1\\}$ | 0.936 | 0.298 | 0.855
$\mathcal{L}_{\text{img}}$+$\mathcal{L}_{\text{feat}}^{l}+\mathcal{L}_{\text{ctx}}^{l}$; $l\in\\{1,2\\}$ | 0.958 | 0.325 | 0.937
$\mathcal{L}_{\text{img}}$+$\mathcal{L}_{\text{feat}}^{l}+\mathcal{L}_{\text{ctx}}^{l}$; $l\in\\{1,2,3\\}$ | 0.943 | 0.315 | 0.929
$\mathcal{L}_{\text{img}}$+$\mathcal{L}_{\text{feat}}^{l}+\mathcal{L}_{\text{ctx}}^{l}$; $l\in\\{1,2,3,4\\}$ | 0.932 | 0.307 | 0.914

#### Performance with different scales of features from D-FE.

According to the results summarized in Tab. IV, the best CD performance is
observed when we utilize deep features from the first two hierarchical scales
of VGG-16 (i.e., $l$=1 and 2). Utilizing additional scales of deep features
result in degradation of the CD performance - so for all of our experiments we
set $l=2$ in $\mathcal{L}_{\text{feat}}$ (Eqn. 6) and
$\mathcal{L}_{\text{ctx}}$ (Eqn. 7).

##  VII Conclusion

Society is becoming increasingly aware of human activities on the global
climate. There is overwhelming evidence that these activities have short- and
long-term effects on almost every aspect of our lives. Using global climate
measurements and simulations, it is now possible to observe changes on a
global scale, such as sea level rise or changes in the Gulf Stream. On the
other hand, accurate predictions of local changes are much more difficult to
obtain. Common examples include land use for agriculture, deforestation,
flooding, forest fires, growth of urban areas, and transportation
infrastructure. Therefore, it is extremely important to monitor these local
changes as these are the factors that ultimately exacerbate the global climate
crisis.

## References

  * [1] Deep white-balance editing. In: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition. pp. 1397–1406 (2020)
  * [2] Bandara, W.G.C., Nair, N.G., Patel, V.M.: Ddpm-cd: Remote sensing change detection using denoising diffusion probabilistic models (2022). https://doi.org/10.48550/ARXIV.2206.11892, https://arxiv.org/abs/2206.11892
  * [3] Bandara, W.G.C., Patel, V.M.: Revisiting consistency regularization for semi-supervised change detection in remote sensing images (2022). https://doi.org/10.48550/ARXIV.2204.08454, https://arxiv.org/abs/2204.08454
  * [4] Bandara, W.G.C., Patel, V.M.: A transformer-based siamese network for change detection. In: IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium. pp. 207–210 (2022). https://doi.org/10.1109/IGARSS46834.2022.9883686
  * [5] Benedek, C., Szirányi, T.: A mixed markov model for change detection in aerial photos with large time differences. In: 2008 19th International Conference on Pattern Recognition. pp. 1–4. IEEE (2008)
  * [6] Benedek, C., Szirányi, T.: Change detection in optical aerial images by a multilayer conditional mixed markov model. IEEE Transactions on Geoscience and Remote Sensing 47(10), 3416–3430 (2009)
  * [7] Caye Daudt, R., Le Saux, B., Boulch, A., Gousseau, Y.: Urban change detection for multispectral earth observation using convolutional neural networks. In: IEEE International Geoscience and Remote Sensing Symposium (IGARSS) (July 2018)
  * [8] Celik, T.: Unsupervised change detection in satellite images using principal component analysis and $k$-means clustering. IEEE Geoscience and Remote Sensing Letters 6(4), 772–776 (2009)
  * [9] Chen, Y., Bruzzone, L.: Self-supervised change detection in multi-view remote sensing images. arXiv preprint arXiv:2103.05969 (2021)
  * [10] Coppin, P., Jonckheere, I., Nackaerts, K., Muys, B., Lambin, E.: Review articledigital change detection methods in ecosystem monitoring: a review. International journal of remote sensing 25(9), 1565–1596 (2004)
  * [11] Daudt, R.C., Le Saux, B., Boulch, A.: Fully convolutional siamese networks for change detection. In: 2018 25th IEEE International Conference on Image Processing (ICIP). pp. 4063–4067. IEEE (2018)
  * [12] De Maesschalck, R., Jouan-Rimbaud, D., Massart, D.L.: The mahalanobis distance. Chemometrics and intelligent laboratory systems 50(1), 1–18 (2000)
  * [13] Deng, J., Wang, K., Deng, Y., Qi, G.: Pca-based land-use change detection and analysis using multitemporal and multisensor satellite data. International Journal of Remote Sensing 29(16), 4823–4838 (2008)
  * [14] Dharani, M., Sreenivasulu, G.: Land use and land cover change detection by using principal component analysis and morphological operations in remote sensing applications. International Journal of Computers and Applications 43(5), 462–471 (2021)
  * [15] Di Pilato, A., Taggio, N., Pompili, A., Iacobellis, M., Di Florio, A., Passarelli, D., Samarelli, S.: Deep learning approaches to earth observation change detection. Remote Sensing 13(20), 4083 (2021)
  * [16] Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Hoersch, B., Isola, C., Laberinti, P., Martimort, P., et al.: Sentinel-2: Esa’s optical high-resolution mission for gmes operational services. Remote sensing of Environment 120, 25–36 (2012)
  * [17] Finlayson, G.D., Mackiewicz, M., Hurlbert, A.: Color correction using root-polynomial regression. IEEE Transactions on Image Processing 24(5), 1460–1470 (2015)
  * [18] Hartigan, J.A., Wong, M.A.: Algorithm as 136: A k-means clustering algorithm. Journal of the royal statistical society. series c (applied statistics) 28(1), 100–108 (1979)
  * [19] He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)
  * [20] Hong, G., Luo, M.R., Rhodes, P.A.: A study of digital camera colorimetric characterization based on polynomial modeling. Color Research & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur 26(1), 76–84 (2001)
  * [21] de Jong, K.L., Bosman, A.S.: Unsupervised change detection in satellite images using convolutional neural networks. In: 2019 International Joint Conference on Neural Networks (IJCNN). pp. 1–8. IEEE (2019)
  * [22] Khelifi, L., Mignotte, M.: Deep learning for change detection in remote sensing images: Comprehensive review and meta-analysis. IEEE Access 8, 126385–126400 (2020)
  * [23] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
  * [24] Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25, 1097–1105 (2012)
  * [25] Kuncheva, L.I., Faithfull, W.J.: Pca feature extraction for change detection in multidimensional unlabeled data. IEEE transactions on neural networks and learning systems 25(1), 69–80 (2013)
  * [26] Leenstra, M., Marcos, D., Bovolo, F., Tuia, D.: Self-supervised pre-training enhances change detection in sentinel-2 imagery. arXiv preprint arXiv:2101.08122 (2021)
  * [27] Lu, D., Mausel, P., Brondizio, E., Moran, E.: Change detection techniques. International journal of remote sensing 25(12), 2365–2401 (2004)
  * [28] Lu, D., Mausel, P., Brondizio, E., Moran, E.: Change detection techniques. International journal of remote sensing 25(12), 2365–2401 (2004)
  * [29] Luo, W., Li, Y., Urtasun, R., Zemel, R.: Understanding the effective receptive field in deep convolutional neural networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. pp. 4905–4913 (2016)
  * [30] Luppino, L.T., Bianchi, F.M., Moser, G., Anfinsen, S.N.: Unsupervised image regression for heterogeneous change detection. IEEE Transactions on Geoscience and Remote Sensing 57(12), 9960–9975 (2019). https://doi.org/10.1109/TGRS.2019.2930348
  * [31] MAHMOUDZADEH, H.: Digital change detection using remotely sensed data for monitoring green space destruction in tabriz (2007)
  * [32] Mañas, O., Lacoste, A., Giro-i Nieto, X., Vazquez, D., Rodriguez, P.: Seasonal contrast: Unsupervised pre-training from uncurated remote sensing data. arXiv preprint arXiv:2103.16607 (2021)
  * [33] Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: European conference on computer vision. pp. 483–499. Springer (2016)
  * [34] Nielsen, A.A., Conradsen, K., Simpson, J.J.: Multivariate alteration detection (mad) and maf postprocessing in multispectral, bitemporal image data: New approaches to change detection studies. Remote Sensing of Environment 64(1), 1–19 (1998)
  * [35] Nielsen, A.A.: The regularized iteratively reweighted mad method for change detection in multi-and hyperspectral data. IEEE Transactions on Image processing 16(2), 463–478 (2007)
  * [36] Pan, S.J., Yang, Q.: A survey on transfer learning. IEEE Transactions on knowledge and data engineering 22(10), 1345–1359 (2009)
  * [37] Qian, Q., Tang, J., Li, H., Zhu, S., Jin, R.: Large-scale distance metric learning with uncertainty. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 8542–8550 (2018)
  * [38] Radke, R.J., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: a systematic survey. IEEE transactions on image processing 14(3), 294–307 (2005)
  * [39] Ren, C., Wang, X., Gao, J., Zhou, X., Chen, H.: Unsupervised change detection in satellite images with generative adversarial network. IEEE Transactions on Geoscience and Remote Sensing (2020)
  * [40] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. pp. 234–241. Springer (2015)
  * [41] Saha, S., Bovolo, F., Bruzzone, L.: Unsupervised deep change vector analysis for multiple-change detection in vhr images. IEEE Transactions on Geoscience and Remote Sensing 57(6), 3677–3693 (2019). https://doi.org/10.1109/TGRS.2018.2886643
  * [42] Saha, S., Bovolo, F., Bruzzone, L.: Unsupervised deep change vector analysis for multiple-change detection in vhr images. IEEE Transactions on Geoscience and Remote Sensing 57(6), 3677–3693 (2019). https://doi.org/10.1109/TGRS.2018.2886643
  * [43] Saha, S., Bovolo, F., Bruzzone, L.: Unsupervised deep change vector analysis for multiple-change detection in vhr images. IEEE Transactions on Geoscience and Remote Sensing 57(6), 3677–3693 (2019)
  * [44] Shi, W., Zhang, M., Zhang, R., Chen, S., Zhan, Z.: Change detection based on artificial intelligence: State-of-the-art and challenges. Remote Sensing 12(10), 1688 (2020)
  * [45] Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. Journal of Big Data 6(1), 1–48 (2019)
  * [46] Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  * [47] Singh, A.: Review article digital change detection techniques using remotely-sensed data. International journal of remote sensing 10(6), 989–1003 (1989)
  * [48] Skifstad, K., Jain, R.: Illumination independent change detection for real world image sequences. Computer vision, graphics, and image processing 46(3), 387–399 (1989)
  * [49] St-Charles, P.L., Bilodeau, G.A., Bergevin, R.: Subsense: A universal change detection method with local adaptive sensitivity. IEEE Transactions on Image Processing 24(1), 359–373 (2014)
  * [50] Sun, S., Mu, L., Wang, L., Liu, P.: L-unet: An lstm network for remote sensing image change detection. IEEE Geoscience and Remote Sensing Letters (2020)
  * [51] Tang, H., Chu, S., Hasegawa-Johnson, M., Huang, T.: Partially supervised speaker clustering. IEEE transactions on pattern analysis and machine intelligence 34(5), 959–971 (2011)
  * [52] Walter, V.: Object-based classification of remote sensing data for change detection. ISPRS Journal of photogrammetry and remote sensing 58(3-4), 225–238 (2004)
  * [53] Wiratama, W., Sim, D.: Fusion network for change detection of high-resolution panchromatic imagery. Applied Sciences 9(7), 1441 (2019)
  * [54] Xing, E., Jordan, M., Russell, S.J., Ng, A.: Distance metric learning with application to clustering with side-information. Advances in neural information processing systems 15, 521–528 (2002)
  * [55] Zhang, M., Shi, W.: A feature difference convolutional neural network-based change detection method. IEEE Transactions on Geoscience and Remote Sensing 58(10), 7232–7246 (2020). https://doi.org/10.1109/TGRS.2020.2981051
  * [56] Zhao, W., Mou, L., Chen, J., Bo, Y., Emery, W.J.: Incorporating metric learning and adversarial network for seasonal invariant change detection. IEEE Transactions on Geoscience and Remote Sensing 58(4), 2720–2731 (2020). https://doi.org/10.1109/TGRS.2019.2953879