A Self-supervised SAR Image Despeckling Strategy Based on
  Parameter-sharing Convolutional Neural Networks
Speckle noise is generated due to the SAR imaging mechanism, which brings
difficulties in SAR image interpretation. Hence, despeckling is a helpful step
in SAR pre-processing. Nowadays, deep learning has been proved to be a
progressive method for SAR image despeckling. Most deep learning methods for
despeckling are based on supervised learning, which needs original SAR images
and speckle-free SAR images to train the network. However, the speckle-free SAR
images are generally not available. So, this issue was tackled by adding
multiplicative noise to optical images synthetically for simulating speckled
image. Therefore, there are following challenges in SAR image despeckling: (1)
lack of speckle-free SAR image; (2) difficulty in keeping details such as edges
and textures in heterogeneous areas. To address these issues, we propose a
self-supervised SAR despeckling strategy that can be trained without
speckle-free images. Firstly, the feasibility of SAR image despeckling without
speckle-free images is proved theoretically. Then, the sub-sampler based on the
adjacent-syntropy criteria is proposed. The training image pairs are generated
by the sub-sampler from real-word SAR image to estimate the noise distribution.
Furthermore, to make full use of training pairs, the parameter sharing
convolutional neural networks are adopted. Finally, according to the
characteristics of SAR images, a multi-feature loss function is proposed. The
proposed loss function is composed of despeckling term, regular term and
perception term, to constrain the gap between the generated paired images. The
ability of edge and texture feature preserving is improved simultaneously.
Finally, qualitative and quantitative experiments are validated on real-world
SAR images, showing better performances than several advanced SAR image
despeckling methods.
Liang Chen, Yifei Yin, Hao Shi, Qingqing Sheng, Wei Li
A Self-supervised SAR Image Despeckling Strategy Based on Parameter-sharing Convolutional Neural Networks

###### Abstract

Speckle noise is generated due to the SAR imaging mechanism, which brings difficulties in SAR image interpretation. Hence, despeckling is a helpful step in SAR pre-processing. Nowadays, deep learning has been proved to be a progressive method for SAR image despeckling. Most deep learning methods for despeckling are based on supervised learning, which needs original SAR images and speckle-free SAR images to train the network. However, the speckle-free SAR images are generally not available. So, this issue was tackled by adding multiplicative noise to optical images synthetically for simulating speckled image. Therefore, there are following challenges in SAR image despeckling: (1) lack of speckle-free SAR image; (2) difficulty in keeping details such as edges and textures in heterogeneous areas. To address these issues, we propose a self-supervised SAR despeckling strategy that can be trained without speckle-free images. Firstly, the feasibility of SAR image despeckling without speckle-free images is proved theoretically. Then, the sub-sampler based on the adjacent-sentropy criteria is proposed. The training image pairs are generated by the sub-sampler from real-word SAR image to estimate the noise distribution. Furthermore, to make full use of training pairs, the parameter sharing convolutional neural networks are adopted. Finally, according to the characteristics of SAR images, a multi-feature loss function is proposed. The proposed loss function is composed of despeckling term, regular term and perception term, to constrain the gap between the generated paired images. The ability of edge and texture feature preserving is improved simultaneously. Finally, qualitative and quantitative experiments are validated on real-world SAR images, showing better performances than several advanced SAR image despeckling methods.

 self-supervised; synthetic aperture radar (SAR); despeckling; deep learning

## I Introduction

As the main source of earth observation in remote sensing technology, synthetic aperture radar (SAR) provides the advantage of acquiring data in all-day and all-weather. Therefore, SAR image has been widely used for many applications, such as target detection, disaster management and geological exploration. However, speckle noise is an inherent factor causing SAR image degradation. Speckle noise is generated from the coherent sum of backscattered signals during imaging. The speckle noise degrades SAR image quality and interferes with interpretation. [1]. Hence, in SAR image pre-processing task, the reduction of speckle noise is an indispensable step.

In the past few years, there are many SAR image despeckling approaches have been proposed. In general, the existing SAR image despeckling methods can be classified as: filter-based methods, transform domain methods, variational methods, Markovian model-based methods, non-local mean (NLM) methods, dictionary learning and deep learning (DL) methods.

Lee filters[5], Frost filters[6] and Kuan filters[7] are the most common filter-based methods. These methods are typically assumed to operate on homogeneous regions. Thus, the edge and texture often fail to be preserved in real-world SAR images. Wavelet based[8] methods can distinguish signal and noise effectively according to the characteristic difference in wavelet domain. However, there are still some unignorable defects. The higher-dimensional features are hard to be expressed, resulting in blurred edge and texture features. Another important branch of speckle suppression is the Total Variation (TV)-based method [9]. By combining a regularization term with a data fidelity term, the TV method achieves a balance between smoothing noise and preserving edge. While effective at reducing noise and maintaining edge texture, this approach has its limitations of reducing SAR images' quality. Markov random field (MRF) methods build the model of contextual information to provide a quantitative depiction about the prior of image information. The MRF methods are widely used for SAR image despeckling [11]. Complex prior limits the application of the MRF-based SAR image despeckling methods.

By contrast, non-local mean (NLM) methods have improved performance significantly. These methods utilize the similarities between image patches to perform weighted filtering across the entire image, which benefits the preservation of details. The methods, such as probabilistic patch based (PPB) filtering method[12] and block-matching 3D (BM3D) algorithm[13], demonstrate the satisfying SAR image despeckling performance. However, the effectiveness of non-local mean methods in reducing noise largely depends
on the setteing of parameters. Moreover, it is inefficient when dealing with large-scale images. As an adaptive and straightforward approach, the sparse representation and dictionary learning have been able to reconstruct the image via pixel-by-pixel processing[14]. Whereas, this technique neglects the pixel correlation, and has a considerable computational burden. Therefore, the despeckling method still needs to be further improved.

In general, most of above methods face a few significant problems. First, the algorithm parameters depend on empirical setting. Second, the performance is scene-dependent and with poor adaptability. Third, there are sometimes artifacts in flat areas.

Recently, with the rapid development of deep learning, many denoising methods based on convolutional neural networks (CNNs) have been proposed. The CNN based methods have shown effective performance, such as DnCNN[15], FFDNet[16] and CBDNet[17]. Subsequently, SAR-CNN[18] is proposed for despeckling on SAR images. To convert speckle noise from multiplicative noise to additive noise, SAR-CNN performs homomorphic processing through log-transformation, and maps the despeckled image back to original domain through the exponential function to obtain the restored image. The CNN based method has demonstrated the effectiveness for SAR image despeckling, but acquisition of multitemporal data makes it challenging. Then, the ID-CNN [19] and SAR-DRN [20] methods are proposed which further improve the ability to remove despeckle noise. However, above supervised learning based despeckling methods focus on novel network architectures, ignoring the key problem: lack of speckle-free SAR images as ground truth in real-world condition. There are mainly two ways to solve this problem. The first strategy is synthetic speckle generation approach[21]. The multiplicative noise is overlayed onto clean grey optical images to simulate speckled images. The speckled images are treated as inputs and the clean grey optical images are treated as ground truth. The above two kinds of images serve as training data. However, the learned texture and edge pattern priors from optical images are inconsistent with the characteristics of real-world SAR images. The mismatch often leads to domain gap which is the source of the problem. The second strategy is multitemporal fusion approach[23]. A pile of real-world SAR images are acquired at varying temporal instants. The clean ground truth is generated through exploiting the temporal incoherence of the stack of real-world SAR images. However, accessing to a large multitemporal database remains an enormous challenge. And the change of the scene content may result in poor accuracy. To sum up, self-supervised methods without speckle-free images have become a hot topic for SAR image despeckling.

In this study, a self-supervised SAR image despeckling strategy is proposed that addresses the drawbacks of supervised learning-based methods. The proposed strategy allows for training directly on real-world SAR images using the improved generation training pairs module, the parameter-sharing convolutional neural network, and the self-supervised loss function. Experiments including quantitative and qualitative comparison are conducted on real-world SAR images. The results showed that, compared with the advanced despeckling methods, our proposed strategy can significantly suppress speckle noise, preserving SAR image features and detailed information simultaneously. The main innovations and contributions of our proposed strategy are summarized as shown below:

1. The proposed adjacent-syntropy sub-sampler allows for direct training on real-world SAR images without speckle-free images. Any existing despeckling networks can be trained without any speckle-free targets by this sub-sampler.
2. The proposed parameter-sharing convolutional neural networks can exploit the information of images more effectively. It takes the similarity between pairs of noisy images into consideration to boost despeckling performance.
3. The proposed multi-feature loss function considering the characteristics and properties of SAR images can accelerate the convergence speed, reduce spatial distortion and preserve edges.

The rest of this paper is organized as follows. Section II introduces related work, which includes self-supervised optical image denoising methods and self-supervised SAR image despeckling methods. Section III illustrates the deatils of our proposed method. Section IV introduces the implementation details of experiment, evaluation metrics, the compared methods, and experimental results on actual SAR images. Section V summarizes the paper.

## II Related Work

As discussed, there is no speckle-free SAR images to use for the supervised methods. On this account, the self-supervised despeckling methods have developed rapidly in recent years. The self-supervised methods are first introduced for optical image denoising. And then the self-supervised methods are extended to SAR image despeckling.

### _Self-supervised denoising methods in optical image_

The concept of self-supervised denoising is introduced by Noise2Noise [25] for the first time. The input and target for training have the same underlying ground truth and the mean of them is equal to underlying ground truth. Only their realizations of the noise process are different. Noise2Noise proved that when pairs of noisy images meet the above conditions, a neural network can be trained on paired noisy images rather than pairs of noisy and clean images. However, making multiple observations of the scene with the same content is a major obstacle. In single-image denoising techniques, this problem does not present itself. Noise2Self [26] and Noise2Void [27] introduced the concept of blind-spot network. They can be trained using only individual noisy images. They depend on meticulous data model, by assuming the noise is spatially independent and acquiring statistical priors about the distribution of the data. However, specifying the data model is hard, particularly in real-world scenarios. In addition, the absence of valuable information caused by blind spot dramatically degrades the despeckling performance. Consequently, the applications of blind-spot networks are limited because of the heavy computation and low accuracy.
To overcome the limitations above, Neighbor2Neighbor [28] was proposed. It aims to train denoising networks solely with noisy images available, instead of modifying the structure of the despeckling network, and does not rely on noise models. Inspired by these works, the study of self-supervised SAR image despeckling has also been promoted.

### _Self-supervised despeckling methods in SAR image_

Many studies have attempted to despeckle without speckle-free SAR images. They are derived from optical image denoising methods. A self-supervised densely dilated CNN (BDSS-CNN) [29] for blind despeckling was designed. This method extends Noise2Noise approach to despeckling by generating paired noisy images through overlaying speckle noise which is multiplicative noise onto optical images. Then, BDSS-CNN is trained with the generated paired noisy images. Finally, the trained despeckling network is applied to despeckle actual SAR images. However, the BDSS-CNN only applies Noise2Noise to SAR image despeckling instead of solving the essential problem of lack for ground truth. Besides, a no-reference based SAR deep learning (NR-SAR-DL) filter [30] was proposed, which employed SAR images obtained from the same scene with the same sensor to train the despeckling network. NR-SAR-DL has good ability of suppressing speckle noise on real-world SAR images. It has good ability of point targets preservation and the radio metrics preservation. The main limitation of NR-SAR-DL is that the change of the scene content may result in poor accuracy. Speckle2void [31] proposed a self-supervised Bayesian framework for SAR despeckling, which extends the single-image blind-spot appraoch. This method depends on accurate noise models and blind-spot pixel priors. To compensate for the autocorrelation of the speckle process, Speckle2void employs a whitening preprocessing and a network with a variable blind-spot size. However, Speckle2void depends on accurate statistical modeling of the data which limits further improvements. SSD-SAR-BS [32], a self-supervised method for SAR image despeckling was subsequently proposed. The method only requires original speckled SAR images and uses Bernoulli sampling to generate image pairs as input and target for training a multiscale despeckling network. This approach allows for training despeckling networks using only real speckled SAR images. Its despeckling performance can be further improved.

Existing self-supervised methods for SAR image despeckling still have shortcomings in terms of computational performance, acquiring training data, poor despeckling performance, etc. In response to these challenges, we designed a self-supervised SAR image despeckling strategy, including a generation training pairs module, a parameter-sharing convolutional neural network, and a self-supervised training loss function.

## III Method

In this section, a self-supervised strategy is proposed to train despeckling network with speckled images. The theoretical proof of the method is given at first. The proposed strategy contains three parts. In the first part, it introduces the generate module of speckled images pairs by using adjacent-syntropy sub-sampler. In the second part, the parameter-sharing network is adopted to train the generated speckled paired images. For the third part, a multi-feature loss function is introduced to maintain texture and improve the performance.

### _Theoretical background of proposed strategy_

Speckle noise is an inherent factor causing SAR image degr

adation. Speckle noise is often modeled as multiplicative noise. \(Y\) is used to represent the speckled SAR image. \(X\) is used to represent the underlying ground truth of speckled SAR image. \(Y\) and \(X\) satisfy the following relationship:

\[\tilde{Y}=X\times N \tag{1}\]

where \(\times\) represents the element-wise product. \(N\) represents speckle noise. Speckle noise is modeled to follow Gamma distribution [33]. Speckle noise has unit mean. The probability density function \(P_{r}(\cdot)\) of \(N\) can be defined as:

\[P_{r}(N)=\frac{L^{L}N^{L-1}e^{-LN}}{\Gamma(L)},N\geq 0,L\geq 1, \tag{2}\]

where \(\Gamma(\cdot)\) represents the Gamma function. \(L\) represents the number of looks.

Considering the difficulty with dealing with multiplicative noise and the abundance of literature on removing additive noise, the homomorphic method is adopted by converting the SAR images to logarithmic domain. So, the equation (1) can be modified to:

\[\tilde{Y}=\tilde{X}+\tilde{N} \tag{3}\]

In the above formula, \(\tilde{X}\) represents the underlying speckle-free SAR image in logarithmic domain. \(\tilde{Y}\)represents the speckled SAR image in logarithmic domain. \(\tilde{N}\) represents the speckle noise in logarithmic domain. Specially,

\[\tilde{X} = \log X \tag{4}\] \[\tilde{Y} = \log Y\] (5) \[\tilde{N} = \log N \tag{6}\]

The noise in the logarithmic domain follows the Fisher-Tippett distribution [34]:

\[P(\tilde{N})=\frac{L^{L}}{\Gamma(L)}e^{L\tilde{N}}\exp(-Le^{\tilde{N}}) \tag{7}\]

In addition, the mean of noise in the logarithmic domain only depends on the number of looks:

\[E[\tilde{N}]=\psi(L)-\log(L) \tag{8}\]

The noisy image pair \((\texttt{g}_{1}(\tilde{Y}),\texttt{g}_{2}(\tilde{Y}))\) is generated by the sub-sampler from a single speckle image \(\tilde{Y}\). The sub-samplers will be described in detail in Section III-C. The noise is considered independent of the underlying ground truth. According to the sampling method, the contents of the two generated images \(\texttt{g}_{1}(\tilde{Y})\) and \(\texttt{g}_{2}(\tilde{Y})\) are almost the same. It means that the gap caused by sampled images will be sufficiently small. Consequently, the speckled pairs satisfy that:

\[E(g_{1}(\tilde{Y}))=E(g_{2}(\tilde{Y}))=E(\tilde{X})+E(\tilde{N})=E(\tilde{X} )+\psi(L)-\log(L) \tag{9}\]
Therefore, when constant \(\log(\text{L})-\Psi(\text{L})\) is added to the training pair, the above equation becomes:

\[E(g|\tilde{Y})+\log(L)-\psi(L))=E(g2(\tilde{Y})+\log(L)-\psi(L))-E(\tilde{X}) \tag{10}\]

\(\text{g}_{1}(\tilde{Y})+\log(L)-\Psi(L)\)is denoted as \(y_{1},\ \text{g}_{2}(\tilde{Y})+\log(L)-\Psi(L)\)is denoted as \(y_{2}\). \(y_{1}\) and \(y_{2}\) are taken as the data pair for training. The equation (10) can be rewritten as:

\[E(y_{1})=E(y_{2})=E(\tilde{X}) \tag{11}\]

Generally, in the visual task of image denoising, pairs of noisy image and clean image (\(y\), \(x\)) are required to train the supervised network \(f_{0}\). The empirical risk minimization task of loss function \(L\) becomes:

\[\operatorname*{arg\,min}_{\theta}\mathbb{E}_{(x,y)}\{L(f_{\theta}(y),x)\} \tag{12}\]

Furthermore, equation (12) is also equivalent to:

\[\operatorname*{arg\,min}_{\theta}\mathbb{E}_{x}\{\mathbb{E}_{y_{1}x}\{L(f_{ \theta}(y),x)\}\} \tag{13}\]

For the SAR images in the logarithm domain, the equation (13) should be written as:

\[\operatorname*{arg\,min}_{\theta}\mathbb{E}_{\tilde{X}}\{\mathbb{E}_{\tilde{ Y}|\tilde{X}}\{L(f_{\theta}(\tilde{Y}),\tilde{X})\}\} \tag{14}\]

Noise2Noise denotes that the optimal network parameters \(\theta\) of Equation (14) also remain unchanged, when the targets are corrupted with noise which has zero mean. Therefore, the speckle-free images can be replaced with speckled images that have the same content and the equal mean. The generated training pair (\(y_{1}\), \(y_{2}\)) meets the training conditions of Noise2Noise. \(y_{1}\) and \(y_{2}\) have the same content and equal mean with \(\tilde{X}\). The despeckling networks can be trained with data pair (\(y_{1}\), \(y_{2}\)). Concretely, in \(L2\) norm minimization task, calculating the expected squared difference between speckled images and corresponding ground truth can explain the feasibility. \(\{y_{i}\}_{i=1}^{N}\) represents the noisy target for training in place of clean ground truth. \(\{\tilde{X}_{i}\}_{i=1}^{N}\) represents the clean ground truth. \(N\) is the number of training pairs. \(E\{y_{i}\}=\tilde{X}_{i}\). The calculation of the expected squared difference between \(\{y_{1}\}_{i=1}^{N}\) and \(\{\tilde{X}_{i}\}_{i=1}^{N}\) is as follows:

\[\mathbb{E}_{y}[\frac{1}{N}\sum_{i}\tilde{X}_{i}-\frac{1}{N}\sum_{ i}y_{i}]^{2} \tag{15}\] \[=\frac{1}{N^{2}}[\mathbb{E}_{y}(\sum_{i}\tilde{X}_{i})^{2}-2 \mathbb{E}_{y}[\sum_{i}\tilde{X}_{i})(\sum_{i}y_{i})]+\mathbb{E}_{y}(\sum_{i }y_{i})^{2}]\] \[=\frac{1}{N^{2}}Var(\sum_{i}y_{i})\] \[=\frac{1}{N}[\frac{1}{N}\sum_{i}\sum_{j}Cov(y_{i},y_{j})]\]

If the noisy targets are not correlated with each other, the result of the above equation can be:

\[\frac{1}{N}[\frac{1}{N}\sum_{i}Var(\tilde{X}_{i})] \tag{16}\]

As \(N\) increases, the error between noisy target for training and clean ground truth decreases. Therefore, if \(N\) is large enough, the error is close to zero.

Figure 1: Overview of our self-supervised despeckling strategy. (a) The illustration of the training phase. (b) The flowchart of the inference phase.
Therefore, the empirical risk minimization task in this paper becomes:

\[\underset{\theta}{\arg\min}\mathbb{E}_{y_{2}}\{\mathbb{E}_{y_{1}|y_{2}}\{L(f_{ \theta}(y_{1}),y_{2})\}\} \tag{17}\]

In this way, the model is trained to obtain despeckled results without speckle-free images.

### _Overview of proposed strategy_

The overview of our proposed self-supervised despeckling strategy is shown in Fig. 1. Training phase and inference phase are both illustrated. In the training phase, there are two parts. In the first part, pairs of speckled images for training are generated. The original SAR image \(Y\) is transformed into log domain as log\(Y\). Then, a pair of sub-sampled images (g\({}_{i}\)(log\(Y\)), g\({}_{i}\)(log\(Y\))), are generated from the log\(Y\) with an adjacent-syntropy sub-sampler \(G=\) (g\({}_{i}\), g\({}_{2}\)). Subsequently, \(C\) is added to the paired images to get training pairs (\(y_{1}\), \(y_{2}\)), which will be put into the despeckling network as input and target respectively. There, \(y_{i}\)=g\({}_{i}\)(log\(Y\))+\(C\), \(y_{2}\)=g\({}_{i}\)(log\(Y\))+\(C\). The \(C\) is the negative number of speckle noise in the logarithmic domain:

\[C\!=\!\log(L)-\psi(L) \tag{18}\]

In the second part, the image pairs (\(y_{i}\), \(y_{2}\)) generated by sub-sampler are used to train despeckling networks. The parameter-sharing network takes \(y_{i}\) as input and takes \(y_{2}\) as target respectively. Under the multi-feature loss function, the better despeckling performance can be achieved. The multi-feature loss function includes three items to suppress speckle noise, accelerate the convergence speed, reduce spatial distortion and preserve edges. \(x_{l}\) is the despeckling result of \(y_{l}\) and \(x_{2}\) is the despeckling result of \(y_{2}\). \(x_{1}=f_{\theta}(y_{1})\) and \(x_{2}=f_{\theta}(y_{2})\). \(\hat{Y}\) is the despeckling result of \(\hat{f}_{\theta}(\log Y+C)\), where \(\hat{f}_{\theta}(\ \boldsymbol{\tau}\ )\) means that the despeckling process do not contribute to updating the gradient. That is, \(\hat{f}_{\theta}(\log Y+C)\) can be considered as an ordinary constant. \(X_{l}\) and \(X_{2}\) are generated images by sub-sampler of \(\hat{Y}\):

\[X_{1}=g_{1}(\hat{f}_{\theta}(\log Y+C)) \tag{19}\]

\[X_{2}=g_{2}(\hat{f}_{\theta}(\log Y+C)) \tag{20}\]

In the inference phase, the SAR image to be despeckled is processed at first. The SAR image is transformed into logarithm domain and added \(C\) on it. Then it is used as the input of trained despeckling network. The networks'  s output is the ground truth of sepckled SAR image in logarithm domain. The final despeckling result is obtained by performing exponential transformation of the output.

### _Adjacent-syntropy sub-sampler_

The logarithmic graph is denoted as log\(Y\). log\(Y\) is \(W\) wide and \(H\) high. According to the adjacent-syntropy rule, a sub-sampler is designed, in which a pair of noisy images (\(y_{i}\), \(y_{2}\)) are achieved from log\(Y\) to train despeckling networks. The illustration of the process to generate an image pair with an adjacent-syntropy sub-sampler is shown in Fig. 2. The

Fig. 2: Illustration of the process to generate image pairs with proposed adjacent-syntropy sub-sampler. The generated paired images, in different colors, are generated in the above way. (a) The horizontal mode. (b) The portrait mode.
generation process of the proposed adjacent-sentropy sub-sampler consists of three steps. Firstly, divide the original image into small patches. Take a square of side length k as the basic unit to divide the \(\log\)_Y_. \(\left[W/k\right]\times\left[H/k\right]\) patches can be achieved. Empirically, \(k\) is set as 2. Secondly, select the pixels for paired images. Two pixels of _i_-th row and _j_-th column patch are selected according to the adjacent-sentropy rule. Remarkably, selection has two modes: horizontal and portrait, which make sure that the two pixels satisfy the condition of being adjacent and in the same direction. Each mode is selected with a probability of 0.5. Once selected, the sub-images are sampled according to the mode. In this way, the underlying ground-truth of paired images are closer as far as possible and have similar texture structure. Then, the two pixels are taken to be the (_i,j_) column element of \(\text{g}_{1}(\log Y)\) and \(\text{g}_{2}(\log Y)\) respectively. Thirdly, repeat the second step to generate paired images. The image pair \(\left\{\text{g}_{1}(\log Y),\text{g}_{2}(\log Y)\right\}\) is finally obtained. The size of \(\left\{\text{g}_{1}(\log Y),\text{g}_{2}(\log Y)\right\}\) is \(\left[W/k\right]\times\left[H/k\right]\). By adding \(C\) to \(\left\{\text{g}_{1}(\log Y),\text{g}_{2}(\log Y)\right\}\), the final training image pair (\(y_{1}\), \(y_{2}\)) is obtained.

In conclusion, the adjacent-sentropy sub sampler can be used to acquire paired images from single speckled image for training. The adjacent characteristic can make sure that the underlying ground-truths of (\(y_{1}\), \(y_{2}\)) are close to each other, since corresponding selected pixels of (\(y_{i}\), \(y_{2}\)) are neighbors in the \(\log\)_Y_. The syntropy characteristic ensures that the texture structures of the sampled sub-images are not much different from each other, and the spatial distribution is similar. The adjacent-sentropy sub sampler meets that the gap between the \(y_{i}\) and \(y_{2}\) is small.

### _Parameter-sharing convolutional neural networks_

Theoretically, paired training images \(y_{i}\) and \(y_{2}\) are hoped to have the same underlying ground truth. However, there is still gap between \(y_{1}\) and \(y_{2}\). Single branch network is greatly affected by the gap. Considering the parameter sharing network can measure the similarity between images, the parameter-sharing network is adopted to narrow the gap of paired training images. Besides, an accurate despeckling model can be trained by exploiting the information of the paired training images twice. To realize parameter sharing, the despeckling network takes \(y_{t}\) to be the inference of \(y_{2}\) and then takes \(y_{2}\) to be the inference of \(y_{t}\) subsequently. The process can be seen as a two-branches network in Fig. 3. Each branch of the parameter-sharing network is a despeckling network such as SAR-CNN or SAR-DRN. The network's inputs are generated training pairs: (\(y_{1}\), \(y_{2}\)), and the outputs are \(x_{l}\) and \(x_{2}\) accordingly.

The SAR-CNN and SAR-DRN are employed as our baseline and call the parameter-sharing networks SAR-CNN\(+\)SDSS (Self-supervised Despeckling Strategy for SAR) and SAR-DRN\(+\)SDSS, respectively.

### _Multi-feature loss function_

In the proposed self-supervised strategy, the multi-feature loss function is designed to remove as much speckle noise as possible while achieving the goal of preserving image texture details. It takes into account the characteristics of SAR images and narrows the gap between the noise pairs generated by the sub-sampler. The proposed multi-feature loss function \(L\) is a linear sum of three terms: the despeckling term, the regularization term and the perception term.

MSE is used as the despeckling term. According to the derivation in Section III-A, MSE can be used as the loss function. To accelerate the convergence of MSE, it is formulated as:

\[L_{deep}=\frac{\left\|{x_{1}-y_{2}}\right\|_{2}^{2}}{\left\|{X_{1}-x_{1}} \right\|_{2}^{2}}+\frac{\left\|{x_{2}-y_{1}}\right\|_{2}^{2}}{\left\|{X_{2}-x_ {2}}\right\|_{2}^{2}} \tag{21}\]

There, \(y_{l}\) and \(y_{2}\) are the noisy image pairs for training respectively, \(x_{l}\) and \(x_{2}\) are the despeckled images of \(y_{l}\) and \(y_{2}\) obtained by the network. \(X_{l}\) and \(X_{2}\) are generated paired images from despeckled image. Considering that directly applying the despeckling term will lead to over-smoothing, a regularization term should be adopted to address the gap

Fig. 3: The presentation of adopted despeckling networks. Each branch of the parameter-sharing network is a despeckling network such as SAR-CNN or SAR-DRN. They are trained with generated paired images \(y_{l}\) and \(y_{2}\). (a) The structure of parameter-sharing SAR-CNN. (b) The structure of parameter-sharing SAR-DRN.
between the generated paired images. In the optimal situation, despeckling network and clean image satisfy that \(f_{\theta}^{*}(\log Y)=\log X\) and \(f_{\theta}^{*}(\texttt{g}_{1}(\texttt{g}_{1}(\log Y))=\texttt{g}_{1}(\log X)\).

The following regularized optimization term applies to the optimal network\(f_{\theta}^{*}\):

\[\begin{split} L_{reg}=&\parallel x_{1}-y_{2}+X_{1}- X_{2}\parallel+\\ &\parallel x_{2}-y_{1}+X_{1}-X_{2}\parallel\end{split} \tag{22}\]

To preserve edges and identify structures, a perceptual loss is used to generate high-quality despeckled images. First, VGG16 is used to extract the feature maps of the despeckled sub-image \(x_{i}\) and sub-image of despeckled image \(X_{i}\). Then MSE is used to compare the features of the two to make the high-level information (content and global structure) close to enhance the details. The perceptual term is expressed as:

\[\begin{split} L_{par}=&\frac{1}{CHW}\left\| \phi(X_{i})-\phi(x_{i})\right\|_{2}^{2},\\ i=& 1,2\end{split} \tag{23}\]

There, \(\phi\) is the VGG network and \(\phi(\cdot)\) represents a feature map of shape \(\mathcal{C}\times H\times W\).

The actual cost function is composed of three terms:

\[\begin{split} L=L_{deep}+\alpha L_{reg}+\beta L_{par}\end{split} \tag{24}\]

Specifically, the \(L_{deep}\) term is responsible for the reconstruction of the despeckled image space. It takes into account the characteristics of speckle noise as well as faster convergence. The \(\texttt{L}_{\text{reg}}\) term is to compensate for the gap between the generated paired images and prevent the despeckling effect from being too smooth. The \(L_{per}\) term is to preserve image texture details and edge information.

## IV Experiment

In this section, the performance about despeckling of our proposed self-supervised strategy is evaluated experimentally. First, the network SAR-CNN and the network SAR-DRN are trained with our strategy. They are full-supervised networks and are employed as our baseline. Second, quantitative and visual comparison experiments are performed to illustrate the superiority of our proposed strategy. Third, the comprehensive ablation studies are conducted to explain the effectiveness of our strategy further.

### _Implementation Details_

The patch size of adjacent-surrropy sub-samplers is fixed as 2. As for the parameters of loss functions, \(\alpha\) is set as 2 and \(\beta\) is set as 1 empirically. Adam optimizer is adopted and its initial learning rate is set as 0.0003. Per 20 epochs, the learning rate is decayed by half. The batch size of training data is set as 4 and the number of epochs is set as 200. The experimental framework is PyTorch 1.4.0. Our dataset is acquired from TerraSAR, Sentinel-1 and RADARSAT2 systems.

### _Evaluation Metrics_

#### Iv-B1 Enl

ENL[35] is a non-referenced metric, which is used to evaluate the performance of suppressing speckle noise by measuring smoothness in homogeneous region. ENL is defined as:

\[\textit{ENL}=\frac{\mu_{d}^{2}}{\sigma_{d}^{2}} \tag{25}\]

where the \(\mu_{d}\) is the mean of despeckled image and \(\sigma_{d}\) is the standard variance of despeckled image. If the value is higher, the performance of despeckling is better.

#### Iv-B2 Tcr

The TCR [36] value is used to evaluate the performance of retaining the scattering information. Specially, it is important to keep the high returns of strong scattering point in target detection tasks. By measuring the difference between scattering information in the speckled image and despeckled image. TCR is defined as:

\[TCR=\left|20\cdot\log 10\frac{\max(\hat{X})}{E(\hat{X})}-20\cdot\log 10\frac{ \max(X)}{E(X)}\right| \tag{26}\]

where \(X\) represents the patches of speckled image and \(\hat{X}\) represents the patches of despeckled images, respectively. The smaller the TCR value, the better the performance of retaining the scattering information.

#### Iv-B3 MoR

The MoR[37] value is used to measure the performance of preserving radiometric information in the despeckled images. MoR is defined as:

\[\textit{MoR}=\frac{1}{\textit{WH}}\sum_{w=1}^{W}\sum_{h=1}^{H}\frac{X_{w,h}^{ HR}}{\hat{X}_{w,h}^{HR}} \tag{27}\]

where \(X\) represents the patches of speckled image and \(\hat{X}\) represents the patches of despeckled images, respectively.

The highest value of MoR is 1. The closer to 1 the MoR value, the better the performance of preserving radiometric information.

#### Iv-B4 Epd-Roa

The EPD-ROA [38] can evaluate the performance of retaining edges for a filter, which is given by:

\[\begin{split} EPD-ROA=\frac{\sum_{i=1}^{M}\left\|\hat{X}_{A}(i) \right./\left.\hat{X}_{B}(i)\right\|}{\sum_{i=1}^{M}\left|X_{A}(i)\right./\left. X_{B}(i)\right|}\end{split} \tag{28}\]

Where \(\hat{X}_{A}(i)\) and \(\hat{X}_{B}(i)\) represent the adjacent pixel values of filtered images along the certain direction, respectively. \(\text{X}_{A}(i)\) and \(\text{X}_{B}(i)\) represent the adjacent pixel values of initial images along the certain direction, respectively. If the EPD-ROA value is higher, the performance of retaining edges is better.

### _Comparison with Baseline_

The SAR-CNN and SAR-DRN are employed as our baseline. Our self-supervised networks are referred to as SAR-CNN\(+\)SDSS and SAR-DRN\(+\)SDSS, respectively. In Fig. 4 and Fig. 5, the comparison results are presented.
It can be seen from Fig. 4 and Fig. 5, either compared with SAR-CNN or compared with SAR-DRN, our SDSS achieves better despeckling performance. Especially when it comes to preserving textures, our strategy has more obvious advantages. The supervised networks are trained with synthetic SAR images and tested on the real-world SAR images. However, the characteristics of SAR images are inconsistent with optical images, especially scattering property. Their performance is not good enough due to the domain gap. As shown in the red box of Fig. 4 and Fig. 5, the texture information is blurred or lost. In addition, scattering information from some strong point targets are weaken. On the opposite, our SDSS strategy solves the domain gap problems and retains edges well. Besides, in the respect of retaining the high returns from strong point targets, our SDSS strategy can preserve the scattering intensity, while the supervised networks reduce the scattering intensity because of their washing despeckled effect.
Figure 6: Comparison results of our proposed SAR-DRN+SDSS on Sentinel-1 image against other methods: Lee, SAR-BM3D, HDRANet and MONet. (a) Despeckling results of different methods. (b) Details of the despeckling images.

Figure 7: Comparison results of our proposed SAR-DRN+SDSS on TerraSAR image against other methods: Lee, SAR-BM3D, HDRANet and MONet. (a) Despeckling results of different methods. (b) Details of the despeckling images.
### _Comparison with State-of-the-art_

To reveal the despeckling performance of the proposed SDSS strategy, the despeckling results of SAR-DRN+SDSS with those of the SAR despeckling methods are compared: the PPP filter, the SAR-BM3D filter, the HDRANet and the MONet. The despeckling results on different SAR images are shown in Fig. 6 to Fig. 8. The quantitative evaluation results of ENL, TCR, MoR and EPD-ROA of the different methods are also listed in Table I.

#### Iv-D1 Qualitative Results

As revealed in Fig. 6 to Fig. 8, the Lee filter can remove speckle noise well, but the homogeneous region has obvious block effect, leading to the blurring of structures. SAR-BM3D performs better than the Lee filter as the image contrast is higher and the structure is clearer. However, slight artifacts can still be observed in the SAR-BM3D despeckling results. The two methods based on supervised deep learning show better despeckling performance compared with the previous traditional methods. They are good at smoothing images to remove speckle noise. But it is accompanied by distorting many of the image features. HDRANet, trained on synthetic SAR images, exists the problem of over-smoothing the heterogeneous area. MONet has better feature retention properties, but the problem of domain gap still exists. Our SDSS strategy, while looking slightly less smooth than the two supervised methods, excels at preserving features in the heterogeneous area and suppressing speckle noise in homogeneous area.

In summary, our SDSS strategy is able to despeckle without any noise-free reference labels. At the same time, the purpose of suppressing speckle noise in homogeneous area while preserving the texture and edge information in heterogeneous area is achieve. Our method achieves comparable or even superior performance to fully supervised methods.

#### Iv-D2 Quantitative Results

The ENL values listed in Table I illustrate that, the HDRANet and MONet have good performance of suppressing the speckle noise. However, due to domain gaps, over-smoothing occurs, which means that the ENL value alone cannot fully evaluate the despeckling performance. The TCR and MoR value are used to evaluate the performance of retaining the scattering information from strong point targets and radiometric preservation respectively. Our proposed strategy, SDSS, achieves the highest TCR and MoR scores. Furthermore, SDSS shows stronger capacity of retaining edges according to the EPD- ROA value. Instead of using optical images to simulate SAR images, our proposed SDSS realize despeckling by learning from real SAR images directly. Thus, the problem of domain gap is solved fundamentally. Specifically, our proposed SDSS showed good ability of suppressing speckle noise, preserving texture information, and preserving radiometric information, according to these evaluation metrics.

Fig. 8: Comparison results of our proposed SAR-DRN+SDSS on TerraSAR image against other methods: Lee, SAR-BM3D, HDRANet and MONet. (a) Despeckling results of different methods. (b) Details of the despeckling images.
From the perspective of visual results and assessment indices, the SDSS strategy has a positive effect on all images.

### _Ablation Study_

Here, ablation studies of our SDSS strategy are conducted for further study. Specially, we assess 1) the influence of the sampling strategy; 2) the performance of the parameter-sharing convolution network; 3) the importance of each term in the multi-feature loss function; 4) the influence of the weight in the multi-feature loss function.

_1) the influence of the sampling strategy_

The validity of our proposed adjacent-syntropy sub-sampler is illustrated by comparing in Section III-C to the random neighbor sub-sampler. The comparison results are illustrated in the Fig. 9.

\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{Figure} & \multirow{2}{*}{Index} & Random & Adjacent-syntropy \\  & & sub-samplers & sub-samplers \\ \hline \multirow{2}{*}{1} & ENL & 100.3635 & **313.3470** \\  & MoR & 0.8728 & **0.8847** \\ \hline \multirow{2}{*}{2} & ENL & 103.0519 & **134.7330** \\  & MoR & 0.9365 & **0.9701** \\ \hline \hline \end{tabular}
\end{table} TABLE II: Quantitative results of sampling strategy.

\begin{table}
\begin{tabular}{c|l|c|c|c|c} \hline \hline No. & Method & ENL & TCR & MoR & EPD-ROA \\ \hline \multirow{4}{*}{Image} & Lee & 43.8345 & 3.9190 & 0.8021 & 0.6972 \\ \cline{2-6}  & SAR-BM3D & 31.2785 & 4.0905 & 0.7952 & 0.7002 \\ \cline{2-6}  & HDRANet & **46.5970** & 5.0730 & 0.8120 & 0.6856 \\ \cline{2-6}  & MONet & 44.2231 & 4.3958 & 0.8260 & 0.6976 \\ \cline{2-6}  & SAR-DRN+SDSS(Proposed) & 40.6545 & **3.9136** & **0.9481** & **0.7053** \\ \hline \multirow{4}{*}{Image} & Lee & **5.6600** & 1.3706 & 0.7943 & 0.8441 \\ \cline{2-6}  & SAR-BM3D & 3.5305 & 1.1768 & 0.6864 & 0.8497 \\ \cline{2-6}  & HDRANet & 4.9029 & 1.5194 & 0.8477 & 0.8394 \\ \cline{2-6}  & MONet & 4.8265 & 1.7199 & 0.9253 & 0.8414 \\ \cline{2-6}  & SAR-DRN+SDSS(Proposed) & 4.2472 & **1.1378** & **0.9515** & **0.8572** \\ \hline \multirow{4}{*}{Image} & Lee & 47.9865 & 3.7501 & 0.8743 & 0.6937 \\ \cline{2-6}  & SAR-BM3D & 41.9027 & 6.7534 & 0.9894 & **0.6989** \\ \cline{1-1} \cline{2-6}  & HDRANet & **61.6822** & 2.5735 & 0.8524 & 0.6954 \\ \cline{1-1} \cline{2-6}  & MONet & 59.9685 & 1.9055 & 0.9692 & 0.6955 \\ \cline{1-1} \cline{2-6}  & SAR-DRN+SDSS(Proposed) & 51.1076 & **1.5773** & **0.9981** & 0.6974 \\ \hline \hline \end{tabular}
\end{table} TABLE I: Quantitative Results for different images

Fig. 9: Visual comparison results of sub-sampler: random neighbor sub-sampler and proposed adjacent-syntropy sub-sampler. (a) The top row is the despeckling results when using random neighbor sub-sampler. The second row is the details of the corresponding images. (b) The top row is the despeckling results when using proposed adjacent-syntropy sub-sampler. The second row is the details of the corresponding images.
The filtered result using the random neighbor sub-sampler tends to show the grid effect in homogeneous regions. It is can be inferred that the randomness sampling disturbs the structure of the image which leads to the inconsistency of noisy pairs. Our sampling strategy takes the texture into consideration to alleviate this phenomenon. Furthermore, the quantitative comparison is shown in Table II.

The comparison displays the vital role of syntropy in the proposed sampling strategy and the proposed adjacent-syntropy sub-sampler achieves better performance.

#### V-A2 the performance of the parameter-sharing convolution network

To validate that the parameter-sharing convolution network can better exploit the information of the images, it is compared with the original network. From Table III, it can be observed that the parameter-sharing network achieved higher scores of ENL, TCR, MoR and EPD-ROA. Though the original network has achieved good performance, the employment of parameter-sharing can take full advantage of the training information of generated paired images, thereby improving despeckling performance. It also can narrow the gap between paired images to improve the accuracy of training.

#### V-A3 the importance of each term in the multi-feature loss function

The three terms in the multi-feature loss function are used to remove noise, narrow gap between the noise pairs generated by the sub-sampler and preserve texture details respectively. Table IV lists the performance of SSDS strategy under different combination of above terms. From Table IV, the following observations can be achieved. First, when the despeckling term is removed, the ENL scores are low. It indicates that the despeckling results remain a mass of speckle noise. So, the despeckling term is vital for removing noise. When the coefficient of regularization term is set as 0, the corresponding despeckled image is over-smoothing and loses texture information. This term plays a role of narrowing gap between the training paired images Third, the removal of perceptual term makes the EPD-ROA value decreases. It strengthens the concentration on the edge and the structures to improve the quality of despeckled images.

#### V-A4 the influence of the weight in the multi-feature loss function

The hyper-parameter \(\alpha\) and \(\beta\)are introduced in Equation (13) to control the proportion of the regularization term and perceptual term in the multi-feature loss function. Table V lists the performance under different combination of \(\alpha\) and \(\beta\) values. As shown in the Table V, the weights act as a controller between smoothness and details. First, a larger \(\alpha\) leads to a better the details retention while the ENL of the despeckled image is reduced, which means poor despeckling effect. Second, the larger the \(\beta\), the better the details retention. However, increasing the ENL will lead to a decrease in the despeckling effect (Probably some of the noise was retained during edge retention) and will no longer have the effect of increasing the EPD-ROA up to a certain point.

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multicolumn{2}{c}{ENL} & \(\alpha=1\) & \(\alpha=2\) & \(\alpha=3\) & \(\alpha=4\) \\ /EPD-ROA & \(\alpha=1\) & \(\alpha=2\) & \(\alpha=3\) & \(\alpha=4\) \\ \hline \multirow{2}{*}{\(\beta=1\)} & 62.9232 & **63.4325** & 62.7235 & 62.4519 \\  & /0.8442 & **/0.8453** & /0.8471 & /0.8453 \\ \multirow{2}{*}{\(\beta=2\)} & 63.4296 & 62.9187 & 62.7030 & 62.4482 \\  & /0.8517 & /0.8471 & /0.8497 & /0.8453 \\ \hline \multirow{2}{*}{\(\beta=3\)} & 63.4226 & 62.9109 & 62.6928 & 62.4115 \\  & /0.8497 & /0.8492 & /0.8464 & /0.8475 \\ \multirow{2}{*}{\(\beta=4\)} & 63.4182 & 62.7830 & 62.3925 & 62.4193 \\  & /0.8464 & /0.8442 & /0.8441 & /0.8453 \\ \hline \hline \end{tabular}
\end{table} TABLE V: Ablation on different weights (\(\alpha,\beta\)) of the regularization term. ENL and EPD-ROA results are evaluated.

\begin{table}
\begin{tabular}{c c c c c c} \hline \hline Data & Method & ENL & TCR & MoR & EPD-ROA \\ Source & & 74.2482 & 0.8975 & 0.9634 & 0.6756 \\ \multirow{2}{*}{TerraSAR} & Parameter-sharing SAR-CNN & 75.3324 & 1.1373 & 0.9659 & 0.6923 \\  & SAR-DRN & 75.9832 & 0.9588 & 0.9832 & 0.6852 \\  & Parameter-sharing SAR-DRN & 76.3283 & 1.2423 & 0.9870 & 0.7071 \\ \multirow{2}{*}{Sentinel-1} & SAR-CNN & 86.2748 & 1.7394 & 0.9265 & 0.7235 \\  & Parameter-sharing SAR-CNN & 86.0372 & 1.9424 & 0.9304 & 0.7764 \\ \multirow{2}{*}{Sentinel-1} & SAR-DRN & 88.2374 & 2.1413 & 0.9558 & 0.7848 \\  & Parameter-sharing SAR-DRN & 88.4928 & 2.3820 & 0.9587 & 0.8149 \\ \hline \hline \end{tabular}
\end{table} TABLE III: Quantitative results of parameter-sharing convolution network.
## V Conclusion

In this work, a self-supervised SAR despeckling strategy (SDSS) is proposed. It is possible to train the existing despeckling network with the speckled SAR images only. In our SDSS, the parameter-sharing networks are trained using the generated noisy pairs by the adjacent-syntropy sub-sampler and the multi-features loss function. This strategy is able to be migrated to the existing supervised despeckling networks.

The first part of our SDSS strategy is the adjacent-syntropy sub-sampler, which overcomes the domain gap problem caused by training with optical images by generating noise pairs that satisfy the same underlying clean target. Second, the use of a parameter-sharing network fully exploits the information within the generated noisy pairs, resulting in improved despeckling performance. Finally, the multi-feature loss function is designed to remove noise as much as possible while preserving texture details, taking into account the spatial and statistical characteristics of the SAR image, and compensating for the gap between the noise pairs generated by the sub sampler.

Experiments performed on actual SAR images have demonstrated that our proposed self-supervised despeckling strategy removes noise significantly, with performance that equals or even surpasses that of several advanced despeckling algorithms.

In the future, we are going to propose a despeckling network with strong interpretability and good despeckling performance based on the self-supervised despeckling strategy combined with the characteristics of SAR.
