CVGG-Net: Ship Recognition for SAR Images Based on Complex-Valued
  Convolutional Neural Network
Ship target recognition is a vital task in synthetic aperture radar (SAR)
imaging applications. Although convolutional neural networks have been
successfully employed for SAR image target recognition, surpassing traditional
algorithms, most existing research concentrates on the amplitude domain and
neglects the essential phase information. Furthermore, several complex-valued
neural networks utilize average pooling to achieve full complex values,
resulting in suboptimal performance. To address these concerns, this paper
introduces a Complex-valued Convolutional Neural Network (CVGG-Net)
specifically designed for SAR image ship recognition. CVGG-Net effectively
leverages both the amplitude and phase information in complex-valued SAR data.
Additionally, this study examines the impact of various widely-used complex
activation functions on network performance and presents a novel complex
max-pooling method, called Complex Area Max-Pooling. Experimental results from
two measured SAR datasets demonstrate that the proposed algorithm outperforms
conventional real-valued convolutional neural networks. The proposed framework
is validated on several SAR datasets.
Dandan Zhao, Zhe Zhang, Dongdong Lu, Jian Kang, Xiaolan Qiu, Yirong Wu
# CVGG-Net: Ship Recognition for SAR Images Based on Complex-Valued Convolutional Neural Network

###### Abstract

Ship target recognition is a vital task in synthetic aperture radar (SAR) imaging applications. Although convolutional neural networks have been successfully employed for SAR image target recognition, surpassing traditional algorithms, most existing research concentrates on the amplitude domain and neglects the essential phase information. Furthermore, several complex-valued neural networks utilize average pooling to achieve full complex values, resulting in suboptimal performance. To address these concerns, this paper introduces a Complex-valued Convolutional Neural Network (CVGG-Net) specifically designed for SAR image ship recognition. CVGG-Net effectively leverages both the amplitude and phase information in complex-valued SAR data. Additionally, this study examines the impact of various widely-used complex activation functions on network performance and presents a novel complex max-pooling method, called Complex Area Max-Pooling. Experimental results from two measured SAR datasets demonstrate that the proposed algorithm outperforms conventional real-valued convolutional neural networks. The proposed framework is validated on several SAR datasets.

 Complex activation function, Complex Area Max-Pooling, Complex-valued convolutional neural network, Synthetic Aperture Radar(SAR), SAR target recognition.

## I Introduction

YNTHETIC aperture radar (SAR) is a high-resolution active microwave imaging sensor that operates without limitations related to weather or time [1], playing a crucial role in both military and civilian applications. SAR target recognition involves using variety of methods to identify the category of targets in SAR images, and it represents a popular research direction in microwave remote sensing applications[2].

Presently, SAR target recognition methods can be roughly categorized into traditional methods, deep learning-based methods, and complex information-based methods. Traditional SAR image target recognition methods rely on two key technologies: feature extraction and target recognition. SAR images possess unique geometric, mathematical, and electromagnetic features that significantly differ from optical images. Common feature extraction methods for SAR images include template matching [3] and feature fusion [4]. Once features are extracted, an appropriate classifier is selected to recognize and classify them. Popular classifiers currently include support vector machines [5-6], sparse representation classification[7], and others. However, most traditional recognition methods are adaptations of optical pattern recognition, characterized by complex feature designs and limited recognition rates.

With the advent of deep learning technology, frameworks such as Convolutional Neural Networks (CNNs) and Fully Convolutional Neural Networks (FCNs) have demonstrated promising performance in SAR image target recognition, increasingly becoming mainstream in the field. Chen et al. [8] and Ding et al. [9] employed CNNs for SAR image target recognition, achieving impressive results on the MSTAR dataset. Concurrently, Lin et al. [10] integrated channel convolution and attention mechanisms, effectively focusing on the target area within SAR images. Deep learning-based methods can automatically extract features and perform recognition tasks, eliminating the need for manual feature design or complex physical modeling. However, the majority of deep learning-based work emphasizes amplitude information, while phase information remains a crucial factor in SAR image target recognition [11]. SAR utilizes microwave coherent imaging, rendering SAR images complex-valued [12]. Compared to real values, complex values offer superior representation and generalization characteristics[13]. Consequently, there is an urgent need to develop SAR image target recognition methods that fully harness the advantages of complex-valued information.

In 2017, literature [14] first introduced the CV-CNN, which employs complex average pooling to achieve better results than real-valued networks on multi-channel POLSAR image classification tasks. Yu et al. [15] proposed the CV-FCNN, consisting only of convolutional layers and utilizing 1x1 complex convolution to learn cross-channel feature information, while Zhang et al. [16] presented the CV-MotionNet, a complex-valued convolutional neural network architecture that eliminates the need for motion compensation in classifying SAR moving ship targets. However, recognizing targets from SAR images can be challenging due to the effects of coherent speckle noise [17], which results in the discretization of target pixels and poor target separability. Although complex-valued SAR images contain rich phase information, relying solely on amplitude information makes it difficult to obtain satisfactory recognition results.

To address these issues, we propose a complex-valued convolutional neural network (CVGG-Net) for target recognition in SAR images that fully utilizes the amplitude and phase information in complex-valued SAR data.
Moreover, we introduce a novel complex max-pooling method, termed "Complex Area Max-Pooling", which aids the network in extracting more effective features. Compared to traditional real-valued CNNs, the proposed method achieves higher accuracy.

## II Proposed Target Recognition Method

In recent years, CNNs have made significant strides in computer vision tasks. The CVGG-Net, proposed in this study, is based on the architecture of VGG[18] and incorporates the complex-valued convolution operation from [19]. Specifically designed for recognizing target objects in SAR images, the network accepts single-channel input. As illustrated in Fig. 1, the proposed CVGG-Net comprises 13 complex-valued convolutional blocks, 5 complex area max-pooling layers, 3 complex fully connected layers, 1 amplitude evaluation layer, and 1 softmax layer.

CVGG-Net is designed with a VGG-like architecture, closely resembling VGG16. All layers within the network are complex-valued, allowing complex convolutional layers to extract information present in the amplitude and phase of complex SAR images. This approach enables the extraction of richer features compared to traditional deep learning methods. Complex area max-pooling is utilized to reduce the number of network parameters and enhance overall robustness. The complex fully connected layers further extract target features, which are then converted to real values in the last layer for calculating cross-entropy loss with labels. Finally, the network leverages a softmax layer for target recognition.

### _Complex-valued convolutional blocks_

A complex-valued convolutional block consists of a complex convolutional layer, a complex batch normalization layer, and a complex activation function layer, as shown in Fig. 2.

#### Ii-A1Complex Convolutional Layer

Complex convolution is an extension of traditional convolution in computer vision that operates in the complex domain. Unlike traditional convolution, which only utilizes amplitude information, complex convolution extracts target features by using both amplitude and phase information present in complex-valued SAR images. This approach results in complex-valued operations being added to the traditional convolution process. Experimental results have demonstrated that complex convolutional layers outperform conventional convolutional layers, indicating their superiority in extracting meaningful target features.

According to [19], when the convolution operation is extended to the complex field, the complex vector \(\mathrm{I=x+yj}\) and the complex convolution kernel \(\mathrm{W=A+B}\) perform corresponding element-wise multiplication and summation operations, where \(\mathrm{j=\sqrt{-1}}\). Separating the real and imaginary values in the complex feature layer, the complex-valued convolution is equivalent to:

\[\begin{array}{l}W*I=(A+B)*(x+yj)\\ =(A*x-B*y)+(A*y+B*x)j\,.\end{array} \tag{1}\]

In the right hand side of the above equation, * stands for traditional (real) convolution. \(\mathrm{x}\) and \(\mathrm{y}\) represent the real and imaginary parts of the complex-valued vector respectively, \(\mathrm{A}\) and \(\mathrm{B}\) represent the real and imaginary parts of the complex convolution kernel, respectively. It can be seen that one complex-valued convolution operation is equivalent to four conventional convolution operations, as shown in Fig. 3.

#### Ii-A2Complex Batch Normalization Layer

In deep neural networks, batch normalization is often employed to stabilize the intermediate output values of each layer, promote model convergence, and mitigate the risk of overfitting [20].

Batch normalization of complex values can be scaled by the square root of the variance of the two principal components, real and imaginary, through dividing the zero-centered data \((X-E(X))\) by the square root of the \(2\times 2\) covariance matrix \(V\)[19].

\[\widetilde{\chi}=\frac{X-E(X)}{\sqrt{V}} \tag{4}\]

\[V=\begin{pmatrix}cov(\Re\{x\},\Re\{x\})&cov(\Re\{x\},\Im\{x\})\\ cov(\Im\{x\},\Re\{x\})&cov(\Im\{x\},\Im\{x\})\end{pmatrix} \tag{5}\]

#### Ii-A3Complex Activation Function Layer

In order to handle complex-valued representations, a family of complex activation functions have been proposed. CRelu introduced in [21], extending the traditional Relu activation function to the complex domain.

\[\mathrm{CRelu(z)=Relu(\Re\{z\}+jRelu(\Im\{z\})} \tag{6}\]

CRelu applies separate Relu activations to both the real and imaginary parts of neurons, satisfying the Cauchy Riemann equation when both parts are either positive or negative.

Existing literature lacks a definitive consensus on the most suitable activation function for complex-valued neural networks. The primary requirement for the function is to be nonlinear,

Fig. 1: Architecture of the CVGG-Net.

Fig. 2: Schematic diagram of complex-valued convolutional blocks.
with gradients that do not suffer from issues such as explosion or vanishing during training [22]. In this paper, we have evaluated several common complex-valued activation functions from the perspective of ease of implementation. After extensive experimentation, we have selected CReLU as the optimal activation function.

### _Complex Area Max-Pooling_

As shown in Fig. 4(a), many researchers split complex-valued data into imaginary and real parts, and perform real value-based complex max-pooling on the real and imaginary parts, respectively. However, this approach is unreasonable and unsuitable for complex-valued SAR data. Fig. 4(b) illustrates the amplitude-based complex max-pooling (CAMaxPool) proposed in [23], which preserves the complex values corresponding to the coordinates with the maximum amplitude. Building on CAMaxPool, this paper further proposes complex area-based max-pooling, which selects the coordinates corresponding to the maximum area, as shown in Fig. 4(c).

These two complex-valued max-pooling methods can be viewed as a choice between the values of f1 and f2. As shown in Fig. 5, f1 represents the length of the vector OA, and f2 is the area of the triangle AOB.

\[\mathrm{f_{1}(z)=\sqrt{x^{2}+y^{2}}} \tag{7}\] \[\mathrm{f_{2}(z)=|xy|} \tag{8}\]

The proposed complex area max-pooling introduces an alternative option for selecting pooling coordinates. In essence, the proposed complex area max-pooling tends to retain elements where both real and imaginary parts are present. Although in the SAR context, we cannot provide an appropriate physical explanation for this pooling strategy, our experiments have demonstrated that area-based max-pooling outperforms canonical amplitude-based complex max-pooling.

## III Experimental Results

### _A.Experimental data and platform_

Two SAR datasets are utilized to validate the proposed framework. One is CSRSDD (Complex SAR images Rotation Ship Detection dataset) annotated in [24]. The other is OpenSARship, a public dataset released by Shanghai JiaoTong University in 2017 [25].

#### Iii-1)CSRSDD

All data were collected in GF-3 Spotlight (SL) mode with 1m resolution and HH or HV polarization mode. Each image size was 1024x1024 pixels, containing more abundant features. The rotating box is used to mark the target, and the annotation format refers to the DOTA dataset format [26]. The data is primarily concentrated in the port area, with the offshore scene accounting for more than 80%, featuring a complex background and significant interference. To suit the recognition task, we cut the targets out according to the annotation files. The amplitude slices are saved as.iff files, and the complex slices as.mat files, with the largest dimensions being 610x944 pixels and the smallest 12x34 pixels, as shown in Fig. 6. Five categories are screened out, as presented in Table I.

Fig. 4: Three kinds of complex max-pooling. (a) Real value-based complex max-pooling. (b) Amplitude-based complex max-pooling. (c) Area-based complex max-pooling.

Fig. 5: Diagram of coordinates.

Fig. 3: Schematic diagram of complex-valued convolution structure.
#### Iv-C1 OpenSARship

The dataset was collected from Sentinel-I images and contains ground range multi-look products and slant range single-look complex products with both VV and VH polarization. In this paper, three types of ship targets--Bulk carrier, Cargo, and Tanker--are selected for research. Ship categories with too few samples are discarded, and each type of ship is randomly divided into training and testing sets according to Table II.

In order to facilitate network processing, all image slices in both CSRSDD and OpenSARship are normalized, filled or cropped to 224x224 pixels.

All experiments in this paper were conducted on Pytorch 1.8.1 and a device with Ubuntu18.04 system. Hardware features include Quadro RTX 8000 GPU (40GB memory), CPU AMD 3950x and 64G RAM. Due to the limitation of GPU memory, we set the batch size to be 32, the initial learning rate as 0.0001, and adopt Adam optimizer for optimization, train it for 100 epochs in total.

### _Comparison of Proposed network and Real-valued Networks_

As comparison for benchmark, a small complex-valued network CVnet5 with five complex-valued convolution blocks is built in this paper. Its structure is shown in Fig. 7.

To verify the effectiveness of the proposed method, we compare it with commonly used real-valued networks such as ResNet18, VGG16, and Net5, which is the real-valued network corresponding to CVnet5. Each experiment is performed 10 times, and the average value is taken as the final experimental result. The results on the CSRSDD dataset and OpenSARship dataset are shown in Table III.
77.17% and 78.86% recognition rates, respectively. With our area-based max-pooling, the network performance is improved by 0.75% and 0.71%, respectively. On the OpenSARShip dataset, the network performance is improved by 0.3% and 0.87%, respectively, with area-based max-pooling. These results indicate that the proposed area-based max-pooling is more effective than amplitude-based max-pooling for complex-valued networks.

### Comparison with other complex-valued networks

To verify the effectiveness of the proposed method, we compared the recognition results with other complex-valued convolutional networks, including CVnet5 and the complex-valued network CV-Net proposed in [17]. As shown in Table VII, the performance of the proposed method is superior to other methods on both the CSRSDD and OpenSARShip datasets. Even when CV-Net is combined with our complex area max-pooling, the recognition rates are increased by 0.07% and 0.23%. This improvement can be attributed to the fact that the CVGG-Net, based on complex-valued convolution blocks and complex area max-pooling, is better suited for SAR image target recognition tasks.

## IV Conclusion

Traditional target recognition methods for SAR images often neglect the phase information, which is crucial for recognition accuracy. Considering that SAR images are inherently complex-valued, we propose a complex-valued convolutional neural network method named CVGG-Net for target recognition in SAR images. Additionally, we introduce a novel complex max-pooling method based on area, which outperforms other complex max-pooling methods. Our experimental results, obtained from CSRSDD and OpenSARShip datasets, demonstrate the effectiveness of our proposed method, including the complex area max-pooling and CVGG-Net algorithms. In the future, we plan to extend complex-valued convolutional neural networks to other fields, such as SAR object detection and semantic segmentation.
