# CATSNet: a context-aware network for Height Estimation in a Forested Area
based on Pol-TomoSAR data

Wenyu Yang, Sergio Vitale, Hossein Aghababaei, Giampaolo Ferraioli, Vito
Pascazio, and Gilda Schirinzi,
This paper has been submitted to IEEE TGRS. At the moment is under review
(first round). Wenyu Yang is with Dipartimento di Ingegneria, Università degli
Studi di Napoli “Parthenope,” 80143 Naples, Italy. (email:
<EMAIL_ADDRESS>Sergio Vitale, Vito Pascazio and
Gilda Schirinzi are with Dipartimento di Ingegneria, Università degli Studi di
Napoli “Parthenope,” 80143 Naples, Italy and also with the National Inter-
University Consortium for Telecommunications — CNIT, 80126 Naples, Italy.
(email<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Giampaolo Ferraioli is with Dipartimento di
Scienze e Tecnologie, Università degli Studi di Napoli “Parthenope,” 80143
Naples, Italy and also with the National Inter-University Consortium for
Telecommunications — CNIT, 80126 Naples, Italy. (email:
<EMAIL_ADDRESS>Hossein Aghababaei is with the
Department of Earth Observation Science, Faculty of Geo-Information Science
and Earth Observation, University of Twente, Enschede 7514AE, The Netherlands.
(e-mail: h.aghababaei@utwente.nl).

###### Abstract

Tropical forests are a key component of the global carbon cycle. With plans
for upcoming space-borne missions like BIOMASS to monitor forestry, several
airborne missions, including TropiSAR and AfriSAR campaigns, have been
successfully launched and experimented. Typical Synthetic Aperture Radar
Tomography (TomoSAR) methods involve complex models with low accuracy and high
computation costs. In recent years, deep learning methods have also gained
attention in the TomoSAR framework, showing interesting performance. Recently,
a solution based on a fully connected Tomographic Neural Network (TSNN) has
demonstrated its effectiveness in accurately estimating forest and ground
heights by exploiting the pixel-wise elements of the covariance matrix derived
from TomoSAR data. This work instead goes beyond the pixel-wise approach to
define a context-aware deep learning-based solution named CATSNet. A
convolutional neural network is considered to leverage patch-based information
and extract features from a neighborhood rather than focus on a single pixel.
The training is conducted by considering TomoSAR data as the input and Light
Detection and Ranging (LiDAR) values as the ground truth. The experimental
results show striking advantages in both performance and generalization
ability by leveraging context information within Multiple Baselines (MB)
TomoSAR data across different polarimetric modalities, surpassing existing
techniques.

###### Index Terms:

Synthetic Aperture Radar (SAR), Polarimetry, Tomography, Forest Height,
Underlying Topography, Deep Learning (DL), U-Net.

##  I Introduction

Although tropical forests only cover about $6\%$ of Earth’s surface, they
provide a wide range of ecosystem services [1][2]. They harbor a
disproportionately high fraction of global terrestrial biodiversity [3][4],
stock about a quarter of total terrestrial carbon, and contribute up to a
third of net primary production to play an important role in the global carbon
cycle [5].

In particular, the forest structure is a significant indicator of productivity
and biomass level, so it becomes a major element for its measurement [6][7].
There are several ways to monitor forests. In-situ measurement provides the
most accurate parameters but represents a costly and time-consuming solution.
Remote sensing data instead offers a wider perspective thanks to the large-
scale, consistent, and short revisiting acquisition intervals. Among remote
sensing tools, LiDAR provides the most accurate vertical resolution, making it
the optimal solution for monitoring forest structure. Its effectiveness is
however constrained by the high cost of airborne-based acquisition campaigns.
SAR Tomography (TomoSAR) equipped with multiple baselines (MB) acquisitions is
the other remote sensing technique that provides 3D imaging with vertical
resolution capability. With the under-foliage penetration capability of radar
signals, TomoSAR is powerfully suited to 3D measurements of forests.
Furthermore, with benefits from polarimetric information, TomoSAR has been
effectively utilized to differentiate scattering mechanisms in the vertical
direction of forestry, enabling the separation of ground and above-ground
contributions in SAR signal [8][9]. To reconstruct the forest profiles from
TomoSAR data, several models have been proposed ranging from Fourier-based
methods to super-resolution solutions. Among them, MUSIC (Multiple Signal
Classification) [10] is confined to be applied to sparse data. Traditional
Fourier-based algorithms such as beamforming and Capon [11] suffer from severe
sidelobes with worse resolution resulting from uneven and limited baseline
sampling [12]. Compressive sensing (CS) [13] [14] methods tend to generate
high-resolution results, but multiple iterations are highly computation-
loaded. Additionally, decomposition-based methods have been studied for the
identification and separation of different scattering mechanisms (SM)
including double bounce and volumetric scattering [15][16] with the help of
Pol-TomoSAR. In [17], the covariance matrix is expressed as a Sum of Kronecker
Products which provides the basis for performing the SM separation by model-
based, model-free, and hybrid approaches. Methods based on Generalized
Likelihood Ratio Test (GLRT) [16] are characterized by the advantages of
canopy and ground separation based on Random-Volume-over-Ground (RVOG) with
TomoSAR data.

In the past few years, deep learning (DL) solutions have garnered significant
interest in the field of TomoSAR applications. A preliminary DL approach [18]
has been proposed for urban structure height measurement with TomoSAR data.
Building heights are reconstructed by formulating the problem as a
classification task. The experiment on simulated data primarily shows its
potential. Moving to the forest scenario, an unsupervised DL-based approach
named PolGAN [19] treats the forest height estimation task as a pan-sharpening
problem to realize the forest height estimation with high accuracy and fine
spatial resolution. The main drawback of PolGAN is the requirement for LiDAR
data also in the testing stage, which, even in low resolution, is not always
available. Recently, [20] proposed a lightweight neural network for improving
the vertical resolution profile generated by beamforming. Indeed, Berenger et
al. train the network without the need of LiDAR data but using simulated
reference height profiles. In [21] the TSNN (Tomographic SAR Neural Network)
algorithm for height estimation in forested areas has been proposed. TSNN is a
pixel-wise solution that formulates height estimation as a classification task
showing remarkable performance on TropiSAR campaign using both full, dual and
single polarized data [22]. Being pixel-wise, TSNN however suffers from the
lack of neighboring pixels’ information, resulting in noisy height maps that
could negatively impact the measurement of forests’ structure.

To go beyond such limitations, this paper proposes a flexible and context-
aware DL solution for forest height reconstruction, which is named CATSNet
(Context-Aware TomoSAR Network). Instead of leveraging single-pixel
acquisitions, a patch-based solution is depicted for taking advantage of
neighborhood information. Leveraging the availability of LiDAR as vertical
reference data, a CNN-based solution has been trained for retrieving forest
and ground height profiles from Pol-TomoSAR data. In particular, the input
patches are derived from the covariance matrix elements of the corresponding
patches of the Pol-TomoSAR data stack. The annotations of the input patches
are the quantized LiDAR-based heights (LiDAR-based CHM and LiDAR-based DTM).
The aim is to extract features related to heights not only from the covariance
matrix elements of the corresponding pixel but also from the neighboring
pixels, which means instead of pixel-wise, the proposed method is conducted at
a patch-wise level. The reason is twofold: the information in the surrounding
target pixels can help in the final estimation since the ground and canopy
height profiles typically exhibit a high spatial correlation, and leveraging
contextual information is crucial for limiting the speckle effect and making a
more robust and flexible solution easily adaptable to different areas. As a
CNN-based solution, CATSNet’s architecture is borrowed from U-Net [23] for its
good use of context while maintaining good localization with few images.
Moreover, special attention has been paid to the generalization ability of the
proposed solution in terms of flexibility in new areas.

The paper is organized as follows: in Section II, the methodology is indicated
on network architecture and training parameters; Study area and dataset
construction are described in Section III. The description of experimental
areas and the analysis of the testing results are presented in Section IV.

##  II Methodology

In this paper, the height estimation problem is formulated as a segmentation
task: an auto-encoder convolutional network is trained for classifying each
pixel of a Pol-TomoSAR stack with the corresponding LiDAR reference value. In
Section II-A, details about the dataset’s construction and the method’s
workflow are provided. The CATSNet architecture is described in Section II-B.
Finally, the training parameters are presented in II-C.

###  II-A Data and Workflow

Figure 1: Workflow of the proposed method. For each range-azimuth pixel of the
MPMB Tomostack, the $\Phi\times N$ real diagonal elements of the matrix $R$
and the real parts and the imaginary parts of elements in the first are
stacked and considered as the channels of the input patch. The annotations
consist of the corresponding quantized LiDAR-based heights: LiDAR-based CHM in
the case of training targeted for prediction of forest height; and LiDAR-based
DTM in the case of training targeted for prediction of ground height. Figure
2: Architecture of CATSNet. The architecture is composed of a contracting
encoder path (left side) and an expansive decoder path (right side). The
encoder path consists of five downsampling levels and the decoder consists of
five upsampling levels.

TomoSAR methods for forest profile estimation are mostly based on the data
covariance matrix. Also in this paper, we rely on the input patches obtained
from the covariance matrix. We assume that TomoSAR acquires $N$ multi-baseline
SLC images for each polarization channel. After several pre-processing
operations applied to the SLC data stack, such as coregistration, phase
flattening, and calibration, we get the covariance matrix of each range-
azimuth pixel of the SLC data stack. Let’s now consider the fully polarimetric
(FP) case.ì, so that the number of polarization channels is $\Phi=3$. In this
case, the Pol-TomoSAR MBMP (Multiple Baselines Multiple Polarizations) data
stack $\boldsymbol{y}$ is defined as

$\begin{split}&\boldsymbol{y}=[{y_{1}}^{HH},\quad{y_{2}}^{HH},\quad\cdots\quad{{y_{N}}^{HH}},\\\
&\quad\quad{y_{1}}^{HV},\quad{y_{2}}^{HV},\quad\cdots\quad{{y_{N}}^{HV}},\\\
&\quad\quad{y_{1}}^{VV},\quad{y_{2}}^{VV},\quad\cdots\quad{{y_{N}}^{VV}}]^{T}\end{split}$
(1)

where $y_{i}^{PQ}$ denotes the pixel value of the SAR image acquired with the
$i$-th baseline using $P$ as the transmitting polarization channel and $Q$ as
the receiving polarization channel, where $P,Q=\\{H,V\\}$, and $T$ is the
transpose operator. Theoretically, the computation of the covariance matrix
$\mathbf{R}$ is operated on different realizations of the data vector
$\mathbf{y}$. However, different realizations of the random data are not
available, thereby the statistical expectation is performed by means of a
spatial averaging operation on SAR images. In the considered FP model, the
size of covariance matrix $\mathbf{R}$ is $3N\times 3N$ and is given by:

$\begin{split}\mathbf{R}=E\\{\boldsymbol{y}\boldsymbol{y}&{}^{\dagger}\\}=\left[\begin{matrix}{\mathbf{C}_{HHHH}}&{\mathbf{\Omega}_{HHHV}}&{\mathbf{\Omega}_{HHVV}}\\\
{\mathbf{\Omega}_{HVHH}}&{\mathbf{C}_{HVHV}}&{\mathbf{\Omega}_{HVVV}}\\\
{\mathbf{\Omega}_{VVHH}}&{\mathbf{\Omega}_{VVHV}}&{\mathbf{C}_{VVVV}}\\\
\end{matrix}\right]\end{split}$ (2)

where the superscript $\dagger$ denotes the Hermitian transpose and
$E\\{\cdot\\}$ is the statistical expectation operator. It can be noted that
in the equation (2), $\mathbf{R}$ consists of $3\times 3$ sub-matrices, each
of size $N\times N$. The diagonal sub-matrices ${\mathbf{C}}$ are contributed
by interferometric information [8], whereas the off-diagonal sub-matrices
${\mathbf{\Omega}}$ are cross-correlation matrices containing both
polarimetric and interferometric information. For the dual polarimetric (DP)
model, the number of polarization channels is $\Phi=2$. For the single
polarimetric (SP) model, $\Phi=1$ and the covariance matrix $\mathbf{R}$
reduces to one of the diagonal sub-matrices ${\mathbf{C}}$.

We construct our input patches with a size of $W\times W\times M$, where $W$
represents the number of pixels in the range and azimuth directions of a local
patch. The parameter $M$ denotes the dimensionality of tomographic data
information in the covariance matrix. Considering $\Phi\cdot N$ real elements
of the diagonal and $\Phi\cdot N-1$ complex elements of the first row of the
covariance matrix $\mathbf{R}$, the value of $M$ is given by $M=3\Phi\cdot
N-2$. This is because the complex elements are split into their real and
imaginary parts.

In the proposed method, quantized LiDAR-based heights are used for the
annotation. Before the quantization operation, a spatial averaging operation
is applied to LiDAR-based data to get a comparable spatial resolution to the
SAR data. The size of the adopted averaging window is the same as the one used
in the SAR covariance matrix computation. Then, a quantization with a $1$ m
step is conducted on the filtered LiDAR-based heights, generating annotations
$\mathbf{s}\in[s_{1},\quad s_{2},\quad\cdots,\quad s_{K}]$ where $K$ is the
number of the height classes. The label patch size ($W\times W$) is set
accordingly with the input patch one. In this context, we choose $W=64$ as
indicated in the workflow depicted in Fig. 1.

###  II-B Neural Network architecture

The architecture considered in this paper is U-Net architecture, as
illustrated in Fig. 2. It consists of a contracting encoder path (left side)
and an expansive decoder path (right side). The number of the input image tile
channels is $M$. The contracting part is composed of five downsampling levels:
each level is composed of two $3\times 3$ convolutional layers, followed by a
rectified linear unit (ReLU) and a $2\times 2$ max pooling operation with
stride 2 for downsampling (except for the last level). After each downsampling
step, the number of feature channels is doubled. On the contrary, the
expansive part is composed of five upsampling levels. As for the encoder part,
each level consists of two $3\times 3$ convolutional layers followed by ReLU
and a $2\times 2$ upsampling operation. On the contrary, in the encoder part,
at each level, the number of feature maps is halved and concatenated with the
corresponding feature map from the contracting path. At the final layer, a
$1\times 1$ convolution is used to map each 32-component feature vector to the
desired number of classes, which is the number of the forest height or ground
height classes. As we can see from Fig. 2, there are $23$ convolutional
layers. The reason behind this choice lies in two main motivations: first, in
this work, the height estimation problem is referred to as a segmentation
task, and CATSNet is a well-assessed solution for such a task; second, it
easily allows patch-based and multi-scale feature extraction thanks to its
auto-encoder nature.

###  II-C Network Training

A multiclass cross-entropy function [24] is used for the loss function. In
this case, $\mathbf{p}$ is the reference distribution containing a unitary
value at the position corresponding to the $i$-th class embedding the LiDAR
quantized value, and $\mathbf{q}$ is the estimated a-posteriori probability
distribution for each class obtained by applying the softmax function to the
output $\mathbf{\hat{H}}$. So the estimated distribution can be written as
$\mathbf{q}=[e^{h_{j}}/\sum_{j}e^{h_{j}}\textit{ }|\textit{
}j=1,\cdots,\zeta]$ and, $\mathbf{p}=[0,\cdots,1,\cdots,0]$ with unitary
values at $i$-th position. Thus, the cross-entropy loss is computed as [21]:

$L_{CE}=-log\left(\frac{e^{h_{i}}}{\sum_{j}e^{h_{j}}}\right)$ (3)

CATSNet is implemented with the Pytorch framework and optimized by minimizing
(3) through the SGD optimizer [25] with a momentum of $0.9$. We start training
on GeForce GTX 1080Ti GPU with 12 GB of memory. The batch size is set as 64
and the learning rate is set as $0.01$ and the factor of learning rate decay
is 0.5 with 200 epochs as the period. The Xavier initialization [26] has been
conducted for the weights of each fully connected layer.

##  III Study Area

In this paper, two study areas have been considered: the Paracou site acquired
during the TropiSAR campaign carried out over forests of French Guiana by
ONERA in August 2009 [27] and the Lopè site acquired during the AfriSAR
campaign carried out over the African tropical forests of Gabon by DLR in
February 2016 [28], respectively. At the same time, small-footprint LiDAR was
also collected for both datasets.

The Paracou site (5.27°N, 52.93°W) is located around 75 km West of Cayenne as
shown in Fig.3 (left) (https://paracou.cirad.fr). The altitude ranges between
5 and 45 m asl over undulating terrain. The vegetation height ranges between 0
and 60 m, and the biomass level ranges between 300 t/ha and 485 t/ha. System
parameters can be found in Table I.

The Lopè site (0.20°S, 11.59°E) is located near the geographical center of
Gabon, ca. 250 km east of Libreville, as shown in Fig.4 (right). The
topography of this site is generally hilly, with an altitude ranging from 200
to 600 m asl [29]. Vegetation over the Lopè site is a forest-savanna mosaic
with different forest types co-occurring in the area where the forest height
is distributed between 30 and 50 m with biomass ranging from about 50 to 600
t/ha [30]. System parameters can be found in Table I.

Figure 3: The geographic location of Paracou site (left) and the image coverage (right). |
---|---
Figure 4: The geographic location of Lopè site (left) and the image coverage (right). TABLE I: TropiSAR and AfriSAR Acquisition parameters Parameters | TropiSAR | AfriSAR
---|---|---
Wavelength | 0.7542 m | 0.69 m
Flight height | 3962 m | 6096 m
Incidence angle | 35.061° | 25-35°
Range resolution | 1 m | 3.9 m
Azimuth resolution | 1.245 m | 2.0 m
Polarization | Full-Pol | Full-Pol
Baselines [m] | $\begin{aligned} 0\\\ -14.4879\\\ -30.1163\\\ -43.7343\\\ -60.0632\\\ -74.9683\end{aligned}$ | $\begin{aligned} 20\\\ 0\\\ -20\\\ -40\\\ -60\\\ -80\end{aligned}$

Aiming to construct a training dataset containing the forest and ground
heights, we make use of the SAR tomography stack of the acquisition and
relative LiDAR data. In particular, for each of the selected study areas, the
tomographic stack is composed of six temporal SAR baselines, whose
descriptions are in Table I. The stack of covariance matrices estimated from
the polarimetric tomographic SAR stack is used as the input for the network,
while the reference LiDAR values are used as reference height profiles.

Paracou | Lopè
---|---
|
Figure 5: Pauli RGB images of testing patch extracted by Paracou (left) and
Lopè areas (right).

In particular, the full-polarization Tomo-stack consists of $18$ SLC data
contributed from HH, HV, and VV polarization channels, so here $N=6$ and
$\Phi=3$. In consequence:

  * •

For fully polarization ($\Phi=3$), the size of the covariance matrix
$\mathbf{R}$ size is $18\times 18$, so the channels of the input image tile
$M$ is $52$.

  * •

For dual polarization ($\Phi=2$), the size of the covariance matrix
$\mathbf{R}$ size is $12\times 12$, so the channels of the input image tile
$M$ is $34$.

  * •

For single polarization ($\Phi=1$), the size of the covariance matrix
$\mathbf{R}$ size is $6\times 6$, so the channels of the input image tile $M$
is $16$.

As reference datasets, LiDAR-based data are available on both sites.

The French Agricultural Research Center for International Development and the
Guyafor project provide the LiDAR-based data of the Paracou site. The LiDAR-
based DTM was produced by triangular interpolation (TIN) of the ground data.
By deducting ground elevation from the raw point cloud and extracting maximum
height with a $1$ m resolution grid, the LiDAR-based CHM was generated. The
forest height and ground height of a geographic position associated with a SAR
image position were obtained by projecting the LiDAR-based data under WGS84
UTM zone 22 into SAR coordinates using an ASCII file supplied by TROPISAR
[31]. More details about LiDAR-based data processing are given in [32]

Regarding the Lopè site, the LiDAR-based data was obtained by NASA’s full-
waveform LiDAR LVIS (Land, Vegetation, and Ice Sensor) system. LVIS was
mounted on the NASA Langley B200 aircraft and flown at 23000ft and the LiDAR
footprint on the ground is around 20m wide. The data access is
https://lvis.gsfc.nasa.gov/Data/Maps/Gabon2016Map.html including Level1B and
Level2 products. In our experiment, the ground topography (or Digital Terrian
Model, DTM) and the canopy top height (or Canopy Height Model, CHM) in Level2
are used as the reference.

From each of Paracou and Lopè study areas, a testing patch with size
$512\times 512$ pixels has been extracted from the whole dataset and shown in
Fig. 5, respectively. The rest of data is randomly split into the training
data set (80%) and the validating data set (20%).

| LiDAR | CATSNet | TSNN | SKP
---|---|---|---|---
CHM |  |  |  |
|  |  |
---
DTM |  |  |  |
|  |  |
---
DSM |  |  |  |
|  |  |
---
Figure 6: Full-Polarization results on Paracou. The comparison within the
reference LiDAR-based values and heights estimated by CATSNet, TSNN and SKP.
The first row is for forest heights, the second row is for ground heights, and
the last row is for canopy heights.

##  IV Experimental Results

In this section, the height estimation ability of the proposed method is
presented. Comparison with three representative Tomographic methods, such as
SKP, GLRT, and deep learning-based TSNN, is carried out on the Paracou area
with FP, DP, and SP TropiSAR data stack. It is worth mentioning that SKP
exploits the separation of different scattering mechanisms based on MPMB
TomoSAR data and GLRT is characterized by the advantages of scattering
mechanisms separation with single polarization TomoSAR data. Thereby, for FP
and DP models, SKP is employed and for the SP model, GLRT is applied.
Moreover, the proposed solution is compared with its more timing efficient
version able to predict the forest and ground height in a single shot without
the need for a separate training procedure for the CHM and DTM height
estimation.

###  IV-A Fully polarization

For the sake of visualization, we present a 3D representation of a real forest
scenario by comparing the reference LiDAR-based CHM, DTM, and DSM (CHM+DTM)
with the estimated ones by CATSNet, TSNN, and SKP in Fig. 6, respectively.

For the forest, TSNN presents generally overestimated forest heights,
including several outliers, and for SKP, there is a serious underestimation
problem. Among them, CATSNet performs the best. For the ground, CATSNet still
presents the best performance. In this case TSNN also gives a good agreement
with the reference, but the reconstruction is more noisy. On the contrary, SKP
tends to generate overestimated ground heights, which is a common problem in
traditional TomoSAR methods for ground height measurement. The overall results
can be observed from the third row of Fig. 6, showing the canopy heights
(forest heights + ground heights), and CATSNet exhibits the best agreement
with LiDAR-based DSM.

The previous considerations are confirmed by joint distribution between
predicted and expected results of Figure 7, where the ideal result is
represented by the black line. The left column shows the result comparison on
forest heights, and the right column is about the ground heights.

As we can see the joint distribution of CATSNet is densely concentrated on the
black line and TSNN is sparsely focused. However, the result of SKP for CHM
presents a severe forest height underestimation problem. So it leads to the
conclusion that CATSNet illustrates a better agreement with the LiDAR-based
CHM than TSNN and SKP. The predicted ground height of testing samples
perfectly located on the black line means the ideal estimation. The joint
distribution of CATSNet presents the best performance compared with TSNN and
SKP. In terms of TSNN, it yields noisy predicted ground heights as the result
of its element-wise characteristics. Regarding SKP, the overestimated problem
is further supported by that the majority of the predicted ground heights are
located above the black line. In short, we can conclude that, with a full
polarization TomoSAR data stack, CATSNet outperforms the forest and ground
heights estimation problem.

| CHM | DTM
---|---|---
CATSNet |  |
TSNN |  |
SKP |  |
Figure 7: Full polarization results over Paracou area. Joint distribution
within the reference LiDAR-based values and heights estimated by CATSNet,
TSNN, and SKP. The left column is for forest heights and the right column is
for ground height. The black line represents the ideal prediction.

###  IV-B Dual and Single Polarizations

To show the robustness of the proposal, the ability to estimate forest and
ground heights based on dual and single polarization TomoSAR data stack is
carried out in this section.

| CATSNet | TSNN | SKP
---|---|---|---
HHHV |  |  |
HHVV |  |  |
HVVV |  |  |
Figure 8: In the Paracou area, the joint distribution between LiDAR-based CHM
and forest heights by CATSNet, TSNN, and SKP which are based on dual-
polarization channels. The black line represents the ideal prediction.

To this aim, the SAR tomographic stack has been limited in the number of
polarization channels and CATSNet has been retrained accordingly, as well as
TSNN. CATSNet, TSNN, and SKP have been tested on the selected testing patch
considering two polarimetric tomographic stacks, while GLRT has been
considered for single polarization comparison. For the sake of reading, only
joint distribution results on the forest reconstruction (CHM) are reported,
but similar consideration can be done for ground reconstruction (DTM). The
joint distribution shown in Fig. 8 for the dual polarimetric case shows the
robustness of CATSNet with respect to any dual polarimetric channel
combination. As matter of fact, quality of prediction is almost stable in all
cases HHHV, HHVV, HVVV. Similar considerations are obtained in the case of
single polarization shown in Fig. 9, where the comparison has been carried out
with TSNN and GLRT.

To reinforce the conclusion we draw in this section, quantitative results,
Root-Mean-Square-Error (RMSE), of the forest height measurement are shown in
Table II and RMSE of the ground height measurement are shown in Table III.

In terms of the forest height, the RMSE of CATSNet is around 2.0 m giving the
best height accuracy, while the one of TSNN is around 3.0 m and SKP gives the
worst result in the full polarization model. For dual and single polarization
models, the RMSE of CATSNet and TSNN don’t present worse results compared to
the results of full polarization. It elucidates the robustness of CATSNet and
TSNN that they would maintain good performance even with less polarization
information. Even in the single polarization model, the forest height
measurement accuracy of CATSNet is still around 2.0 m which strongly supports
CATSNet’s advantage.

Regarding the measurement of the ground height, CATSNet in full polarization
presents the best performance with RMSE, 1.1365 m, which is around 0.4 m
better than TSNN and around 5.3 m better than SKP. When moving on to dual
polarization models, CATSNet and TSNN both go worse, and SKP even performs
well in the HHVV polarization channel. For single polarization, the RMSE of
the ground heights by CATSNet are all better than 2.0 m which is around 1.0 m
better than TSNN. GLRT shows the worst results.

TABLE II: RMSE of the forest height measurement on Paracou Campaign | Experiments | CATSNet | TSNN | SKP | GLRT
---|---|---|---|---|---
TropiSAR | Full-pol | 2.0220 | 3.0144 | 6.8195 | $-$
TropiSAR | $\begin{aligned} HHHV\\\ HHVV\\\ HVVV\end{aligned}$ | $\begin{aligned} 2.1601\\\ 1.9906\\\ 2.1235\end{aligned}$ | $\begin{aligned} 2.7294\\\ 2.9383\\\ 2.6388\end{aligned}$ | $\begin{aligned} 8.6684\\\ 7.0381\\\ 12.0206\\\ \end{aligned}$ | $\begin{aligned} -\\\ -\\\ -\end{aligned}$
TropiSAR | $\begin{aligned} HH\\\ HV\\\ VV\end{aligned}$ | $\begin{aligned} 2.0928\\\ 1.9040\\\ 2.0112\end{aligned}$ | $\begin{aligned} 3.2188\\\ 2.7131\\\ 2.7752\end{aligned}$ | $\begin{aligned} -\\\ -\\\ -\end{aligned}$ | $\begin{aligned} 9.5828\\\ 11.2247\\\ 11.6350\end{aligned}$
TABLE III: RMSE of the ground height measurement on Paracou Campaign | Experiments | CATSNet | TSNN | SKP | GLRT
---|---|---|---|---|---
TropiSAR | Full-pol | 1.1365 | 1.7169 | 6.4001 | $-$
TropiSAR | $\begin{aligned} HHHV\\\ HHVV\\\ HVVV\end{aligned}$ | $\begin{aligned} 1.4533\\\ 1.2335\\\ 1.5966\end{aligned}$ | $\begin{aligned} 2.2125\\\ 1.7623\\\ 2.3779\end{aligned}$ | $\begin{aligned} 7.5480\\\ 3.2627\\\ 15.0028\end{aligned}$ | $\begin{aligned} -\\\ -\\\ -\end{aligned}$
TropiSAR | $\begin{aligned} HH\\\ HV\\\ VV\end{aligned}$ | $\begin{aligned} 1.4987\\\ 1.8983\\\ 1.8840\end{aligned}$ | $\begin{aligned} 2.8751\\\ 3.1609\\\ 3.1779\end{aligned}$ | $\begin{aligned} -\\\ -\\\ -\end{aligned}$ | $\begin{aligned} 4.1808\\\ 9.6536\\\ 6.2612\end{aligned}$
| CATSNet | TSNN | GLRT
---|---|---|---
HH |  |  |
HV |  |  |
VV |  |  |
Figure 9: In the Paracou area, the joint distribution between LiDAR-based CHM
and forest heights by CATSNet, TSNN, and GLRT which are based on single
polarization channels. The black line represents the ideal prediction.

###  IV-C Unified CATSNet

The experiment results shown above are realized by implementing the CATSNet
solution on forest heights and ground heights individually. Specifically,
CATSNet trained with the LiDAR forest heights as the reference is used for the
forest height estimation, while CATSNet trained with the LiDAR ground heights
as the reference is used for the ground estimation. Such an approach is based
on a double training conduct and a double testing phase. With this in mind, we
train the CATSNet with both forest height and ground height at the same time
and get the trained Unified CATSNet that can be used to predict forest and
ground heights in one conduction to reduce computation load and improve time
efficiency. We refer to this solution as unified CATSNet.

The structure of the Unified CATSNet is the same as the one described in
section II-B just adapting the final layer to the expected number of classes.

| CHM | DTM
---|---|---
LiDAR |  |
CATSNet |  |
Unified CATSNet |  |
|  |
Figure 10: Full-Polarization results on Paracou. Reconstruction comparison within the reference LiDAR-based values and heights estimated by CATSNet and Unified CATSNet. The left column is for forest heights and the right column is for ground heights. | CHM | DTM
---|---|---
CATSNet |  |
Unified CATSNet |  |
Figure 11: Full-Polarization results on Paracou. Joint distribution within the
reference LiDAR-based values and heights estimated by CATSNet and Unified
CATSNet. The left column is for forest heights and the right column is for
ground heights. The black line represents the ideal prediction.

Experiments on Unified CATSNet are also conducted over the Paracou area in the
full polarization model. The 3D reconstruction results are shown in Fig. 10
where the left column shows the comparison of CATSNet and Unified CATSNet with
LiDAR values as the reference on the forest height estimation. For the forest,
it is hard to visually observe the performance difference and decide on a
better one. Quantitatively, Unified CATSNet (RMSE = 1.9156 m) performs a
little better than CATSNet (RMSE = 2.0220 m). Similarly, the ground height
estimation results are shown in the right column of Fig. 10, CATSNet and
Unified CATSNet both have excellent agreement with the LiDAR-based DTM.
Regarding RMSE, CATSNet is equipped with 1.1365 m which is shown in Table III
and the RMSE of unified CATSNet is 1.3618 m. Combining the joint distribution
shown in Fig. 11, all results demonstrate that Unified CATSNet is reliable for
both forest and ground heights estimation in one conduct. Additionally,
compared with CATSNet, we can interestingly notice that unified CATSNet shows
a little better performance for forest height estimation and a little worse
performance for ground height estimation. This suggests that in case of
limited computation resources, unified CATSNet can be a good alternative to
the separated ones.

Moreover, it has to be noted that in the training process, we didn’t put any
balanced weight on the loss functions for the forest height prediction or the
ground height prediction, so CATSNet automatically chose a compromised
performance balance between the forest height and ground height prediction. To
this end, users can choose the CATSNet model and the loss function based on
their needs in terms of performance and computation loads.

##  V Generalization on new areas: fine-tuning strategy

In the previous section, CATSNet has proven adept at measuring forest and
ground heights with TropiSAR data on FP, DP, and SP models. To evaluate the
robustness of CATSNet to areas with diverse characteristics and different
acquisition parameters such as baselines, wavelength, flight height, etc.,
this section provides the experimental results of CATSNet on another area,
Lopè, with AfriSAR data. Considering the outcomes of Section IV-C suggest the
Unified CATSNet being a sub-optimal solution, we prefer to focus on CATSNet
for such evaluation. In particular, the generalization ability of CATSNet is
tested following two approaches: training from scratch and fine-tuning. The
former consists of directly training from scratch the proposed solution on the
new dataset; instead, the latter considers the CATSNet pre-trained on Paracou
data and fine-tuning it on the new dataset.

###  V-A Training from the scratch

In particular, CATSNet and TSNN have been retrained on the Lopè dataset and
tested on the extracted testing patch. The results of the forest and ground
reconstruction are shown in Fig. 12 (the first and the third column).

LiDAR-CHM | LiDAR-DTM
---|---
|
|  |  |
CATSNet | CATSNet + Fine-tuning | CATSNet | CATSNet + Fine-tuning
|  |  |
TSNN | TSNN + Fine-tuning | TSNN | TSNN + Fine-tuning
|
---
|
---
Figure 12: Full-Polarization results on Lopè. The comparison within the reference LiDAR-based values and heights estimated by CATSNet, CATSNet + Fine-tuning, TSNN, and TSNN + Fine-tuning. The left column is for forest heights and the right column is for ground heights. CHM | DTM
---|---
|  |  |
CATSNet | CATSNet + Fine-tuning | CATSNet | CATSNet + Fine-tuning
|  |  |
TSNN | TSNN + Fine-tuning | TSNN | TSNN + Fine-tuning
Figure 13: Joint distribution between predicted and estimated results on Lopè:
CHM in the left column and DTM on the right one. The black line represents the
ideal prediction.

Although the robustness of CATSNet is evident in both reconstructions, at a
more detailed analysis some difficulties arise also for the proposed solution.
Considering the joint distribution shown in Figure 13 (first and the third
column), both forest and ground suffer from many misleading predictions. The
joint distribution of CHM shows many outliers, while DTM joint distributions
are more concentrated but far from the qualitative results achieved on Paracou
data.

These results show how different datasets may lead to different quality
outcomes. There could be several causes: different numbers of training
examples, nonidentical vertical resolutions, diverse coherence among stack
acquisition, and so on.

###  V-B Fine-tuning

To improve the performance on different datasets, a fine-tuning step may be
performed on a robust pre-trained solution. In particular, taking advantage of
the good results on Paracou, instead of training the proposed solution from
scratch, its pre-trained model on Paracou has been used as a starting point
and fine-tuned on the Lopè data. In comparison, the same procedure has been
followed for TSNN. The obtained results are shown in Figures 12-13 in the
second and fourth columns. With a better starting point (Paracou pre-
training), the flexibility and adaptability of CATSNet are still more evident.
The profile reconstruction provided by CATSNet is getting closer to the
reference following smoother variability for the CHM and reducing the
underestimation in DTM. Instead, TSNN seems not robust from this perspective,
since its results seem to not have any gain from the fine-tuning strategy. The
distribution of predicted points lay over the black line much better thanks to
the fine-tuning procedure, while the same can not be seen for TSNN.

Let’s examine quantitative results, RMSE. For the forest height estimation,
the RMSE of CATSNet is around 0.2 m worse than TSNN. With the operation of
fine-tuning, CATSNet’s performance has been boosted by around 1.3 m. On the
contrary, TSNN with fine-tuning doesn’t show any advantage over RMSE. In the
same vein, for the ground height measurement, CATSNet has better performance
with 0.4 m higher accuracy than TSNN. Importantly, the accuracy of ground
height measurement is further improved by CATSNet with fine-tuning. TSNN shows
similar performance on the ground height estimation; the RMSE is around 2.5 m
with and without fine-tuning.

As we can see, this is a fundamental aspect when designing a deep learning
solution. Deep learning solutions are powerful but, as data-driven approaches,
need high-quality datasets. In this case, the ability of CATSNet to overcome
the issue related to Lopè training with a simple fine-tuning step indicates
that a patch-based approach taking into account neighborhood information
allows an easy extension and robustness to several user testing cases.

TABLE IV: RMSE of the forest height measurement on Lopè Campaign | Experiments | CATSNet | TSNN
---|---|---|---
AfriSAR | Full-pol | 4.0221 | 3.8389
AfriSAR | Fine-tuning | 2.7754 | 3.9101
TABLE V: RMSE of the ground height measurement on Lopè Campaign | Experiments | CATSNet | TSNN
---|---|---|---
AfriSAR | Full-pol | 2.1912 | 2.5736
AfriSAR | Fine-tuning | 2.0980 | 2.5979

##  VI Conclusion

In this study, we propose to apply the U-Net model for height estimation based
on the TomoSAR data set and name this method as CATSNet. CATSNet formulates
the inversion of the forest and ground heights as the classification task: the
elements of the covariance matrix of MB TomoSAR data patches act as the input,
and the LiDAR-based heights are used for the reference. It needs to be
underlined that CATSNet is conducted at the patch level enables making use of
the context and achieves good localization accuracy.

Firstly, CATSNet has been trained and tested in the Paracou area with FP, DP,
and SP TropiSAR data. In the comparison with another deep learning-based
method, TSNN, and two typical TomoSAR techniques, SKP and GLRT, CATSNet shows
the capability to generate precise forest and ground height as well as
maintain good spatial resolution, while the pixel-wise TSNN suffers from noisy
prediction. Traditional TomoSAR methods are confined to complex models, have
high computation costs, and have worse performance. More importantly, not only
with FP TomoSAR data but also with DP and SP TomoSAR data, CATSNet shows
compelling performance for height construction compared to the other methods.
To this end, CATSNet is a valid candidate for height measurement in the
situation of limited Pol-TomoSAR data.

In addition, experiments in the Lopè area with AfriSAR data reinforce the
robustness of CATSNet, which works well with data sets acquired with different
parameters. CATSNet fine-tuning provides us with an alternative solution to
apply deep learning-based CATSNet with fewer data sets, broadening the
application scenarios. Lastly, unified CATSNet is also recommended for height
reconstruction with a lower computation load and easier experimental
operation.

In conclusion, we deem that CATSNet is a versatile patch-wise approach with
good performance for diverse application situations and needs.

## Acknowledgments

The authors would like to thank the Europe Space Agency (ESA) for providing
the TropiSAR and AfriSAR P-band TomoSAR data sets and thank the
NASA/ESA/DLR/AGEOS Afrisar Campaign for providing the LiDAR data. Lastly,
thank Chuanjun Wu for his kind help with LiDAR data.

## References

  * [1] Fabrizio Lombardini and Andreas Reigber,  “Forest and woodland systems,”  in Ecosystems and Human Well-Being: Current State and Trends. IEEE, 2003, vol. 3, pp. 2014–2016.
  * [2] Nicolas Labrière, Bruno Locatelli, Yves Laumonier, Vincent Freycon, and Martial Bernoux,  “Soil erosion in the humid tropics: A systematic quantitative review,”  Agriculture, Ecosystems & Environment, vol. 203, pp. 127–139, 2015\.
  * [3] Simon P.,  “Tropical forests and the changing earth system,”  Philosophical Transactions of The Royal Society B Biological Sciences, vol. 1465, no. 361, pp. 195–210, 2005.
  * [4] Victor Arroyo-R. Ferry Slik and etc.,  “An estimate of the number of tropical tree species,”  Proceedings of the National Academy of Sciences, vol. 112, no. 24, pp. 7472–7477, 2015.
  * [5] Richard T Corlett,  “The impacts of droughts in tropical forests,”  Trends in plant science, vol. 21, no. 7, pp. 584–593, 2016.
  * [6] Brady S Hardiman, Gil Bohrer, Christopher M Gough, Christoph S Vogel, and Peter S Curtis,  “The role of canopy structural complexity in wood net primary production of a maturing northern deciduous forest,”  Ecology, vol. 92, no. 9, pp. 1818–1827, 2011.
  * [7] Hiroaki T Ishii, Shin-ichi Tanabe, and Tsutom Hiura,  “Exploring the relationships among canopy structure, stand productivity, and biodiversity of temperate forest ecosystems,”  Forest Science, vol. 50, no. 3, pp. 342–355, 2004.
  * [8] H. Aghababaee and M. R. Sahebi,  “Model-based target scattering decomposition of polarimetric sar tomography,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 56, pp. 972–983, 2017.
  * [9] M. Pardini and K. Papathanassiou,  “On the estimation of ground and volume polarimetric covariances in forest scenarios with sar tomography.,”  IEEE Geoscience and Remote Sensing Letters, vol. 14, pp. 1860–1864, 2017.
  * [10] F. Lombardini, F. Cai, and D. Pasculli,  “Three dimensional focusing with multipass sar data,”  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 6, no. 2, pp. 960–968, April 2013.
  * [11] Petre Stoica, Randolph L Moses, et al.,  Spectral analysis of signals, vol. 452,  Pearson Prentice Hall Upper Saddle River, NJ, 2005.
  * [12] S. Tebaldini,  “Single and multipolarimetric sar tomography of forested areas: A parametric approach,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 5, pp. 2375–2387, May 2010.
  * [13] A. Budillon, A. Evangelista, and G. Schirinzi,  “Three-dimensional sar focusing from multipass signals using compressive sampling,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 1, pp. 488–499, January 2011.
  * [14] Esteban Aguilera, Matteo Nannini, and Andreas Reigber,  “Wavelet-based compressed sensing for sar tomography of forested areas,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 12, pp. 5283–5295, 2013.
  * [15] S. Tebaldini and F. Rocca,  “Multibaseline polarimetric sar tomography of a boreal forest at p- and l-bands,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 50, pp. 232–246, 2012.
  * [16] H. Aghababaee, A. Budillon, G. Ferraioli, V. Pascazio, and G. Schirinzi,  “On the separation of ground and canopy scatterings using single polarimetric multi-baseline sar tomography,”  Proceedings of IEEE International Geoscience and Remote Sensing Symposium, vol. 2019, pp. 975–978, 2019.
  * [17] S. Tebaldini,  “Algebraic synthesis of forest scenarios from multibaseline polinsar data,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 47, pp. 4132–4142, 2009.
  * [18] A. Budillon, A. C. Johnsy, G. Schirinzi, and S. Vitale,  “Sar tomography based on deep learning,”  Proceedings of IEEE International Geoscience and Remote Sensing Symposium, pp. 3625–3628, 2019.
  * [19] Qi Zhang, Linlin Ge, Scott Hensley, Graciela Isabel Metternicht, Chang Liu, and Ruiheng Zhang,  “Polgan: A deep-learning-based unsupervised forest height estimation based on the synergy of polinsar and lidar data,”  ISPRS Journal of Photogrammetry and Remote Sensing, vol. 186, pp. 924–2716, 2022.
  * [20] Zoé Berenger, Loïc Denis, Florence Tupin, Laurent Ferro-Famil, and Yue Huang,  “A deep-learning approach for sar tomographic imaging of forested areas,”  IEEE Geoscience and Remote Sensing Letters, vol. 20, pp. 1–5, 2023\.
  * [21] Wenyu Yang, Sergio Vitale, Hossein Aghababaei, Giampaolo Ferraioli, Vito Pascazio, and Gilda Schirinzi,  “A deep learning solution for height estimation on a forested area based on pol-tomosar data,”  IEEE Transactions on Geoscience and Remote Sensing, pp. 1–1, 2023\.
  * [22] Wenyu Yang, Sergio Vitale, Hossein Aghababaei, Giampaolo Ferraioli, Vito Pascazio, and Gilda Schirinzi,  “A deep learning solution for height inversion on forested areas using single and dual polarimetric tomosar,”  IEEE Geoscience and Remote Sensing Letters, vol. 20, pp. 1–5, 2023\.
  * [23] Olaf Ronneberger, Philipp Fischer, and Thomas Brox,  “U-net: Convolutional networks for biomedical image segmentation,”  in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 2015, pp. 234–241.
  * [24] Kevin P Murphy,  Machine learning: a probabilistic perspective,  MIT press, 2012.
  * [25] Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell,  “Caffe: Convolutional architecture for fast feature embedding,”  in Proceedings of the 22nd ACM international conference on Multimedia, 2014, pp. 675–678.
  * [26] X. Glorot and Y.. Bengio,  “Understanding the difficulty of training deep feedforward neural networks,”  Proceedings of Machine Learning Researc, vol. 9, pp. 249–256, 2010\.
  * [27] Pascale C. Dubois-Fernandez, Thuy Le Toan, Sandrine Daniel, Hélène Oriot, Jerôme Chave, Lilian Blanc, Ludovic Villard, Malcolm W. J. Davidson, and Michel Petit,  “The tropisar airborne campaign in french guiana: Objectives, description, and observed temporal behavior of the backscatter signal,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 8, pp. 3228–3241, 2012.
  * [28] Irena Hajnsek, Matteo Pardini, Ralf Horn, Rolf Scheiber, Marc Jaeger, Martin Keller, Daniel Gesswein, Kostas Papathanassiou, and Andreas Reigber,  “3-d sar imaging of african forests: Results from the afrisar campaign at p- and l-band,”  in Proceedings of EUSAR 2016: 11th European Conference on Synthetic Aperture Radar, 2016, pp. 1–4.
  * [29] Nicolas Labrière, Shengli Tao, Jérôme Chave, Klaus Scipal, Thuy Le Toan, Katharine Abernethy, Alfonso Alonso, Nicolas Barbier, Pulchérie Bissiengou, Tânia Casal, Stuart J. Davies, Antonio Ferraz, Bruno Hérault, Gaëlle Jaouen, Kathryn J. Jeffery, David Kenfack, Lisa Korte, Simon L. Lewis, Yadvinder Malhi, Hervé R. Memiaghe, John R. Poulsen, Maxime Réjou-Méchain, Ludovic Villard, Grégoire Vincent, Lee J. T. White, and Sassan Saatchi,  “In situ reference datasets from the tropisar and afrisar campaigns in support of upcoming spaceborne biomass missions,”  IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, no. 10, pp. 3617–3627, 2018.
  * [30] Aida Cuni-Sanchez, Lee JT White, Kim Calders, Kathryn J Jeffery, Katharine Abernethy, Andrew Burt, Mathias Disney, Martin Gilpin, Jose L Gomez-Dans, and Simon L Lewis,  “African savanna-forest boundary dynamics: a 20-year study,”  PLoS One, vol. 11, no. 6, pp. e0156934, 2016.
  * [31] Z. Liao, B. He, X. Bai, et al.,  “Improving forest height retrieval by reducing the ambiguity of volume-only coherence using multi-baseline polinsar data,”  IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 11, pp. 8853–8866, 2019.
  * [32] G. Vincent al.,  “Accuracy of small footprint airborne lidar in its predictions of tropical moist forest stand structure,”  Remote Sensing of Environment, vol. 125, pp. 23–33, October 2012\.