# Scale Aware Adaptation for Land-Cover Classification
in Remote Sensing Imagery

Xueqing Deng  EECS, UC Merced {xdeng7, ytian8<EMAIL_ADDRESS>Yi Zhu
Work was done prior to joining Amazon. Amazon Web Services<EMAIL_ADDRESS>Yuxin Tian  EECS, UC Merced {xdeng7, ytian8<EMAIL_ADDRESS>Shawn
Newsam  EECS, UC Merced {xdeng7, ytian8<EMAIL_ADDRESS>
###### Abstract

Land-cover classification using remote sensing imagery is an important Earth
observation task. Recently, land cover classification has benefited from the
development of fully connected neural networks for semantic segmentation. The
benchmark datasets available for training deep segmentation models in remote
sensing imagery tend to be small, however, often consisting of only a handful
of images from a single location with a single scale. This limits the models’
ability to generalize to other datasets. Domain adaptation has been proposed
to improve the models’ generalization but we find these approaches are not
effective for dealing with the scale variation commonly found between remote
sensing image collections. We therefore propose a scale aware adversarial
learning framework to perform joint cross-location and cross-scale land-cover
classification. The framework has a dual discriminator architecture with a
standard feature discriminator as well as a novel scale discriminator. We also
introduce a scale attention module which produces scale-enhanced features.
Experimental results show that the proposed framework outperforms state-of-
the-art domain adaptation methods by a large margin. The open-sourced codes
are available on Github: https://github.com/xdeng7/scale-aware_da.

##  1 Introduction

High-resolution remote sensing imagery is becoming increasingly available due
to the number and revisit rate of traditional satellite and airborne image
capture platforms as well as the advent of newer platforms like drones. This
imagery provides convenient and large-scale coverage and so is being applied
to a number of societally important problems such as land cover segmentation
[29], traffic monitoring [34], urban planning [5], vehicle detection [11],
building extraction [51], geolocolization[42] etc. While remote sensing (RS)
image analysis has benefited from advances in deep learning in the computer
vision community, there often remains unique challenges that limit the
straightforward application of standard approaches to the RS case.

Figure 1: Object sizes vary both within and between ground-level image
datasets such as in the driving scenes at the top. Segmentation models trained
on one dataset are already scale-invariant and so standard domain adaptation
techniques are not designed to explicitly account for scale variation. In
contrast, object sizes usually do not vary within RS image datasets since all
the images have the same ground sample distance (GSD). We propose a novel
framework that explicitly accounts for scale when adapting between RS image
datasets with different scales such as at the bottom.

Another issue that limits the performance of automated RS image analysis,
particularly deep learning approaches, is that the availability of the
annotated ground truth data needed for training has not kept pace with the
imagery (or its diversity). As an example, current semantic segmentation
datasets for land cover classification, which are very labor intensive to
produce, contain limited labeled samples from only a few locations. The ISPRS
Vaihingen dataset [1] contains just $33$ labeled tiles with 6 semantic
classes. The recent SkyScapes [4] dataset has more classes with 30 but still
contains only $16$ images. DeepGlobe [15] is the largest collection,
containing $1146$ images with 7 classes. Deep learning models trained on these
(relatively) small datasets have difficulty generalizing to other image sets,
i.e. large-scale WorldView imagery. They suffer from the so-called domain
shift problem [12, 43]. One of the great benefits of using RS imagery is its
fast revisit time and large coverage for Earth observation. However, this
scalability is limited by domain shift problems.

Domain shift in RS imagery can happen along a number of dimensions including
differences in sensor, spectra, resolution, etc. which have been widely
explored [7, 47, 8]. There has been much less work, somewhat surprisingly, on
the cross-location domain shift problem in which a model trained on one
location that has ground truth data, the source location, is applied to
another location without ground truth data, the target location. The work that
has been done simply applies standard domain adaptation techniques [7, 50,
16]. However, none of this work explicitly considers the important notion of
scale which we believe is often integral to the cross-location problem
especially for tasks where there is limited training data like semantic
segmentation in RS imagery. We therefore propose a novel scale adaptive
framework for improved cross-location domain adaptation. The framework is
adversarial and includes separate feature and scale discriminators as well as
a scale attention module.

Domain shift caused by location also exists in other datasets such as road
scene imagery. However, we found that scale is more important for overhead
(RS) than (most) ground-level imagery. A simple visual investigation
illustrates why scale is more important. Fig. 1 contains a pair of images from
different driving scene datasets and a pair of images from different RS image
datasets. (The ground truth segmentation masks are overlaid on the images.)
The objects in the road scenes vary in size both within a dataset (even within
an image) as well as between datasets. A deep learning model trained on one
dataset learns scale-invariant features so is better able to generalize to the
other dataset. However, in the RS imagery, due to the planar nature of the
scene and the nadir viewpoint, the size of the objects does not vary (much)
within a dataset if all the images have the same resolution or ground sample
distance (the physical size of a pixel on the ground typically indicated in
meters or similar) which is usually the case. The RS imagery in the source
domain in Fig. 1 has a GSD of 9cm so all the cars measure around $17\times 22$
pixels assuming $1.5\times 2$ meter cars. The GSD of the target domain is just
5cm so all the cars are proportionally larger and measure around $30\times 40$
pixels. A model trained on one dataset will not learn the scale-invariant
features needed to label the differently sized cars in the other dataset. Data
augmentation during training through random scaling is not an effective
solution. Additional examples of showing that scale is more important for RS
than regular imagery can be found in the supplementary material.

We therefore develop a novel domain adaptation framework that explicitly
accounts for scale changes in order to improve generalization in cross-
location semantic segmentation for RS imagery. The proposed framework contains
dual adversarial discriminators including a standard feature discriminator and
a novel scale discriminator as well as a scale attention module. There are
separate discriminators for feature and scale adaptation between the source
and target domains. The scale attention module selectively weights
concatenated multi-scale features to obtain scale-enhanced features. Our novel
contributions include:

  * •

We establish and demonstrate that explicitly accounting for scale variation is
integral to RS image domain adaptation yet no existing work does this. We
propose an innovative scale adaptive framework for cross-location semantic
segmentation which includes a novel scale discriminator and a scale attention
module to facilitate training. We demonstrate that scale-aware adaptation
results in significant performance gains.

  * •

We show our proposed approach outperforms state-of-the-art domain adaptation
methods on several cross-dataset segmentation tasks such as Potsdam
$\leftrightarrow$ Vaihingen and DeepGlobe $\leftrightarrow$ Vaihingen.

##  2 Related Work

Semantic segmentation There is a large body of literature on semantic
segmentation[9, 10, 20, 56]. Here, we review only the most related work in
terms of handling scale [54, 19, 31, 26]. Fully Convolutional Network (FCN)
based methods [33] have made great progress in semantic segmentation. To
capture multi-scale information, DeepLab networks [9], PSPNet [54] and CaseNet
[26] develop mechanisms to integrate multi-scale features. Benefiting from the
above exploration on regular images, a number of works [32, 37, 3, 55, 18, 17]
have applied the techniques to pixel-wise land-cover classification in RS
imagery. These methods focus on multi-scale analysis but not explicitly on
scale adaptation as proposed in this paper.

Domain adaptation Numerous domain adaptation methods have been developed to
improve image classification by aligning the feature distributions between the
source and the target domains [45, 22, 39, 2]. Domain adaptation for
segmentation has recently started to receive more attention due to the expense
of performing pixel-wise annotation. Many approaches have been proposed for
road scene segmentation [43, 13, 23, 24, 44, 48, 27, 58, 14]. Since scale is
much less significant for road scene images, these studies focus mainly on
adapting texture, appearance, etc. There has been some work on using
Generative Adversarial Networks (GAN) for domain adaptation in RS image
segmentation [6, 50]. However, these approaches just apply standard computer
vision methods without considering challenges specific to RS imagery. We
instead propose a framework that addresses the importance of scale when
adapting between domains.

Attention Attention was originally proposed to exploit long-range dependencies
in machine translation [46]. It has since been adopted in a number of computer
vision tasks [53, 49, 52, 57]. Self-attention has been used as a non-local
operation to learn positional relations in video classification [49], to learn
a better image generator [52], as well as to learn both channel and spatial
attention for scene segmentation [21] and land cover segmentation [38]. In
order to augment the extraction of multi-scale information, we propose a scale
attention module (channel attention) which improves the scale adaptation.

##  3 Problem Formulation

We formulate our problem as cross-scale and cross-location domain adaptation
for semantic segmentation in RS imagery (pixel-wise land-cover
classification). We assume the domain shift is caused by different scales and
locations between the source and target datasets. We recognize, though, that
different locations do not necessarily have different scales. Our framework is
unified in that it can deal with domain shift caused by scale or by location
or by both.

We denote the different locations as $\mathcal{S}$ and $\mathcal{T}$ and the
different scales as $\theta$ and $\sigma$. We denote an image ${x}$ from
source location $\mathcal{S}$ with scale $\theta$ as
${x}_{\mathcal{S}^{\theta}}\in{X\\{location=\mathcal{S},scale=\theta\\}}$ and
its label as ${y}_{\mathcal{S}^{\theta}}\in$
${{{Y\\{location=\mathcal{S},scale=\theta\\}}}}$. $X$ and $Y$ represent all
the images and labels in one domain. Our goal is to adapt, in an unsupervised
manner, a segmentation model $\mathcal{G}$ trained using images
${X}_{\mathcal{S}^{\theta}}$ and labels ${Y}_{\mathcal{S}^{\theta}}$ from
source domain $\mathcal{S}^{\theta}$ to perform segmentation in target domain
$\mathcal{T}^{\sigma}$ which has only images ${X}_{\mathcal{T}^{\sigma}}$ to
produce predicted segmentation masks $\hat{{Y}_{\mathcal{T}^{\sigma}}}$.

##  4 Methods

In this section, we describe our scale aware domain adaptation framework. We
first revisit conventional domain adaptation methods which only have feature
discriminators. We then describe our new scale discriminator for addressing
the scale variation between domains. Finally, we describe our scale attention
module for obtaining scale-enhanced features for improved segmentation.

Figure 2: Our proposed scale aware adaptation framework which contains two
adversarial learning components, one for feature adaptation and another for
scale adaptation. We use DeepLabV3+ as our segmentation network. ASPP is an
Atrous Spatial Pyramid Pooling module. We also propose a scale attention
module (SAM) to produce scale-enhanced features. See the text for more
details.

###  4.1 Domain Adaptation Revisited

Recent domain adaptation methods for semantic segmentation are adversarially
based [43, 23]. An image from either the source or target domain is input to
the segmentation network. The resulting feature maps or segmentation
predictions are then fed to a discriminator which tries to determine the
domain of the input. The goal of the segmentation network now is to not only
produce an accurate segmentation of the source domain image (for which we have
labels) but to also fool the discriminator. This forces the internal
representations of the model to align between domains so it can better exploit
its supervised training in the source domain when it is applied to the target
domain.

The segmentation network is now updated using two losses, the segmentation
loss when the input is from the source domain and the adversarial loss when
the input is from the target domain. Given any segmentation network
$\mathcal{G}$ (e.g., [54, 10]), source image ${x}_{\mathcal{S}^{\theta}}$ will
result in prediction
${p}_{\mathcal{S}^{\theta}}=\mathcal{G}({x}_{\mathcal{S}^{\theta}})$ and
target image ${x}_{\mathcal{T}^{\sigma}}$ will result in prediction
${p}_{\mathcal{T}^{\sigma}}=\mathcal{G}({x}_{\mathcal{T}^{\sigma}})$. Note
that we choose to use the low-dimensional softmax output predictions instead
of the feature maps following [43] since they contain rich spatial information
shared between locations which makes it easier for the network to adapt. When
the input is from the source domain, the multi-class cross-entropy
segmentation loss

$\mathcal{L}_{seg}({x}_{\mathcal{S}^{\theta}})=-{y}_{\mathcal{S}^{\theta}}\log({p}_{\mathcal{S}^{\theta}}),$
(1)

is computed where ${y}_{\mathcal{S}^{\theta}}$ is the annotated label. A
segmentation loss cannot be computed for a target image since its label is not
available. So, in order to adapt the segmentation network to the target
domain, a feature discriminator $\mathcal{D}_{feat}$ is added and an
adversarial loss is calculated

$\mathcal{L}_{adv\\_feat}({x}_{\mathcal{T}^{\sigma}})=-\log{(\mathcal{D}_{feat}({p}_{\mathcal{T}^{\sigma}})}).$
(2)

This is a binary cross-entropy loss, designed to fool the discriminator by
forcing the space of target predictions ${p}_{\mathcal{T}^{\sigma}}$ to match
the space of source predictions ${p}_{\mathcal{S}^{\theta}}$. At this point,
the overall objective for updating the segmentation network is a combination
of $\mathcal{L}_{seg}$ and $\mathcal{L}_{adv\\_feat}$

$\mathcal{L}({x}_{\mathcal{S}^{\theta}},{x}_{\mathcal{T}^{\sigma}})=\mathcal{L}_{seg}+\lambda_{f}\mathcal{L}_{adv\\_feat},$
(3)

where $\lambda_{f}$ is the weight of the adversarial loss.

The feature discriminator $\mathcal{D}_{feat}$ is updated using its own
adversarial loss

$\begin{split}\mathcal{L}_{\mathcal{D}_{feat}}({p})=-{(1-z)}\log(\mathcal{D}_{feat}(p))+z\log(\mathcal{D}_{feat}(p))\end{split}$
(4)

where $z=0$ if the output prediction map $p$ is from the target location
$\mathcal{T}$, and $z=1$ if it is from the source location $\mathcal{S}$. The
segmentation network and the feature discriminator are optimized in an
alternating manner. When one is being updated, the other is frozen.

###  4.2 Scale Discriminator

The standard domain adaption framework above achieves decent results for
cross-location segmentation when the source and target domain have similar
scale, i.e., ${x}_{\mathcal{S}^{\theta}}$ and ${x}_{\mathcal{T}^{\theta}}$.
However, it does not do well when the scale varies, i.e.,
${x}_{\mathcal{S}^{\theta}}$ and ${x}_{\mathcal{T}^{\sigma}}$. When the scale
of target dataset is different from the source, the performance of a model
trained with just a feature discriminator decreases by $20\%$ in the RS case
(details in the supplementary material).

We therefore propose a dual discriminator network which includes the standard
feature discriminator as well as a new scale discriminator. We split the
adaptation task into two sub-tasks, one that focuses on (cross-location)
feature adaptation and another that focuses on scale adaptation. The scale
discriminator has the same network structure as the feature discriminator. The
framework is shown in Fig. 2.

We now have three kinds of input images instead of two. Besides the source
${x}_{\mathcal{S}^{\theta}}$ and target ${x}_{\mathcal{T}^{\sigma}}$ images,
we derive a resized version of the target image ${x}_{\mathcal{T}^{\theta}}$
via bilinear interpolation whose scale matches that of the source. These three
types of inputs allow us to create two adversarial flows, one that considers
images from the same location but different scales, and another that considers
images with the same scale but from different locations. The key to our
framework is the new, matched scale image ${x}_{\mathcal{T}^{\theta}}$. It
allows our network to focus on adapting features between the source and target
locations without also having to adapt for scale.

The feature discriminator $\mathcal{D}_{feat}$ is now updated using images
with the same scale but from different locations using the adversarial loss

$\begin{split}\mathcal{L}_{\mathcal{D}_{feat}}({p_{\theta}})=-{(1-z)}\log(\mathcal{D}_{feat}(p_{\theta}))+z\log(\mathcal{D}_{feat}(p_{\theta})).\end{split}$
(5)

Note that the difference between (4) and (5) is that the output prediction $p$
in (5) has the same scale $\theta$ no matter whether it is from the source or
target location.

The scale discriminator $\mathcal{D}_{scale}$ is updated using images from the
same location but with different scales using the adversarial loss

$\begin{split}\mathcal{L}_{\mathcal{D}_{scale}}({p}_{\mathcal{T}})=-{(1-z)}\log(\mathcal{D}_{scale}({p}_{\mathcal{T}})+z\log(\mathcal{D}_{scale}({p}_{\mathcal{T}})),\end{split}$
(6)

where $z=0$ if $p_{\mathcal{T}}$ is has the target scale $\sigma$, and $z=1$
if it has the source scale $\theta$.

We now turn to the update of the segmentation network. Similar to (2), a
feature adversarial loss is calculated using the feature discriminator to
adapt the segmentation network to the target features

$\mathcal{L}_{adv\\_feat}({x}_{\mathcal{T}^{\theta}})=-\log{(\mathcal{D}_{feat}({p}_{\mathcal{T}^{\theta}})}).$
(7)

In order to adapt the segmentation network to the target scale, a scale
adversarial loss is computed as

$\displaystyle\mathcal{L}_{adv\\_scale}({x}_{\mathcal{T}^{\sigma}})=-\log(\mathcal{D}_{scale}({p}_{\mathcal{T}^{\sigma}}).$
(8)

The overall objective for updating the segmentation network is the sum of the
three losses,

$\mathcal{L}({x}_{\mathcal{S}^{\theta}},{x}_{\mathcal{T}^{\sigma}})=\mathcal{L}_{seg}+\lambda_{f}\mathcal{L}_{adv\\_feat}+\lambda_{s}\mathcal{L}_{adv\\_scale}.$
(9)

Here, $\lambda_{s}$ and $\lambda_{f}$ are the hyperparameters for loss
weights.

Figure 3: Proposed scale attention module (SAM). The input is feature maps from the segmentation network. The output is scale-enhanced features. MatMul: matrix multiplication. Table 1: Comparing our approach which incorporates scale adaptation to resampling the target imagery with and without standard domain adaptation. Underline indicates the test set is resampled to match the training set. mIoU: higher is better. Method | Training set | GSD | Test set | GSD | mIoU
---|---|---|---|---|---
No domain adaptation | Potsdam | 5cm | Vaihingen | 9cm | 32.62
No domain adaptation | 5cm | 5cm | 30.85
No domain adaptation | 9cm | 9cm | 31.74
Standard domain adaptation | 5cm | 9cm | 40.74
Standard domain adaptation | 5cm | 5cm | 41.77
Standard domain adaptation | 9cm | 9cm | 43.09
Our approach | 5cm | 9cm | 47.66

###  4.3 Scale Attention Module

The ASPP module in the DeepLab networks has demonstrated its effectiveness at
handling multi-scale information for semantic segmentation. However, the input
to the ASPP module is the low resolution feature maps which do not contain
rich spatial information. We therefore adopt a self-attention mechanism to
learn scale-enhanced feature maps for improved domain adaptation.

We develop a scale attention module (SAM) to re-weight the multi-scale
concatenated features as shown in Fig. 3. The input to our SAM consists of
five feature maps extracted from different layers of an encoder network
composed of a DeepLabV3+ model with a ResNet101 backbone. These feature maps
are the outputs of each residual group and the final ASPP module. Adaptive
pooling is used to transform the maps to the same spatial dimension. They are
then concatenated into a single multi-scale feature map for performing self-
attention.

The concatenated feature map ${f}\in\mathbb{R}^{H\times W\times C}$, where
$H,W,C$ denote the height, width and number of channels, is first reshaped to
$\mathbb{R}^{N\times C}$, where $N=H\times W$. The scale attention
$\mathcal{A}({f})$ is then computed as

$\mathcal{A}({f})=\text{softmax}(\alpha({f})^{T}\beta({f})).$ (10)

Here, $\alpha$ and $\beta$ are two $1\times 1$ convolutional layers and $T$
indicates the transpose operation. The scale attention measures the impact of
each channel based on various scales on all the other channels.

The final re-weighted feature map $\mathcal{O}({f})$ is computed using the
scale-based attention weights through

$\mathcal{O}({f})=\mathcal{A}({f})\gamma({f}),$ (11)

where $\gamma$ is another $1\times 1$ convolutional layer to transform the
input. Finally, we reshape $\mathcal{O}({f})$ back to the original dimension
$\mathbb{R}^{H\times W\times C}$ and feed it to the segmentation head for the
final prediction. Note that we do not use the residual connection in our self-
attention module since we only want the scale enhanced feature map.

We emphasize that our proposed SAM computes reweighted features along channels
from different scales. The difference between the se-layer in Squeeze-and-
Excitation Networks [25] and channel attention [21] is that the former uses
only single-scale features while channel attention uses residuals to keep the
original features. The goal of SAM is to enhance the features to include
different scales in order to help the adversarial training when the scale
discriminator is added.

##  5 Experiments

###  5.1 Datasets

We evaluate our methods on three semantic segmentation datasets, two from the
ISPRS 2D Semantic Labeling Challenge [1] and a third from the DeepGlobe land
cover classification challenge [15].

ISPRS 2D Semantic Labeling Challenge This challenge includes two datasets,
Vaihingen and Potsdam, both labeled with six classes: impervious surface,
building, low vegetation, tree, car and clutter/background. The Vaihingen set
contains 33 image tiles with size 2494 $\times$ 2064, of which 16 are fully
annotated with class labels. The spatial resolution is 9 cm. We select five
images for validation (IDs: 11, 15, 28, 30 and 34) and the remaining 11 for
training, following [36, 41]. The Potsdam set contains 38 tiles with size
6000$\times$6000, of which 24 are annotated. The spatial resolution is 5cm. We
select seven images for validation (IDs: 2_11, 2_12, 4_10, 5_11, 6_7, 7_8 and
7_10) and the remaining 17 for training, again following [36, 41].

DeepGlobe Land Cover Classification Challenge This challenge introduces the
first public dataset offering high-resolution sub-meter satellite imagery
focusing on rural areas [15]. It contains 1146 satellite images of size
2448$\times$2448, split into training/validation/test sets with 803/171/172
images. The images are from the DigitalGlobe Vivid+ dataset and have a pixel
resolution of 50 cm. The classes include urban, agriculture, rangeland,
forest, water, barren and unknown. The DeepGlobe dataset is more challenging
due to its large coverage and dense annotations.

###  5.2 Implementation Details

We implement our framework using the PyTorch toolbox [40] on a Titan V GPU
with 12 GB memory.

Segmentation network We choose the state-of-the-art semantic segmentation
network DeepLabV3+ [10] with a ResNet101 backbone as our model architecture.
The segmentation network $\mathcal{G}$ is trained using a Stochastic Gradient
Descent (SGD) optimizer with Nesterov acceleration where the momentum is $0.9$
and the weight decay is $10^{-4}$. The initial learning rate is set to
$2.5\times 10^{-4}$ and is decayed using a polynomial decay policy with a
power of $0.9$.

Adversarial discriminator We design our scale and feature discriminators using
a fully convolutional network architecture following [43]. The discriminator
consists of 5 convolutional layers with $4\times 4$ kernels and a stride of 2
as well as a padding of 1. Each convolutional layer is followed by a
leaky_ReLU activation unit with a negative slope of $0.2$. The channel number
for each layer is set to 64, 128, 256, 512 and 1. The input to both
discriminators is the predicted segmentation maps. To train the
discriminators, we use the Adam optimizer [28] with an initial learning rate
of $10^{-4}$ and default momentum. We adopt the same polynomial decay policy
as with training the segmentation network. We set the adversarial loss
weights, $\lambda_{f}$ and $\lambda_{s}$, to 0.005.

Evaluation metrics We use segmentation metrics to evaluate land cover
classification performance. In particular, we compute the per class
Intersection over Union (IoU) and the mean over all classes (mIoU) as
percentages (%) where higher is better. We also compute the IoU gap with a
model trained using the labeled target images where lower is better. This
model serves as the oracle and can be considered an upper limit on the
performance.

Figure 4: Visualization of the feature distributions using t-SNE. From left to
right: before adaptation, domain adaptation NOT considering scale, domain
adaptation considering scale. Source domain: ISPRS Potsdam, target domain:
ISPRS Vaihingen. Our approach results in distributions that are more aligned
and distinct. t-SNE hyperparameters are consistent between visualizations.
(Zoom in to view details)

##  6 Experimental analysis

We evaluate the proposed framework as follows. We first compare our approach
to standard methods for addressing scale mismatch such as image resampling and
data augmentation (details in the supplementary material). This confirms our
motivation. We then perform an ablation study to measure the contributions of
the components of our framework. Finally, we compare our method to state-of-
the-art to domain adaptation approaches.

###  6.1 General baselines

Scale adaptation (ours) v.s. image resampling A simple yet reasonable approach
to address scale mismatch between source and target domains is simply to
resize the resolution of one to match the other. We experimented with this
using bilinear interpolation. (This assumes that the resolution of each is
known which is our case.) It turns out, perhaps surprisingly, that this is not
effective if there are other domain shifts such as location. Table 1 compares
the results between resampling and our proposed method. The second and third
rows show the results of training on 5cm Potsdam imagery and then testing on
9cm and 5cm (resampled) Vaihingen. The fourth through sixth rows incorporate
standard non-scale aware domain adaptation, AdaptSegNet [43]. The results show
some improvement, but they are still inferior to our method shown on row seven
which achieves an mIoU of 47.66% _without resampling the target dataset_. This
confirms that scale and other sources of domain mismatch are not necessarily
independent and should be handled jointly. Similar results for
Potsdam$\rightarrow$Vaihingen can be found in the supplementary materials.

Impact of $D_{scale}$ We further analyze the proposed $D_{scale}$ by
visualizing the feature representations using t-SNE [35]. As shown in Fig. 4
(left), given two datasets with different scales, the high-dimensional feature
representations from the encoder are not aligned when there is no adaptation.
The source-only model shows strong clustering of the source domain, but fails
to have similar influence on the target domain. After applying domain
adaptation i.e. AdaptSegNet (middle), the feature representations are
projected into an overlapping space. This is due to the effectiveness of
adversarial learning which forces the two distributions to be aligned.
However, the class clusters are barely visible in the target domain and so the
model has difficulty identifying the class boundaries. With our adaptation
(right), the features are not only aligned but the class clusters are
distinct. This further supports our claim that scale and other sources of
domain mismatch are not necessarily independent and are best handled together.

Table 2: Ablation study of adapting from Vaihingen scale-1 to Vaihingen
scale-1/2. $\mathcal{D}_{feat}$ and $\mathcal{D}_{scale}$ indicate the feature
and scale discriminators. SAM is the scale attention module. w/o DA means no
domain adaptation. Vaihingen scale-1 $\rightarrow$ scale-1/2
---
Method | $\mathcal{D}_{feat}$ | $\mathcal{D}_{scale}$ | SAM | mIoU | IoU gap
w/o DA |  |  |  | 50.21 | 17.43
|  | ✓ | 53.75 | 13.89
w/ DA | ✓ |  |  | 52.37 | 15.27
✓ | ✓ |  | 54.75 | 12.90
✓ |  | ✓ | 56.02 | 11.62
✓ | ✓ | ✓ | 57.29 | 10.35
scale-1/2 |  |  |  | 67.54 | 0
Table 3: Bi-directional domain adaptation: Potsdam $\leftrightarrow$
Vaihingen. ∗ indicates our DeepLabV3+ implementation. mIoU: higher is better.
IoU gap: lower is better. Target-only serves as the oracle representing an
upper limit on the performance. Potsdam $\rightarrow$ Vaihingen
---
| Imp. Sur. | Build. | Low vege. | Tree | Car | mIoU | IoU gap
Source-only | 22.85 | 52.57 | 21.56 | 46.72 | 19.39 | 32.62 | 38.41
ADDA∗ | 42.93 | 50.91 | 27.02 | 30.18 | 10.09 | 32.23 | 38.80
CyCADA∗ | 49.39 | 55.29 | 28.03 | 32.04 | 10.49 | 35.05 | 35.98
AdaptSegNet∗ | 53.72 | 56.08 | 24.74 | 39.68 | 29.49 | 40.74 | 30.28
Ours ($\mathcal{D}_{scale}$) | 55.24 | 58.23 | 26.64 | 50.87 | 36.45 | 45.49 | 25.54
Ours ($\mathcal{D}_{scale}$ \+ SAM) | 55.22 | 64.46 | 31.34 | 50.40 | 39.86 | 47.66 | 23.37
Target-only | 77.75 | 86.32 | 59.81 | 72.81 | 58.44 | 71.03 | 0.00
Vaihingen $\rightarrow$ Potsdam
Source-only | 31.06 | 37.35 | 44.13 | 16.76 | 31.29 | 32.12 | 45.0
ADDA∗ | 41.33 | 44.21 | 36.07 | 29.81 | 15.11 | 33.31 | 43.81
CyCADA∗ | 39.02 | 42.35 | 35.09 | 27.89 | 10.25 | 30.92 | 46.20
AdaptSegNet∗ | 45.81 | 41.97 | 46.08 | 35.35 | 37.42 | 41.33 | 35.79
Ours ($\mathcal{D}_{scale}$) | 49.36 | 47.08 | 51.49 | 37.17 | 39.91 | 45.00 | 32.12
Ours ($\mathcal{D}_{scale}$ \+ SAM) | 49.76 | 46.82 | 52.93 | 40.23 | 42.97 | 46.54 | 30.58
Target-only | 79.25 | 85.84 | 73.21 | 68.36 | 78.93 | 77.12 | 0.00

To show the effectiveness of the proposed approach, we consider a simple
scenario where two datasets are from the same location but with different
scales. We 1) investigate how well standard domain adaptation can adapt for
scale change even in isolation of location change, and 2) perform an ablation
study of our framework. To do this, we use bilinear interpolation to resample
images from ISPRS Vaihingen at 1/2-scale. The original images Vaihingen
scale-1 have a GSD of 9cm and serve as the source dataset. The resampled
images Vaihingen scale-1/2 have a GSD of 18cm and serve as the target. Table 2
shows that standard domain adaptation with only a feature discriminator
improves by $2.16\%$ mIoU over a non-adaptation baseline ($50.21\%\rightarrow
52.37\%$). Oracle performance, where the model is trained on the resampled
images Vaihingen scale-1/2, is $67.54\%$. The $15.27\%$ IoU gap between
standard domain adaptation and the oracle demonstrates the limited ability of
standard domain adaptation to handle scale variation. Table 2 also shows the
individual contributions of our proposed scale discriminator
($\mathcal{D}_{scale}$) and scale attention module (SAM). Adding the scale
discriminator results in an improvement of $2.38\%$ ($52.37\%\rightarrow
54.75\%$). Adding the scale attention module results in an improvement of
$3.65\%$ ($52.37\%\rightarrow 56.02\%$). We find that $\mathcal{D}_{scale}$
and SAM are complementary. Combining both results in the highest mIoU,
$57.29\%$. We also observe that our proposed SAM, as a feature enhancement
technique, provides benefits even without domain adaptation. Incorporating
just SAM into the baseline results in an improvement of $3.54\%$ mIoU
($50.21\%\rightarrow 53.75\%$).

In summary, this set of experiments shows that scale is intricately linked to
the features in CNNs and that scale-specific adaptation and enhancement is
important and advantageous.

###  6.2 Comparison Study: Small Domain Gap

We now perform cross-location domain adaptation between Potsdam and Vaihingen.
We consider this as a small domain gap scenario since both locations are in
Germany and the difference in scale is moderate. We compare our approach to
three recent state-of-the-art domain adaptation methods for semantic
segmentation, ADDA [45], CyCADA [23] and AdaptSegNet [43]. For fair
comparison, we implement our own versions ($*$) with a DeepLabV3+ segmentation
network. Table 3 contains the quantitative results and Fig. 5 shows the
qualitative results. More qualitative results can be found in the
supplementary materials.

Table 4: Bi-directional domain adaptation: DeepGlobe $\leftrightarrow$ Vaihingen. ∗ indicates our DeepLabV3+ implementation. mIoU: higher is better. IoU gap: lower is better. Target-only serves as the oracle representing an upper limit on the performance. | DeepGlobe $\rightarrow$ Vaihingen | Vaihingen $\rightarrow$ DeepGlobe
---|---|---
| Urban | R. land | Forest | mIoU | Urban | R. land | Forest | mIoU
Source-only | 14.29 | 13.60 | 0.00 | 9.30 | 0.17 | 1.05 | 2.04 | 1.09
ADDA* | 21.77 | 19.38 | 7.28 | 16.14 | 25.04 | 3.97 | 20.99 | 16.67
CyCADA* | 20.38 | 20.39 | 3.48 | 14.75 | 24.37 | 3.29 | 24.30 | 17.32
AdaptSegNet* | 22.28 | 17.71 | 5.81 | 15.27 | 26.82 | 3.98 | 31.04 | 20.61
Ours($D_{scale}$) | 26.98 | 19.16 | 11.29 | 19.14 | 28.34 | 4.74 | 39.49 | 24.19
Ours($D_{scale}$+SAM) | 27.35 | 23.43 | 12.94 | 21.24 | 29.02 | 5.42 | 41.72 | 25.39
Target-only | 82.04 | 59.81 | 72.81 | 71.55 | 68.09 | 28.77 | 75.06 | 57.31
Figure 5: Visualization of Potsdam $\rightarrow$ Vaihingen. The second and
third columns are the visualizations of channel feature maps, blue: low value,
red: high value. Our class predictions have crisper boundaries and are more
similar to the ground truth than standard methods. White: road, blue:
building, cyan: low vegetation, green: trees, yellow: cars, red: clutter.

We make several observations from Table 3. First, standard domain adaptation
does not work well on the cross-scale, cross-location problem. Take Vaihingen
$\rightarrow$ Potsdam for example. CyCADA∗ performs even worse than the
baseline, which is a model trained using the source dataset only without any
domain adaptation. Second, our proposed scale adaptive framework achieves much
higher performance in the cross-scale, cross-location scenario. With just our
new scale discriminator, we improve over the previous best by $2.81\%$
($42.19\%\rightarrow 45.00\%$). Adding the scale attention module further
boosts our accuracy to $46.54\%$, which is $4.45\%$ higher than AdaptSegNet∗
and $15.62\%$ higher than CyCADA∗.

Visualization of SAM To further show the effectiveness of SAM, we visualize in
Fig. 5 the feature maps of the most weighted channel before (column 2) and
after (column 3) applying SAM. We observe that SAM emphasizes feature maps at
a more appropriate scale/size and results in better boundary estimates.

Visual comparisons Fig. 5 shows visualizations of segmentation prediction maps
adapting Potsdam to Vaihingen (columns 4-7). We see that the cross-scale,
cross-location segmentation problem is very challenging for RS imagery. The
results without adaptation are poor. Most predictions are clutter/background
(incorrect). With standard domain adaptation, AdaptSegNet∗ generates improved
but coarse predictions. Our framework results in more accurate predictions
with crisper boundaries especially with SAM. Additional visualizations can be
found in the supplementary materials.

###  6.3 Comparison Study: Large Domain Gap

We now perform cross-location domain adaptation between DeepGlobe and
Vaihingen. We consider this as a large domain gap scenario since the location
varies significantly and the difference in scale is large (GSDs of 50cm and
9cm). Also, DeepGlobe is satellite imagery while Potsdam and Vaihingen are
aerial imagery which introduces increased sensor and spectrum variation. To
align the Vaihingen and DeepGlobe segmentation classes, we merge impervious
surface and building as urban, match low vegetation to rangeland, and match
trees to forest. The results of bi-directional domain adaptation between
DeepGlobe and Vaihingen are shown in Table 4. We make several observations.
First, due to the large domain gap, the source-only model performs quite
poorly. For example, the DeepGlobe $\rightarrow$ Vaihingen mIoU is only
$9.30\%$. Second, our scale adaptive framework again handles this now even
more challenging scenario much better than standard domain adaptation. It
achieves $5.97\%$ higher mIoU than AdaptSegNet∗ and $6.49\%$ higher mIoU than
CyCADA∗. Similar improvements can be observed for Vaihingen $\rightarrow$
DeepGlobe.

###  6.4 Discussion

#### Limitations

One limitation of our approach is that the scale of both the source and target
domains must be known (or at least the scale difference). This is usually not
a problem in RS image analysis since this information is typically available.
We have also investigated methods to automatically estimate the spatial
resolution of RS imagery using CNNs [30]. Another issue is that resampling
images, using bilinear interpolation for example, is not equivalent to images
whose scale varies due to having been acquired at different altitudes, etc.
Despite this, our results show improved performance over standard domain
adaptation. And, again, our primary goal is to illustrate the importance of
scale in domain adaptation in RS segmentation.

##  7 Conclusion

We establish that scale is a significant factor for domain adaptation in RS
image segmentation. We propose a scale adaptive adversarial learning framework
to address the challenging cross-scale, cross-location problem. This framework
includes a new scale discriminator that explicitly adapts for differences in
scale. It also includes a scale attention module to produce scale-enhanced
features. Our framework outperforms standard domain adaptation methods,
achieving state-of-the-art results for small (Potsdam $\leftrightarrow$
Vaihingen) and large (DeepGlobe $\leftrightarrow$ Vaihingen) domain gap
scenarios.

##  8 Acknowledgement

This work was funded in part by a National Science Foundation grant,
#IIS-1747535. We gratefully acknowledge the support of NVIDIA Corporation
through the donation of the GPU card used in this work.

## References

  * [1] ISPRS 2D Semantic Labeling Challenge.  http://www2.isprs.org/commissions/comm3/wg4/semantic-labeling.html.
  * [2] N. Ammour, L. Bashmal, Y. Bazi, M. M. Al Rahhal, and M. Zuair.  Asymmetric Adaptation of Deep Features for Cross-Domain Classification in Remote Sensing Imagery.  IEEE Geoscience and Remote Sensing Letters, 2018.
  * [3] N. Audebert, B. Saux, and S. Lefèvre.  Beyond RGB: Very High Resolution Urban Remote Sensing with Multimodal Deep Networks.  ISPRS Journal of Photogrammetry and Remote Sensing, 2018.
  * [4] S. Azimi, C. Henry, L. Sommer, A. Schumann, and E. Vig.  SkyScapes Fine-Grained Semantic Understanding of Aerial Scenes.  In International Conference on Computer Vision (ICCV), 2019.
  * [5] J. E. Ball, D. T. Anderson, and C. S. Chan.  Comprehensive Survey of Deep Learning in 6 Remote Sensing: Theories, Tools, and Challenges for the Community.  Journal of Applied Remote Sensing, 2017.
  * [6] L. Bashmal, Y. Bazi, H. AlHichri, M. AlRahhal, N. Ammour, and N. Alajlan.  Siamese-GAN: Learning Invariant Representations for Aerial Vehicle Image Categorization.  Remote Sensing, 2018.
  * [7] B. Benjdira, Y. Bazi, A. Koubaa, and K. Ouni.  Unsupervised Domain Adaptation Using Generative Adversarial Networks for Semantic Segmentation of Aerial Images.  Remote Sensing, 2019.
  * [8] M. Bosch, G. Christie, and C. Gifford.  Sensor Adaptation for Improved Semantic Segmentation of Overhead Imagery.  In Winter Conference on Applications of Computer Vision (WACV), 2019\.
  * [9] L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. Yuille.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs.  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017.
  * [10] L. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation.  In European Conference on Computer Vision (ECCV), 2018.
  * [11] X. Chen, S. Xiang, C. Liu, and C. Pan.  Vehicle Detection in Satellite Images by Hybrid Deep Convolutional Neural Networks.  IEEE Geoscience and Remote Sensing Letters, 2014.
  * [12] Y. Chen, W. Chen, Y. Chen, B. Tsai, Y. Wang, and M. Sun.  No More Discrimination: Cross City Adaptation of Road Scene Segmenters.  In International Conference on Computer Vision (ICCV), 2017.
  * [13] Y. Chen, W. Li, and L. Van Gool.  ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [14] Y. Chen, Y. Lin, M. Yang, and J. Huang.  CrDoCo: Pixel-Level Domain Transfer With Cross-Domain Consistency.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [15] I. Demir, K. Koperski, D. Lindenbaum, G. Pang, J. Huang, S. Basu, F. Hughes, D. Tuia, and R. Raska.  DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images.  In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018.
  * [16] X. Deng, H. L. Yang, N. Makkar, and D. Lunga.  Large Scale Unsupervised Domain Adaptation of Segmentation Networks with Adversarial Learning .  In 2019 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2019.
  * [17] X. Deng, Y. Zhu, and S. Newsam.  What Is It Like Down There? Generating Dense Ground-Level Views and Image Features from Overhead Imagery Using Conditional Generative Adversarial Networks.  In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, 2018.
  * [18] X. Deng, Y. Zhu, Y. Tian, and S. Newsam.  Generalizing Deep Models for Overhead Image Segmentation Through Getis-Ord Gi* Pooling.  arXiv preprint arXiv:1912.10667, 2019.
  * [19] H. Ding, X. Jiang, B. Shuai, A. Qun Liu, and G. Wang.  Context Contrasted Feature and Gated Multi-Scale Aggregation for Scene Segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [20] J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu.  Dual Attention Network for Scene Segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [21] J. Fu, J. Liu, H. Tian, Y. Li, Y. Bao, Z. Fang, and H. Lu.  Dual Attention Network for Scene Segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [22] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky.  Domain-Adversarial Training of Neural Networks.  The Journal of Machine Learning Research (JMLR), 2016.
  * [23] J. Hoffman, E. Tzeng, T. Park, J. Zhu, P. Isola, K. Saenko, A. Efros, and T. Darrell.  CyCADA: Cycle-Consistent Adversarial Domain Adaptation.  In International Conference on Machine Learning (ICML), 2018.
  * [24] J. Hoffman, D. Wang, F. Yu, and T. Darrell.  FCNs in the Wild: Pixel-Level Adversarial and Constraint-based Adaptation.  arXiv preprint arXiv:1612.02649, 2016.
  * [25] J. Hu, L. Shen, and G. Sun.  Squeeze-and-Excitation Networks.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [26] X. Jin, C. Lan, W. Zeng, Z. Zhang, and Z. Chen.  CaseNet: Content-Adaptive Scale Interaction Networks for Scene Parsing.  arXiv preprint arXiv:1904.08170, 2019.
  * [27] M. Kim and H. Byun.  Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  * [28] D. P Kingma and J. Ba.  Adam: A Method for Stochastic Optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [29] N. Kussul, M. Lavreniuk, S. Skakun, and A. Shelestov.  Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data.  IEEE Geoscience and Remote Sensing Letters, 2017.
  * [30] H. Liang and S. Newsam.  Estimating the Spatial Resolution of Overhead Imagery Using Convolutional Neural Networks.  In International Conference on Image Processing (ICIP), 2019.
  * [31] D. Lin, Y. Ji, D. Lischinski, D. Cohen-Or, and H. Huang.  Multi-Scale Context Intertwining for Semantic Segmentation.  In European Conference on Computer Vision (ECCV), 2018.
  * [32] Y. Liu, B. Fan, L. Wang, J. Bai, S. Xiang, and C. Pan.  Semantic Labeling in Very High Resolution Images via a Self-Cascaded Convolutional Neural Network.  ISPRS Journal of Photogrammetry and Remote Sensing, 2018.
  * [33] J. Long, E. Shelhamer, and T. Darrell.  Fully Convolutional Networks for Semantic Segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015.
  * [34] X. Ma, Z. Dai, Z. He, J. Ma, Y. Wang, and Y. Wang.  Learning Traffic as Images: A Deep Convolutional Neural Network for Large-Scale Transportation Network Speed Prediction.  Sensors, 2017.
  * [35] L. van der Maaten and G. Hinton.  Visualizing Data Using t-SNE.  Journal of Machine Learning Research (JMLR), 2008.
  * [36] E. Maggiori, Y.a Tarabalka, G. Charpiat, and P. Alliez.  High-Resolution Aerial Image Labeling with Convolutional Neural Networks.  IEEE Transactions on Geoscience and Remote Sensing, 2017.
  * [37] V. Mnih and G. E. Hinton.  Learning to Detect Roads in High-Resolution Aerial Images.  In European Conference on Computer Vision (ECCV), 2010.
  * [38] L. Mou, Y. Hua, and X. X. Zhu.  A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
  * [39] Z. Murez, S. Kolouri, D. Kriegman, R. Ramamoorthi, and K. Kim.  Image to Image Translation for Domain Adaptation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [40] A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer.  Automatic Differentiation in PyTorch.  In Advances in Neural Information Processing Systems (NeurIPS) Autodiff Workshop, 2017.
  * [41] J. Sherrah.  Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery.  arXiv preprint arXiv:1606.02585, 2016.
  * [42] Y. Tian, X. Deng, Y. Zhu, and S. Newsam.  Cross-Time and Orientation-Invariant Overhead Image Geolocalization Using Deep Local Features.  In IEEE Winter Conference on Applications of Computer Vision (WACV), 2020.
  * [43] Y. Tsai, W. Hung, S. Schulter, K. Sohn, M. Yang, and M. Chandraker.  Learning to Adapt Structured Output Space for Semantic Segmentation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [44] Y. Tsai, K. Sohn, S. Schulter, and M. Chandraker.  Domain Adaptation for Structured Output via Discriminative Representations.  In International Conference on Computer Vision (ICCV), 2019.
  * [45] E. Tzeng, J. Hoffman, K. Saenko, and T. Darrell.  Adversarial Discriminative Domain Adaptation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  * [46] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, Ł. Kaiser, and I. Polosukhin.  Attention is All You Need.  In Advances in Neural Information Processing Systems (NeurIPS), 2017\.
  * [47] M. Volpi, G. Camps-Valls, and D. Tuia.  Spectral Alignment of Multi-Temporal Cross-Sensor Images with Automated Kernel Canonical Correlation Analysis.  ISPRS Journal of Photogrammetry and Remote Sensing, 2015.
  * [48] T. Vu, H. Jain, M. Bucher, M. Cord, and P. Pérez.  DADA: Depth-aware Domain Adaptation in Semantic Segmentation.  In International Conference on Computer Vision (ICCV), 2019.
  * [49] X. Wang, R. Girshick, A. Gupta, and K. He.  Non-Local Neural Networks.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.
  * [50] L. Yan, B. Fan, S. Xiang, and C. Pan.  Adversarial Domain Adaptation with a Domain Similarity Discriminator for Semantic Segmentation of Urban Areas.  In International Conference on Image Processing (ICIP), 2018.
  * [51] J. Yuan.  Learning Building Extraction in Aerial Scenes with Convolutional Networks.  IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2017.
  * [52] H. Zhang, I. Goodfellow, D. Metaxas, and A. Odena.  Self-Attention Generative Adversarial Networks.  In International Conference on Machine Learning (ICML), 2019.
  * [53] Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, and Y. Fu.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks.  In European Conference on Computer Vision (ECCV), 2018.
  * [54] H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia.  Pyramid Scene Parsing Network.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
  * [55] L. Zhou, C. Zhang, and M. Wu.  D-LinkNet: LinkNet With Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction.  In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2018.
  * [56] Y. Zhu, K. Sapra, F. A. Reda, K. J. Shih, S. Newsam, A. Tao, and B. Catanzaro.  Improving Semantic Segmentation via Video Propagation and Label Relaxation.  In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2019.
  * [57] Z. Zhu, M. Xu, S. Bai, T. Huang, and X. Bai.  Asymmetric Non-local Neural Networks for Semantic Segmentation.  In International Conference on Computer Vision (ICCV), 2019.
  * [58] Y. Zou, Z. Yu, X. Liu, B. Kumar, and J. Wang.  Confidence Regularized Self-Training.  In International Conference on Computer Vision (ICCV), 2019.


# Electronic and mechanical properties of few-layer borophene

Hongxia Zhong  Kaixiang Huang  Guodong Yu  Shengjun Yuan<EMAIL_ADDRESS>School of Physics and Technology, Wuhan University, Wuhan, 430072, People’s
Republic of China

###### Abstract

We report first principle calculations of electronic and mechanical properties
of few-layer borophene with the inclusion of interlayer van der Waals (vdW)
interaction. The anisotropic metallic behaviors are preserved from monolayer
to few-layer structures. The energy splitting of bilayer borophene at $\Gamma$
point near the Fermi level is about 1.7 eV, much larger than the values (0.5–1
eV) of other layered semiconductors, indicating much stronger vdW interactions
in metallic layered borophene. In particular, the critical strains are
enhanced by increasing the number of layers, leading to much more flexibility
than that of monolayer structure. On the one hand, because of the buckled
atomic structures, the out-of-plane negative Poisson’s ratios are preserved as
the layer-number increases. On the other hand, we find that the in-plane
negative Poisson’s ratios disappear in layered borophene, which is very
different from puckered black phosphorus. The negative Poisson’s ratio will
recover if we enlarge the interlayer distance to 6.3 Å, indicating that the
physical origin behind the change of Poisson’s ratios is the strong interlayer
vdW interactions in layered borophene.

###### pacs:

62.20.-x, 62.20.dj, 31.15.A-, 73.22.-f

##  I Introduction

Boron, next to carbon in the periodic table, has more than sixteen bulk and
numerous low dimensional allotropesSun  _et al._ (2017); Ogitsu  _et al._
(2009). The polymorphism is originated from the electron deficiency, resulting
in the multi-center B–B bonds, which is much more complicated than that in
carbonOgitsu  _et al._ (2009). Among the polymorphic structures, two-
dimensional (2D) boron materials (borophene) have attracted extensive
theoretical interests because of their remarkable physical and chemical
propertiesZhou  _et al._ (2014); Lau and Pandey (2007); Zhou  _et al._ (2016);
Lopez-Bezanilla and Littlewood (2016). A class of borophene have therefore
been designedWu  _et al._ (2012). However, there was no evidence that the 2D
boron sheets could be fabricated experimentally until 2015Mannix  _et al._
(2015). Mannix _et al_ first reported the synthesis of 2-P$mmm$ borophene in
ultra-high vacuum conditions on Ag (111) surfaces by physical vapor deposition
in Ref. Mannix  _et al._ , 2015. Scanning tunneling microscopy
characterization of the one-atom-thick 2D sheets revealed a hexagonal
arrangement of boron atoms with an extra atom in the middle. Later, Feng _et
al_ have also grown two boron sheets, a $\beta_{12}$ sheet and $\chi_{3}$
sheet on Ag (111) surfaces, and the $\beta_{12}$ phase is found to be with
gapless Dirac conesFeng  _et al._ (2016a, b, 2017). These two phases are in
triangular lattices but in flat geometry and with a periodic arrangement of
atom vacanciesFeng  _et al._ (2016a, b). The observed phase depends on the
deposition rate and temperature in the experiment, confirming the predicted
large polymorphism of borophene.

The successful fabrications of borophene have inspired much follow-up works,
especially for the buckled 2-P$mmm$ structure. Owing to its anisotropic atomic
structure, the buckled borophene shows highly anisotropic metallic properties,
very different from the semi-metallic grapheneNeto  _et al._ (2009) and
semiconducting transition-metal dichalcogenides (TMDCs) Mannix  _et al._
(2015); Lherbier  _et al._ (2016); Alvarez-Quiceno  _et al._ (2017); Peng  _et
al._ (2016); Wang  _et al._ (2012); Liang  _et al._ (2013); Qiu  _et al._
(2013); Ramasubramaniam (2012); Zhong  _et al._ (2015); Xia  _et al._ (2014).
The predicated Fermi velocity of hydrogenated borophene ($3.5\times 10^{6}$
m/s) is nearly four times higher than that of graphene ($8.2\times 10^{5}$
m/s)Malko  _et al._ (2012); Cheng  _et al._ (2017); Xu  _et al._ (2016). The
investigated optical spectra attest high optical transparency (up to 100%
transmission) predicted up to roughly 3 eV, making borophene more transparent
than grapheneLherbier  _et al._ (2016). Borophene thus can be considered as a
good candidate of transparent conductive 2D material for photovoltaics and
touch screens, due to its robust metallicity, high Fermi velocity, and ultra-
high optical transparency. Furthermore, pristine borophene has been predicted
to exhibit phonon-mediated superconductivity with critical temperature
$T_{\mathrm{c}}$ in the range of 10–20 KPenev  _et al._ (2016); Xiao  _et al._
(2016). For the mechanical properties, the buckled borophene shows
considerable toughness, and it has been demonstrated that the in-plane Young’s
modulus along the armchair direction (398 GPa$\cdot$nm)Mannix  _et al._ (2015)
can rival that of graphene (340 GPa$\cdot$nm)Neto  _et al._ (2009). Notably,
monolayer borophene is calculated to exhibit negative in-plane and out-of-
plane Poisson’s ratios. The negative Poisson’s ratios of monolayer borophene
make it worth further exploitation in many applications, such as
medicineScarpa (2008), tougher compositesSun and Pugno (2013), national
security and defensesPrawoto (2012); Evans and Alderson (2000). In the
following, we focus solely on the buckled 2-P$mmm$ borophene, a promising 2D
material which has fantasy physical properties.

Previous theoretical studies of 2-P$mmm$ borophene mainly focused on the
physical properties of monolayer boropheneLherbier  _et al._ (2016); Alvarez-
Quiceno  _et al._ (2017); Peng  _et al._ (2016). A comprehensive study of the
electronic and elastic properties of the few-layer borophene is still lacking.
Moreover, the physical properties of layered 2D materials are highly dependent
on their thickness, such as the band crossover in hexagonal TMDCs which is
originated from the interlayer van der Waals (vdW) interaction and variations
of screeningMak  _et al._ (2010); Zhang  _et al._ (2014); Splendiani  _et al._
(2010). Therefore, it is of both fundamental and practical interests to attain
a better understanding of the interlayer vdW interactions among the layered
borophene. We note that it is inclined to form 3D boron clusters instead of 2D
layered borophene by depositing additional boron atoms on monolayer flat
$\beta_{12}$ sheet and $\chi_{3}$ borophene supported by Ag (111) surfaces in
Ref. Feng  _et al._ , 2016a, as a result of the saturation of the Ag–B
interfacial interactions. However, in the experiments there were occasionally
small islands observed of second layer, although with different structures
comparing to the first layerFeng  _et al._ (2016a). We thus believe that few-
layer buckled 2-P$mmm$ borophene can be fabricated with different experimental
method or a proper choose of the substrate.

In this article, we present a comprehensive theoretical study of the
electronic and elastic properties of few-layer borophene, using a first-
principle approach including the vdW interaction. We first give a brief
description of the numerical methods, and then discuss the chemical bonding
nature of borophene, the structure and electronic properties of layered
structure with different types of interlayer stacking, and determine the
preferable stacking mode which has the lowest total energy. We will then focus
on the layered structure with this stacking mode, and investigate the
electronic and mechanical properties of bilayer, trilayer, and four-layer
borophene. The out-of-plane and in-plane Possion’s ratios will be studied in
detail, by considering the influence of the interlayer vdW interaction. We
will summarize our main findings in the conclusion.

##  II Methods

Our calculations are performed using the projector augmented wave (PAW)
methodBlochl (1994) implemented in the Vienna ab initio simulation package
(VASP) codeKresse and Furthmuller (1996); Kresse and Joubert (1999). Perdew,
Burke, and Ernzerhof (PBE) form of the generalized gradient approximation
(GGA) exchange-correlation functional with van der Waals corrections (vdW-
DFT)Klimes  _et al._ (2011); Dion  _et al._ (2004); Perdew  _et al._ (1996),
and the PAW pseudo potentialsBlochl (1994) are adopted. The cut-off energy is
set to 500 eV after convergence tests. A $\Gamma$-centered Monkhorst-Pack
$k$-pointMonkhorst and Pack (1976); Pack and Monkhorst (1977) grid of
$15\times 13\times 1$ for one borophene unit cell is chosen for relaxations
and the grid of $25\times 23\times 1$ for property calculations. In our
current calculations, the total energy is converged to less than $10^{-5}$ eV.
The maximum force is less than 0.02 eV/Å during the optimization. A vacuum
space between neighboring supercells is set to be more than 25 Å to avoid
spurious interactions. The Crystal Orbital Hamilton Population (COHP) curves
are calculated using the LOBSTE softwareMaintz  _et al._ (2016).

For 2D orthorhombic borophene, there are four non-zero elastic stiffness
constants $C_{11}$, $C_{22}$, $C_{66}$ and $C_{12}$, and the stress-strain
relationship is obtained from Hooke’s law under plane-stress conditionZhou and
Huang (2008).

$\left(\begin{array}[]{c}\sigma_{xx}\\\ \sigma_{yy}\\\
\sigma_{xy}\end{array}\right)=\left(\begin{array}[]{ccc}C_{11}&C_{12}&0\\\
C_{21}&C_{22}&0\\\
0&0&C_{66}\end{array}\right)\left(\begin{array}[]{c}\varepsilon_{xx}\\\
\varepsilon_{yy}\\\ 2\varepsilon_{xy}\end{array}\right)$ (1)

Where $C_{ij}(i,j=1,2,6)$ is the in-plane stiffness tensor and is equal to the
second partial derivative of strain energy $E_{\mathrm{s}}$ as a function of
strain $\bm{\varepsilon}$ in the range $-2\%<\bm{\varepsilon}<2\%$ with an
increment of 0.5%, based on the following formulaZhang  _et al._ (2015)

$E_{\mathrm{S}}=\frac{1}{2}C_{11}\varepsilon_{xx}^{2}+\frac{1}{2}C_{22}\varepsilon_{yy}^{2}+C_{12}\varepsilon_{xx}\varepsilon_{yy}+2C_{66}\varepsilon_{xy}^{2}.$
(2)

The engineering strain is defined as $\varepsilon=(L-L_{0})/L_{0}$, where $L$
and $L_{0}$ are the lattice constants of the strained and unstrained
structures, respectively. Here, we get the elastic constants $C_{ij}$ using
the VASPKIT codeWang (2013). Then, the Young’s modulus $E$ and Poisson’s ratio
$\nu$ can be derived asWei and Peng (2014a)

$\displaystyle E_{x}=\frac{C_{11}C_{22}-C_{12}C_{21}}{C_{22}}$
$\displaystyle,E_{y}=\frac{C_{11}C_{22}-C_{12}C_{21}}{C_{11}},$ (3)
$\displaystyle\nu_{xy}=\frac{C_{21}}{C_{22}}$
$\displaystyle,\quad\nu_{yx}=\frac{C_{12}}{C_{11}}.$ (4)

##  III Structural and electronic properties

###  III.1 Chemical bonding nature of borophene

Figure 1: (a) Side and top views of monolayer borophene. The atoms occupying
the A and B sites are on different planes, separated by a distance
$\varDelta=0.941\mbox{\AA}$. The blacked rectangle indicates a unit cell. The
bond B1–B2 is along the armchair direction, and the bond B1–B3 is along the
zigzag direction. (b) COHP curves of monolayer borophene containing B1–B2 and
B1–B3 interactions.

The ball-stick structures of monolayer buckled borophene are presented in Fig.
1(a). Our benchmark calculations on monolayer borophene give lattice constants
of $a=1.613~{}\mbox{\AA}$ and $b=2.880~{}\mbox{\AA}$, in good agreement with
previous valuesLherbier  _et al._ (2016); Alvarez-Quiceno  _et al._ (2017);
Peng  _et al._ (2016); Xiao  _et al._ (2016); Wang  _et al._ (2016). There is
a buckling along $\bm{b}$ direction with height
$\varDelta=0.941~{}\mbox{\AA}$, while no corrugations are observed along
$\bm{a}$ direction. To analyze the nature of the chemical bonding in
borophene, Fig. 1(b) shows our bonding analysis based on -pCOHP. The Fermi
level is dominated by B1–B2 interactions. The amount of occupied bonding B–B
between B1–B2 atoms is larger than those between B1–B3 atoms. It clearly shows
that B1–B2 and B1–B3 bonding interactions consist with the similar interatomic
B1–B2 (1.613 Å) and B1–B3 (1.855 Å) lengths. A delocalization of electrons
over B1 and B3 atoms is through the formation of multi-center bonds along the
zigzag direction. Such kind of resonant bonding can also explain the slightly
longer bond length compared to classical B–B single bond (1.686 Å)Braunschweig
and Dewhurst (2013). On the other hand, B1 and B2 atoms form strong $\sigma$
bond along the armchair direction.

###  III.2 Geometry and stability of layered borophene

Since the structures of bilayer borophene are rather complicated, we only
consider six stacking configurations (AA, AB, AAp, ABp, AAb and ABb) with high
symmetry in this work, as shown in Fig. 2(a)–(c). There are three kinds of top
views of these six stacking modes. For AA- and AB-stacking modes, the top
layer is directly stacked on the bottom layer. The AAp (ABp) and AAb
(ABb)-stacking modes can be viewed as shifting half of the bond length along
either B1–B2 or B1–B3 bond direction. Two kinds of side views, coming from the
fact that the bottom layer could have the same buckling order as the top or
the opposite, have been discussed. All sublayers are initially separated by a
distance of 3.0 Å, and we use the optimized lattice constant of the monolayer
borophene as the initial lattice constants for bilayer structures. All
structures are totally free to relax (positions of the atoms and the lattice
constants).

Figure 2: Six high-symmetry configurations of bilayer borophene are (a) AA and
AB with the same top view, (b) AAp and ABp, (c) AAb and ABb. (d) Total energy
of these six considered stacking modes as a function of the interlayer
distance between two nearest boron atoms of the top and bottom layer along
$\bm{z}$ direction. The interlayer distance varies around the equilibrium
distance of each configuration and all atoms are relaxed with this constraint.
The lowest calculated total energy (in this case, the total energy associated
with the AA configuration) was set to zero, and the others were calculated
with respect to this one.

The lattice parameters of the investigated structures are the same of
$a=1.610~{}\mbox{\AA}$ and $b=2.898~{}\mbox{\AA}$, with the same buckling
height of 0.941 Å. The most notable difference among the six stacking modes is
the interlayer distance between the top and bottom layers, varying from 3.072
Å in the AA-stacking to 3.391 Å in the AB-stacking. It can be seen that the
energy of the system highly depends on its interlayer distance. To better
understand this dependence and to ensure those stacks obtained from the free
relaxions reach minimum energy of the system rather than a local minimum, we
study the evolution of the total energy as a function of the interlayer
distance for each system in Fig. 2(d). First, these six configurations have an
energy minimum without phase transitions. Because different stacking order
leads to different $\pi$–$\pi$ interaction distance between delocalized states
and thereby different interaction strength, the order of stability of the
considered structures is as follows: $\mathrm{AA>AAp>AAb>ABb>AB>ABp}$. AA-
stacking structure is found to be the most energetically preferred
configuration with the smallest interlayer distance and corresponding
strongest interlayer interaction. It is very different from other 2D materials
whose preferred configurations are in AB-stacking, such as grapheneOhta  _et
al._ (2006), silicenePadilha and Pontes (2015), black phosphoreneDai and Zeng
(2014), and hexagonal TMDCs Ramasubramaniam  _et al._ (2011). Furthermore, the
corresponding interlayer distance 3.072 Å is slightly smaller than the value
(3.214 Å) of AB-stacked phosphorusDai and Zeng (2014), which has also a
buckled monolayer structure. Based on the AA-stacking bilayer borophene, we
take the stacking sequence of AAA and AAAA into trilayer and four-layer
borophene structures, respectively.

###  III.3 Electronic properties of layered borophene

Figure 3: (a)–(d) Band structures of monolayer, bilayer, trilayer, and four-
layer borophene, respectively. The Fermi level is set to zero. Blue rectangles
show the band splitting near the Fermi level.

The electronic band structures of monolayer and AA-stacking bilayer, trilayer,
and four-layer borophene are shown in Fig. 3. Compared with the case of
monolayer borophenePeng  _et al._ (2016), the Fermi level is crossed by more
bands because of the band splitting. Hence, the robust metal feature is
retained with the increasing of the layer number. The local band gaps
resulting from the buckling along $\Gamma$–Y and S–X directions still exist.
The band gap along $\Gamma$–Y direction decreases from 3.263 eV (bilayer) to
2.117 eV (four-layer), as a result of the increasing band splitting with the
increasing layer number. Because there is no splitting along S–X direction,
the band gap along this high-symmetry line barely depends on the layer number.
Therefore, layered borophene behaves anisotropic in electronic properties
resulting from the anisotropic atomic structure, and the electrical
conductivity is expected to be confined along the uncorrugated armchair
direction.

Compared to the band structure of monolayer borophene, the addition of layers
results in the band splitting around the $\Gamma$ point in the band structures
of layered borophene. The values of band splitting at the $\Gamma$ point for
bilayer, trilayer, and four-layer borophene are 1.701, 2.716 and 3.113 eV,
respectively. The value of bilayer borophene band splitting is much larger
than that (0.5–1 eV) of bilayer MoS2Ramasubramaniam  _et al._ (2011) and black
phosphorusDai and Zeng (2014). This indicates that the interlayer interaction
in metallic layered borophene is much stronger than those in other
semiconducting 2D materials. To understand the interlayer interaction
contribution, we plot the isosurfaces of the charge density corresponding to
the VB1 and VB2 of bilayer borophene as insets of Fig. 3(b), respectively.
According to the spatial distribution of the charge density, we can recognize
the antibonding and bonding characteristics of the VB1 and VB2 states, which
come from the hybridization between the electronic structures of these two
sublayers. The bonding characteristics of the VB2 at $\Gamma$ point shows
clearly a large overlap of the wave functions from the top and bottom layers,
confirming the strong interlayer interaction in layered borophene.

##  IV Mechanical properties

###  IV.1 Ideal strength and critical strain of layered borophene

Starting with the optimized borophene structures, tensile strain is applied in
either uniaxial (armchair or zigzag) or biaxial direction to explore the ideal
tensile strength and the critical strain (the strain at which ideal strength
reaches). With each uniaxial strain applied, the lattice constant along the
transverse direction and boron atoms are fully relaxed. For biaxial strain,
equibiaxial tension is applied and boron atoms in the unit cell are fully
relaxed. We calculate the stress-strain relation of 2D layered borophene
systems using the method described in the 2D black phosphoreneWei and Peng
(2014b). In a 2D system, the stress is expressed by multiplying the Caucy
stresses and $Z/n$ to obtain the equivalent stress, where $Z$ is the thickness
of unit cell along the vacuum direction and $n$ is the layer number of the
system. To validate our calculations, we compute the mechanical properties of
monolayer borophene, such as the elastic stiffness constants and Poisson’s
ratios shown in Table 1, which are consistent with previous valuesMannix  _et
al._ (2015); Wang  _et al._ (2016).

Figs. 4(a)–(c) present our calculated stain-stress relations. The stress-
strain behaviors of layered borophene become nonlinear as the applied strain
increases, similar to the case of monolayer structureWang  _et al._ (2016).
From monolayer to layered borophene, the ideal strength along the armchair
direction slightly increases from 24.0 N/m (monolayer)Wang  _et al._ (2016) to
25.2–26.3 N/m (few-layer). This suggests that the outstanding large tensile
strength of borophene is enhanced in layered structure, which is crucial for
the mechanical application of few-layer borophene. This enhancement can be
explained by the change of the $\sigma$ bond. The $\sigma$ bond length in
layered structure is 1.610 Å, which is shorter than that of monolayer (1.613
Å). On the other hand, the ideal strengths of layered borophene are 9.5–9.8
N/m along the zigzag direction, smaller than that of monolayer (12.4 N/m). The
decrease originates from the enhancement of multi-center bonds by reducing the
corresponding bond lengths from monolayer (1.855 Å)Wang  _et al._ (2016) to
multilayer (1.836 Å). For the biaxial tension case, the curve has a maximum
value of 21.0 N/m, larger than that of monolayer (19.2 N/m)Wang  _et al._
(2016). The ideal strength of borophene is smaller than those of graphene
(36.74–40.41 N/m), but larger than those of silicene (5.26–7.59 N/m), MoS2
(9.59–14.75 N/m), and black phosphorene (4.44–9.99 N/m)Wang  _et al._ (2016).

Unlike the increasing tensile strength along the armchair direction and
decreasing strength along the zigzag direction from mono to layered borophene,
the critical strain is always increasing in all engineered directions with the
increasing number of layers, similar to the trend of black phosphoreneWei and
Peng (2014b). For example, the critical strains are 14% (armchair), 15–16%
(zigzag), and 14–16% (biaxial) for few-layer borophene, which are larger than
those corresponding critical strains (10% armchair, 12% zigzag, 13% biaxial)
in monolayer structureWang  _et al._ (2016). The increase of critical strains
means that the mechanical flexibility of borophene is enhanced from monolayer
to multilayer. We note that the increasing critical strains for few-layer
borophene are still smaller than those of other 2D materials, such as graphene
(19–27%), black phosphorene (27–33%), and MoS2 (18–26%)Wang  _et al._ (2016).
To summarize, few-layer borophene exhibit strong anisotropic responses for
these three types of applied strains from the stress-strain curves.

###  IV.2 Buckling height of layered borophene

Figure 4: The stress-strain relations for (a) bilayer (b) trilayer (c) four-
layer borophene. The critical strains are 14% (along the armchair direction)
and 15–16% (along the biaxial direction). Figure 5: The calculated dependence
of buckling heights of (a) bilayer (b) trilayer (c) four-layer borophene under
three types of tension. The buckling heights decrease sharply at the critical
strain (14%) point, and drop to zero at the uniaxial along $\bm{a}$ of 20%.

Buckling height is a critical parameter for buckled 2D materials, obviously
different from other flat 2D systems. We therefore investigate the dependences
of buckling height of layered borophene on three types of applied tension in
Fig. 5. From monolayer to multilayer, the trends of buckling height dependent
on tension are nearly the same, showing highly anisotropic and non-monotonic.
The buckling heights decrease sharply at the strain of 14% along the armchair
direction, and 15–16% along the biaxial direction, exactly corresponding to
the critical strains of few-layer systems. The layered borophene turns into a
graphene-like planar structure instead of the original buckling structure when
the strain approaches 19% (16–17%) along the armchair (biaxial) direction.
Such turning means that the original borophene structure becomes unstable and
is destroyed because of the phonon instability. On the contrary, if tension is
applied along the zigzag direction, the buckling height increases
monotonically with the increasing strain. It means that the out-of-plane
Poisson’s ratios are negative for layered borophene, similar to monolayer
boropheneWang  _et al._ (2016). This is because B1–B3 bonding along the zigzag
direction decreases with increasing strain along this direction. The
anisotropic out-of-plane Poisson’s ratios in few-layer borophene are different
from other flat 2D isotropic materials, for example, the layered graphene,
$h$-BN, and MoS2 have negative, near zero, and positive out-of-plane Poisson’s
ratios, respectivelyWoo  _et al._ (2016).

###  IV.3 Mechanical constants of layered borophene

Table 1: The calculated elastic stiffness constants, Young’s modulus, and Poisson’s ratios for monolayer and layered borophene. There are four non-zero elastic constants for 2D borophene, because of their orthogonal primitive cell. Here, we get the elastic constants $C_{11}$, $C_{22}$, $C_{66}$ and $C_{12}$ by fitting the data of elastic strain energy $E_{\mathrm{s}}$ ($\bm{\varepsilon}$) as a function of $\bm{\varepsilon}$ in the strain range $-2\%\leq\bm{\varepsilon}\leq 2\%$ with an increment of 0.5%. The Young’s modulus and Poisson’s ratios can be derived from the elastic constants. Poisson’s ratio is defined by the ratio of the strain in the vertical direction to that of the applied direction. The calculated values of monolayer structure are in good agreement with previous theoretical results. System | Elastic stiffness constants | Young’s modulus | Poisson’s ratio
---|---|---|---
(GPa$\cdot$nm) | (GPa$\cdot$nm)
$C_{11}$ | $C_{22}$ | $C_{66}$ | $C_{12}$ | $E[100]$ | $E[010]$ |  $\nu$[100] |  $\nu$[010]
1-layer | 396.6 | 158.4 | 86.5 |  $-$3.47 | 397 | 158 |  $-$0.022 |  $-$0.009
1-layerWang and Geng (2017) | 377.0 | 161.0 | 84.0 | 1.00 | 377 | 162 | 0.005 | 0.002
1-layerWang and Geng (2017) | 405.0 | 172.0 | 96.0 |  $-$1.00 | 405 | 172 |  $-$0.006 |  $-$0.003
1-layerMannix  _et al._ (2015) | 398.0 | 170.0 | 94.0 |  $-$7.00 | 398 | 170 |  $-$0.040 |  $-$0.020
2-layer | 380.0 | 143.8 | 75.2 | 7.44 | 380 | 144 | 0.052 | 0.020
3-layer | 361.1 | 141.5 | 72.5 | 11.29 | 360 | 141 | 0.080 | 0.031
4-layer | 337.9 | 136.0 | 70.8 | 12.37 | 338 | 136 | 0.091 | 0.037

In addition to the stress-strain curves and buckling height dependence, we
also calculate elastic constants, Young’s modulus, and summarize them together
with Poisson’s ratios in Table 1. Due to the anisotropy of the borophene
structure, the elastic constants, Young’s modulus, and Poisson’s ratios have
different values along the zigzag and the armchair directions. From monolayer
to few-layer borophene, the Young’s modulus are decreasing from 396.6
GPa$\cdot$nm (armchair) and 158.4 GPa$\cdot$nm (zigzag) to 337.9 GPa$\cdot$nm
(armchair) and 136.0 GPa$\cdot$nm (zigzag). This decreasing trend also appears
in the buckled black phosphorusWei and Peng (2014b). For both monolayer and
few-layer borophene, the Young’s modulus along the armchair direction are
about 2.5 times larger than their counterparts along the zigzag direction,
indicating that it is more difficult to apply strain along the armchair
direction. One may notice that the Young’s modulus of four-layer borophene
along the armchair direction is still very large. This is because the
interlayer interactions have negligible influence to the strong $\sigma$ bond
along the armchair direction. The large Young’s modulus along the armchair
direction suggest that few-layer borophene, similar to monolayer borophene,
demonstrates super-hardness compared to other 2D materials. This makes
borophene a great candidate for practical large-magnitude-strain engineering.

Figure 6: The vertical strain $\bm{\varepsilon}_{\mathrm{V}}$ versus applied
strain $\bm{\varepsilon}$ along the armchair direction for monolayer (a) and
bilayer (b) borophene, respectively. The positive (negative)
$\bm{\varepsilon}$ means a tensile (compressive) strain. The vdW interaction
is in principle inverse proportion to the layer distance.

Poisson’s ratios measure the fundamental mechanical responses of solid against
external loads. The out-of-plane Poisson’s ratio, which is related to the
change of the buckling height, has been discussed in previous section. We will
thus focus on the following in-plane Poisson’s ratios in few-layer borophene.
For an applied strain $\bm{\varepsilon}$ along the armchair direction in the
monolayer, the responding strain $\bm{\varepsilon}_{\mathrm{V}}$ occurs along
the zigzag direction as shown in Fig. 6(a). When a strain ($\bm{\varepsilon}$)
is applied along the armchair direction, the $\bm{\varepsilon}_{\mathrm{V}}$
roughly increases with the increasing strain. That is, the larger the lattice
constant $\bm{a}$ is, the larger the lattice constant $\bm{b}$ is. If a strain
is applied along the zigzag direction, the $\bm{\varepsilon}_{\mathrm{V}}$
tends to increase with the increasing strain in the range from -16% to 10%.
This range is within the critical strains along the zigzag direction for
monolayer borophene. Monolayer borophene therefore shows negative in-plane
Poisson’s ratio of -0.022 along [100] and -0.009 along [010] direction,
confirming the anisotropic mechanical properties and in good agreement with
previous resultsMannix  _et al._ (2015).

For few-layer borophene, it is surprising to find that the negative in-plane
Poisson’s ratio in monolayer borophene dramatically changes into positive, for
example, 0.052 along [100] and 0.020 along [010] directions in bilayer
borophene, as shown in Table 1. We should emphasise that this observation is
totally different from other 2D materials, even for puckered black phosphorus
and arsenene in which the negative Poisson’s ratios are preserved from
monolayer to multilayerWang  _et al._ (2017); Han  _et al._ (2015). In order
to check the reliability of the positive Poisson’s ratio in few-layer
borophene, we show in Fig. 6(b) the responding strain
$\bm{\varepsilon}_{\mathrm{V}}$ for applied strain $\bm{\varepsilon}$ for
bilayer borophene as an example, intuitively reflecting the Poisson’s ratio.
Very different from the case of monolayer borophene, the vertical strain
$\bm{\varepsilon}_{\mathrm{V}}$ is negative (positive) when the engineered
strain is positive (negative) and smaller (larger) than 14% (-10%) in bilayer
borophene, indicating a positive Poisson’s ratio along the armchair direction.
For the applied strain along the zigzag direction, the curve of
$\bm{\varepsilon}_{\mathrm{V}}$ versus $\bm{\varepsilon}$ is similar to the
case of armchair direction, indicating also a positive Poisson’s ratio. We
note that the engineered strains considered here are within the range of the
corresponding critical strains, and these results confirm the positive
Poisson’s ratios shown in Table 1.

Figure 7: The vertical strain $\bm{\varepsilon}_{\mathrm{V}}$ versus applied
strain $\bm{\varepsilon}$ along the armchair direction for varying interlayer
distance 4.0 Å, 5.0 Å, 6.3 Å for bilayer borophene, respectively.

The dramatic change of the in-plane Poisson’s ratio, i.e., from negative in
the monolayer to positive in the multilayer, does not appear in other 2D
materials. In fact, this novel phenomenon is a direct consequence of the very
strong interlayer vdW interactions appeared in layered borophene, as we
discussed in previous sections (see the strong interlayer bonding states shown
in Fig. 3(b)). The interlayer interactions in 2D materials are in general much
weaker comparing to the intralayer interactions, even in puckered atomic
structures such as black phosphorus and arsenene. As a further check of the
influence of the vdW interaction, we investigate the change of Poisson’s ratio
by varying the interlayer distance in few-layer borophene. Using bilayer
borophene as an example, we analyse the curve of vertical strain
$\bm{\varepsilon}_{\mathrm{V}}$ for applied $\bm{\varepsilon}$ along the
armchair direction, as displayed in Fig. 7. It is clear that with different
interlayer distance, the curves of vertical strain
$\bm{\varepsilon}_{\mathrm{V}}$ versus $\bm{\varepsilon}$ become very
different. When the interlayer distance approaches 6.3 Å, the curve becomes
similar to that of monolayer borophene, with the negative Poisson’s ratio
recovered. This confirms that the vdW interlayer interactions are much
stronger in metallic borophene than that in semiconducting 2D materials (such
as flat TDMCs and puckered black phosphorus), resulting in a shorter
interlayer distance and therefore intensively altering the mechanical
properties of few-layer borophene. Furthermore, as discussed in Ref. Woo  _et
al._ , 2016; Dobson  _et al._ , 2006, the energy of the interlayer vdW
interactions for 2D thick metals is proportional to $d^{-2}$ (where $d$ is the
interlayer distance), while the asymptotic vdW energy of parallel structures
is proportional to $d^{-4}$ for 2D insulators. Thus, the decay speed of vdW
interactions in few-layer metallic borophene is significantly slower than that
in semiconducting black phosphorus and arsenene.

##  V Conclusion

In conclusion, we have studied electronic and mechanical properties of few-
layer borophene based on the buckled 2-P$mmm$ monolayer structure as
synthesized by Mannix _et al_ in Ref. Mannix  _et al._ , 2015. We find that
the AA-stacking mode is the most stable one among the six high-symmetry
stacking configurations for bilayer structures. From mono to layered
borophene, the robust anisotropic metallic features are maintained, with large
energy splitting at $\Gamma$ point ($\sim$1.5 eV), confirming strong
interlayer vdW interactions. Since the layered structures can withstand larger
critical strains than that in monolayer, layered borophene exhibit more
flexibility than monolayer one. Because of the preserved multi-center bonds
along the zigzag direction, the out-of-plane negative Poisson’s ratios are
preserved. In contrast, the in-plane negative Poisson’s ratios in the
monolayer become positive in layered borophene. This novel phenomenon is a
direct consequence of the very strong vdW interlayer interactions, and the
negative Poisson’s ratios could recover if the interlayer distance is
increased to 6.3 Å artificially. The dramatic change of the in-plane Poisson’s
ratio from monolayer to multilayer does not appear in other 2D materials, even
in puckered black phosphorus and arsenene. We hope that our theoretical
results will inspire considerable experimental enthusiasm of few-layer
borophene, especially for potential applications in novel electronic and
mechanical devices.

###### Acknowledgements.

We acknowledges the financial support from Thousand Young Talent Plan (China),
and also thank the supercomputing system in the Supercomputing Center of Wuhan
University for our numerical calculations.

## References

  * Sun  _et al._ (2017) X. Sun, X. F. Liu, J. Yin, J. Yu, Y. Li, Y. Hang, X. C. Zhou, M. L. Yu, J. D. Li, G. A. Tai, and W. L. Guo, Adv. Funct. Mater. 27, 1603300 (2017).
  * Ogitsu  _et al._ (2009) T. Ogitsu, F. Gygi, J. Reed, Y. Motome, E. Schwegler, and G. Galli, J. Am. Chem. Soc. 131, 1903 (2009).
  * Zhou  _et al._ (2014) X. F. Zhou, X. Dong, A. R. Oganov, Q. Zhu, Y. J. Tian, and H. T. Wang, Phys. Rev. Lett. 112, 085502 (2014).
  * Lau and Pandey (2007) K. C. Lau and R. Pandey, J. Phys. Chem. C 111, 2906 (2007).
  * Zhou  _et al._ (2016) X. F. Zhou, A. R. Oganov, Z. H. Wang, I. A. Popov, A. I. Boldyrev, and H. T. Wang, Phys. Rev. B 93, 085406 (2016).
  * Lopez-Bezanilla and Littlewood (2016) A. Lopez-Bezanilla and P. B. Littlewood, Phys. Rev. B 93, 241405 (2016).
  * Wu  _et al._ (2012) X. J. Wu, J. Dai, Y. Zhao, Z. W. Zhuo, J. L. Yang, and X. C. Zeng, ACS Nano 6, 7443 (2012).
  * Mannix  _et al._ (2015) A. J. Mannix, X. F. Zhou, B. Kiraly, J. D. Wood, D. Alducin, B. D. Myers, X. L. Liu, B. L. Fisher, U. Santiago, J. R. Guest, M. J. Yacaman, A. Ponce, A. R. Oganov, M. C. Hersam, and N. P. Guisinger, Science 350, 1513 (2015).
  * Feng  _et al._ (2016a) B. Feng, J. Zhang, Q. Zhong, W. Li, S. Li, H. Li, P. Cheng, S. Meng, L. Chen, and K. Wu, Nat. Chem. 8, 563 (2016a).
  * Feng  _et al._ (2016b) B. J. Feng, J. Zhang, R. Y. Liu, T. Iimori, C. Lian, H. Li, L. Chen, K. H. Wu, S. Meng, F. Komori, and I. Matsuda, Phys. Rev. B 94, 041408 (2016b).
  * Feng  _et al._ (2017) B. Feng, O. Sugino, R. Y. Liu, J. Zhang, R. Yukawa, M. Kawamura, T. Iimori, H. Kim, Y. Hasegawa, H. Li, L. Chen, K. Wu, H. Kumigashira, F. Komori, T. C. Chiang, S. Meng, and I. Matsuda, Phys. Rev. Lett. 118, 096401 (2017).
  * Neto  _et al._ (2009) A. H. C. Neto, F. Guinea, N. M. R. Peres, K. S. Novoselov, and A. K. Geim, Rev. Mod. Phys. 81, 109 (2009).
  * Lherbier  _et al._ (2016) A. Lherbier, A. R. Botello-Mendez, and J. C. Charlier, 2D Mater. 3, 045006 (2016).
  * Alvarez-Quiceno  _et al._ (2017) J. C. Alvarez-Quiceno, R. H. Miwa, G. M. Dalpian, and A. Fazzio, 2D Mater. 4, 025025 (2017).
  * Peng  _et al._ (2016) B. Peng, H. Zhang, H. Z. Shao, Y. F. Xu, R. J. Zhang, and H. Y. Zhua, J. Mater. Chem. C 4, 3592 (2016).
  * Wang  _et al._ (2012) Q. H. Wang, K. Kalantar-Zadeh, A. Kis, J. N. Coleman, and M. S. Strano, Nat. Nanotechnol. 7, 699 (2012).
  * Liang  _et al._ (2013) Y. F. Liang, S. T. Huang, R. Soklaski, and L. Yang, Appl. Phys. Lett. 103, 042106 (2013).
  * Qiu  _et al._ (2013) D. Y. Qiu, F. H. da Jornada, and S. G. Louie, Phys. Rev. Lett. 111, 216805 (2013).
  * Ramasubramaniam (2012) A. Ramasubramaniam, Phys. Rev. B 86, 115409 (2012).
  * Zhong  _et al._ (2015) H. X. Zhong, S. Y. Gao, J. J. Shi, and L. Yang, Phys. Rev. B 92, 115438 (2015).
  * Xia  _et al._ (2014) F. Xia, H. Wang, and Y. Jia, Nat. Commun. 5, 4458 (2014).
  * Malko  _et al._ (2012) D. Malko, C. Neiss, F. Vines, and A. Gorling, Phys. Rev. Lett. 108, 086804 (2012).
  * Cheng  _et al._ (2017) T. Cheng, H. F. Lang, Z. Z. Li, Z. F. Liu, and Z. R. Liu, Phys. Chem. Chem. Phys. 19, 23942 (2017).
  * Xu  _et al._ (2016) L. C. Xu, A. J. Du, and L. Z. Kou, Phys. Chem. Chem. Phys. 18, 27284 (2016).
  * Penev  _et al._ (2016) E. S. Penev, A. Kutana, and B. I. Yakobson, Nano Lett. 16, 2522 (2016).
  * Xiao  _et al._ (2016) R. C. Xiao, D. F. Shao, W. J. Lu, H. Y. Lv, J. Y. Li, and Y. P. Sun, Appl. Phys. Lett. 109, 122604 (2016).
  * Scarpa (2008) F. Scarpa, IEEE Signal Process. Mag. 25, 128 (2008).
  * Sun and Pugno (2013) Y. T. Sun and N. Pugno, Materials 6, 699 (2013).
  * Prawoto (2012) Y. Prawoto, Comput. Mater. Sci. 58, 140 (2012).
  * Evans and Alderson (2000) K. E. Evans and A. Alderson, Adv. Mater. 12, 617 (2000).
  * Mak  _et al._ (2010) K. F. Mak, C. Lee, J. Hone, J. Shan, and T. F. Heinz, Phys. Rev. Lett. 105, 136805 (2010).
  * Zhang  _et al._ (2014) Y. Zhang, T. R. Chang, B. Zhou, Y. T. Cui, H. Yan, Z. K. Liu, F. Schmitt, J. Lee, R. Moore, Y. L. Chen, H. Lin, H. T. Jeng, S. K. Mo, Z. Hussain, A. Bansil, and Z. X. Shen, Nat. Nanotechnol. 9, 111 (2014).
  * Splendiani  _et al._ (2010) A. Splendiani, L. Sun, Y. Zhang, T. Li, J. Kim, C. Y. Chim, G. Galli, and F. Wang, Nano Lett. 10, 1271 (2010).
  * Blochl (1994) P. E. Blochl, Phys. Rev. B 50, 17953 (1994).
  * Kresse and Furthmuller (1996) G. Kresse and J. Furthmuller, Comput. Mater. Sci. 6, 15 (1996).
  * Kresse and Joubert (1999) G. Kresse and D. Joubert, Phys. Rev. B 59, 1758 (1999).
  * Klimes  _et al._ (2011) J. Klimes, D. R. Bowler, and A. Michaelides, Phys. Rev. B 83, 195131 (2011).
  * Dion  _et al._ (2004) M. Dion, H. Rydberg, E. Schroder, D. C. Langreth, and B. I. Lundqvist, Phys. Rev. Lett. 92, 246401 (2004).
  * Perdew  _et al._ (1996) J. P. Perdew, K. Burke, and M. Ernzerhof, Phys. Rev. Lett. 77, 3865 (1996).
  * Monkhorst and Pack (1976) H. J. Monkhorst and J. D. Pack, Phys. Rev. B 13, 5188 (1976).
  * Pack and Monkhorst (1977) J. D. Pack and H. J. Monkhorst, Phys. Rev. B 16, 1748 (1977).
  * Maintz  _et al._ (2016) S. Maintz, V. L. Deringer, A. L. Tchougreeff, and R. Dronskowski, J. Comput. Chem. 37, 1030 (2016).
  * Zhou and Huang (2008) J. Zhou and R. Huang, J. Phys. Chem. Solids 56, 1609 (2008).
  * Zhang  _et al._ (2015) S. H. Zhang, J. Zhou, Q. Wang, X. S. Chen, Y. Kawazoe, and P. Jena, Proc. Natl. Acad. Sci. U.S.A. 112, 2372 (2015).
  * Wang (2013) V. Wang, “Vaspkit, a post-processing program for the vasp code,”  (2013).
  * Wei and Peng (2014a) Q. Wei and X. Peng, Appl. Phys. Lett. 104, 251915 (2014a).
  * Wang  _et al._ (2016) H. F. Wang, Q. F. Li, Y. Gao, F. Miao, X. F. Zhou, and X. G. Wan, New J. Phys. 18, 073016 (2016).
  * Braunschweig and Dewhurst (2013) H. Braunschweig and R. D. Dewhurst, Angew. Chem. Int. Ed. 52, 3574 (2013).
  * Ohta  _et al._ (2006) T. Ohta, A. Bostwick, T. Seyller, K. Horn, and E. Rotenberg, Science 313, 951 (2006).
  * Padilha and Pontes (2015) J. E. Padilha and R. B. Pontes, J. Phys. Chem. C 119, 3818 (2015).
  * Dai and Zeng (2014) J. Dai and X. C. Zeng, J. Phys. Chem. Lett. 5, 1289 (2014).
  * Ramasubramaniam  _et al._ (2011) A. Ramasubramaniam, D. Naveh, and E. Towe, Phys. Rev. B 84, 205325 (2011).
  * Wei and Peng (2014b) Q. Wei and X. Peng, Appl. Phys. Lett. 104, 251915 (2014b).
  * Woo  _et al._ (2016) S. Woo, H. C. Park, and Y.-W. Son, Phys. Rev. B 93, 075420 (2016).
  * Wang and Geng (2017) V. Wang and W.-T. Geng, J. Phys. Chem. C 121, 10224 (2017).
  * Wang  _et al._ (2017) H. Wang, X. Li, P. Li, and J. Yang, Nanoscale 9, 850 (2017).
  * Han  _et al._ (2015) J. W. Han, J. F. Xie, Z. Y. Zhang, D. Z. Yang, M. S. Si, and D. S. Xue, Appl. Phys. Express 8, 041801 (2015).
  * Dobson  _et al._ (2006) J. F. Dobson, A. White, and A. Rubio, Phys. Rev. Lett. 96, 073201 (2006).