# Bilateral Reference for High-Resolution Dichotomous Image Segmentation

Peng Zheng1,2,5,7† Dehong Gao3 Deng-Ping Fan1∗ Li Liu4
Jorma Laaksonen5 Wanli Ouyang6 Nicu Sebe7
1 Nankai University, 2 Alibaba Group, 3 Northwest Polytechnical University,
4 National University of Defense Technology, 5 Aalto University,
6 Shanghai AI Laboratory, 7 University of Trento
<EMAIL_ADDRESS>
###### Abstract

We introduce a novel bilateral reference framework (BiRefNet) for high-
resolution dichotomous image segmentation (DIS). It comprises two essential
components: the localization module (LM) and the reconstruction module (RM)
with our proposed bilateral reference (BiRef). The LM aids in object
localization using global semantic information. Within the RM, we utilize
BiRef for the reconstruction process, where hierarchical patches of images
provide the source reference and gradient maps serve as the target reference.
These components collaborate to generate the final predicted maps. We also
introduce auxiliary gradient supervision to enhance the focus on regions with
finer details. Furthermore, we outline practical training strategies tailored
for DIS to improve map quality and training process. To validate the general
applicability of our approach, we conduct extensive experiments on four tasks
to evince that BiRefNet exhibits remarkable performance, outperforming task-
specific cutting-edge methods across all benchmarks. Our codes are available
at https://github.com/ZhengPeng7/BiRefNet.

\begin{overpic}[width=424.94574pt]{images/Fig1.pdf}
\put(25.5,10.5){{\color[rgb]{1,.5,0}GT}}
\put(37.85,10.5){{\color[rgb]{1,.5,0}Ours}}
\put(25.5,0.5){{\color[rgb]{1,.5,0}IS-Net}}
\put(37.85,0.5){{\color[rgb]{1,.5,0}UDUN}}
\par\put(75.5,10.5){{\color[rgb]{1,.5,0}GT}}
\put(87.85,10.5){{\color[rgb]{1,.5,0}Ours}}
\put(75.5,0.5){{\color[rgb]{1,.5,0}IS-Net}}
\put(87.85,0.5){{\color[rgb]{1,.5,0}UDUN}} \par\end{overpic}

Figure 1:  Visual comparison between the results of our proposed BiRefNet and
the latest state-of-the-art methods (_e.g._ , IS-Net [31] and UDUN [28]) for
high-resolution dichotomous image segmentation (DIS). Details of segmentation
are zoomed in for better display.

††$\dagger$ Peng finished the majority of this work when he was a visiting
scholar at Nankai University.††$\ast$ Corresponding author
(dengpfan@gmail.com).

##  1 Introduction

With the advancement in high-resolution image acquisition, image segmentation
technology has evolved from traditional coarse localization to achieving high-
precision object segmentation. This task, whether it involves salient [11] or
concealed object detection [10, 9], is referred to as high-resolution
dichotomous image segmentation (DIS) [31] and has attracted widespread
attention and use in the industry, _e.g._ , by Samsung, Adobe, and Disney.

For the new DIS task, recent works have considered strategies such as
intermediate supervision [31], frequency prior [48], and unite-divide-unite
[28], and have achieved favorable results. Essentially, they either split the
supervision [28, 31] at the feature-level or introduce an additional prior
[48] to enhance feature extraction. These strategies are, however, still
insufficient to capture very fine features (see Fig. 1). Based on our
observations, we found that weak features in image objects can be well
reflected by obtaining gradient features through derivative operations on the
original image. In addition, when certain positions exhibit high similarity in
color and texture to the background, the gradient features are probably too
weak. For such cases, we further introduce ground truth (GT) features for side
supervision, allowing the framework to learn the characteristics of these
positions. We name the incorporation of the image reference and the
introduction of both the gradient and GT references as _bilateral reference_.

\begin{overpic}[width=424.94574pt]{images/comp_other_methods.pdf}
\put(7.8,-1.5){(a)} \put(27.7,-1.5){(b)} \put(47.4,-1.5){(c)}
\put(71.6,-1.5){(d)} \par\put(2.4,4.5){Enc} \put(12.0,4.5){Dec}
\put(21.5,4.5){Enc} \put(31.2,4.5){Dec} \put(41.5,4.5){Enc}
\put(50.5,4.5){Dec} \put(66.0,4.5){Enc} \put(75.0,4.5){Dec} \end{overpic}
Figure 2: Comparison between our proposed BiRefNet and other existing methods
for HR segmentation tasks. (a) Common framework [32]; (b) Image pyramid as
input [12, 46]; (c) Scaled images as inward reference [17, 22]; (d) BiRefNet:
patches of original images at original scales as inward reference and gradient
priors as outward reference. Enc = encoder, Dec = decoder.

We propose a novel progressive bilateral reference network BiRefNet to handle
the high-resolution DIS task with separate localization and reconstruction
modules. For the reconstruction module, we further design the inward and
outward references as bilateral references (BiRef), in which the source image
and the gradient map are fed into the decoder at different stages. Instead of
resizing the original images to lower-resolution versions to ensure
consistency with decoding features at each stage [17, 22], we keep the
original resolution for intact detail features in inward reference and
adaptively crop them into patches for compatibility with decoding features. In
addition, we investigate and summarize practical strategies for the training
on high-resolution (HR) data, including long training and region-level loss
for better segmentation in parts of fine details, and multi-stage supervision
to accelerate learning in fine details.

Our main contributions can be summarized as follows:

  1. 1.

We present a bilateral reference network (BiRefNet) to perform high-quality
DIS, which comprises the localization module (LM) and the reconstruction
module (RM), to decompose DIS into two easier subtasks.

  2. 2.

We propose a bilateral reference module, which consists of an inward reference
with source image guidance and an outward reference with gradient supervision.
It shows great efficacy in the reconstruction of HR results.

  3. 3.

We explore and summarize various practical strategies tailored for DIS to
easily improve performance, prediction quality, and convergence acceleration.

  4. 4.

The proposed BiRefNet shows its excellent performance and strong
generalization capabilities to achieve state-of-the-art performance on not
only the DIS5K task but also on HRSOD and COD with 8.0%, 2.6%, and 7.4%
average $S_{m}$ improvements, respectively.

##  2 Related Work

###  2.1 High-Resolution Class-agnostic Segmentation

High-resolution class-agnostic segmentation has been a typical computer vision
objective for decades, and many related tasks have been proposed and attracted
much attention, such as dichotomous image segmentation (DIS) [31], high-
resolution salient object detection (HRSOD) [44], and concealed object
detection (COD) [9]. To provide HRSOD with standard benchmarks, several
typical HRSOD datasets (_e.g._ , HRSOD [44], UHRSD [40], HRS10K [5]) and
numerous approaches [17, 36, 44, 40] have been proposed. Zeng et al. [44]
employed a global-local fusion of the multi-scale input in their network. Xie
et al. [40] used cross-model grafting modules to process images at different
scales from multiple backbones (lightweight [13] and heavy [25]). Pyramid
blending was also used in [17] for a lower computational cost. Concealed
objects are difficult to locate due to similar-looking surrounding distractors
[8]. Therefore, image priors, such as frequency [47], boundary [34], gradient
[16],  _etc_ , are used as auxiliary guidance to train COD models.
Furthermore, a higher resolution has been found beneficial for detecting the
target [16, 15, 14]. To produce more precise and fine-detail segmentation
results, Yin et al. [42] employed progressive refinement with masked separable
attention. Li et al. [22] incorporated the original images at different scales
to aid in the refining process.

High-resolution DIS is a newly proposed task that focuses more on the complex
slender structure of target objects in high-resolution images, making it even
more challenging. Qin et al. [31] proposed the DIS5K dataset and IS-Net with
intermediate supervision to alleviate the loss of fine areas. In addition,
Zhou et al. [48] embedded a frequency prior to their DIS network to capture
more details. Pei et al. [28] applied a label decoupling strategy [39] to the
DIS task and achieved competitive segmentation performance in the boundary
areas of objects. Unlike previous models that used compressed/resized images
to enhance HR segmentation results, we utilized intact HR images as
supplementary information for better HR predictions.

###  2.2 Progressive Refinement in Segmentation

In the image matting task, trimaps have been used as a pre-positioning
technique for more precise segmentation results [21, 41]. In Fig. 2 we
illustrate different approaches of relevant networks and compare the
differences [32, 45, 31, 48, 28]. Many approaches have been proposed based on
the progressive refinement strategy. Yu et al. [43] used the predicted LR
alpha matte as a guide for refined HR maps. In BASNet [29], the initial
results are revised with an additional refiner network. The CRM [33]
continuously aligns the feature map with the refinement target to aggregate
detailed features. In ICNet [46], the original images are also downscaled and
added to the decoder output at different stages for refinement.

\begin{overpic}[width=424.94574pt]{images/LRNet.pdf}
\put(13.0,41.5){\normalsize{} {$\mathcal{I}$}} \put(74.0,41.5){\normalsize{}
{$\mathcal{M}$}} \par\put(24.2,36.0){\normalsize{} {$\mathcal{F}^{e}_{1}$}}
\put(24.2,26.5){\normalsize{} {$\mathcal{F}^{e}_{2}$}}
\put(24.2,16.5){\normalsize{} {$\mathcal{F}^{e}_{3}$}}
\put(24.2,7.55){\normalsize{} {$\mathcal{F}^{e}_{4}$}}
\put(51.8,38.2){\normalsize{} {$\mathcal{F}^{d}_{1}$}}
\put(51.8,28.4){\normalsize{} {$\mathcal{F}^{d}_{2}$}}
\put(51.8,19.0){\normalsize{} {$\mathcal{F}^{d}_{3}$}}
\put(51.8,8.6){\normalsize{} {$\mathcal{F}^{d}_{4}$}}
\par\put(33.8,6.5){\normalsize{} {$\mathcal{F}^{e}$}}
\put(41.5,6.5){\normalsize{} {$\mathcal{F}^{d}$}}
\par\put(53.0,41.3){\normalsize{} {$\mathcal{F}^{d+}_{1}$}}
\put(52.3,31.6){\normalsize{} {$\mathcal{F}^{d+}_{2}$}}
\put(52.3,21.9){\normalsize{} {$\mathcal{F}^{d+}_{3}$}}
\put(52.3,12.4){\normalsize{} {$\mathcal{F}^{d+}_{4}$}}
\par\put(41.0,41.5){\normalsize{} {$\mathcal{F}^{l}_{1}$}}
\put(41.0,32.2){\normalsize{} {$\mathcal{F}^{l}_{2}$}}
\put(41.0,22.5){\normalsize{} {$\mathcal{F}^{l}_{3}$}}
\put(41.0,12.5){\normalsize{} {$\mathcal{F}^{l}_{4}$}}
\par\put(68.5,33.5){\normalsize{} {$\mathcal{F}^{d}_{3}$}}
\put(62.5,26.0){\normalsize{} {$\mathcal{F}^{d^{\prime}}_{4}$}}
\put(62.5,9.0){\normalsize{} {$\mathcal{F}^{d+}_{4}$}}
\put(73.0,9.7){\normalsize{} {$\\{\mathcal{P}_{k=1}^{N}\\}$}}
\par\put(76.0,30.0){\footnotesize{$\mathcal{A}_{4}^{G}$}}
\put(87.0,17.0){\footnotesize{$\mathcal{M}_{3}$}}
\put(95.0,25.9){\footnotesize{$\hat{G}_{4}$}}
\par\put(47.0,0.9){\footnotesize{${L}_{\text{CE}}$}}
\put(92.0,37.3){\footnotesize{${L}_{\text{BCE, IoU}}$}}
\put(87.0,6.8){\footnotesize{${L}_{\text{BCE, IoU}}$}}
\put(94.0,6.8){\footnotesize{${L}_{\text{BCE}}$}} \end{overpic} Figure 3:
Pipeline of the proposed bilateral reference Network (BiRefNet). BiRefNet
mainly consists of the localization module (LM) and the reconstruction module
(RM) with bilateral reference (BiRef) blocks. Please refer to Sec. 3.1 for
details.

In addition to images and GT, auxiliary information is also used in existing
methods. For example, Tang et al. [35] cropped patches on the boundary to
further refine them. In the LapSRN network [19, 20] for image super-
resolution, Laplacian pyramids are also generated to help with image
reconstruction at higher resolution. Although these methods successfully
employed refinement to achieve better results, models are not guided to focus
on certain areas, which is a problem in DIS. Therefore, we introduce gradient
supervision in our outward reference to guide features sensitive to areas with
richer fine details.

##  3 Methodology

###  3.1 Overview

As shown in Fig. 2(d), our proposed BiRefNet is different from the previous
DIS methods. On the one hand, our BiRefNet explicitly decomposes the DIS task
on HR data into two modules,  _i.e._ , a localization module (LM) and a
reconstruction module (RM). On the other hand, instead of directly adding the
source images [12] or the priors [48] to the input, BiRefNet employs our
proposed bilateral reference in the RM, making full use of the source images
at the original scales and the gradient priors. The complete framework of our
BiRefNet is illustrated in Fig. 3.

###  3.2 Localization Module

For a batch of HR images $\mathcal{I}\in{}\mathbb{R}^{N\times 3\times H\times
W}$ as input, the transformer encoder [25] extracts features at different
stages,  _i.e._ ,
$\mathcal{F}_{1}^{e},\mathcal{F}_{2}^{e},\mathcal{F}_{3}^{e},\mathcal{F}_{4}^{e},\mathcal{F}^{e}$
with resolutions as $\\{[\frac{H}{k},\frac{W}{k}],k=4,4,8,16,32\\}$. The
features of the first four stages $\\{\mathcal{F}_{i}^{e}\\}_{i=1}^{4}$ are
transferred to the corresponding decoder stages with lateral connections
(1$\times$1 convolution layers). Meanwhile, they are stacked and concatenated
in the last encoder block to generate $\mathcal{F}^{e}$.

The encoder output feature $\mathcal{F}^{e}$ is then fed into a classification
module and an Atrous Spatial Pyramid Pooling (ASPP) module [2]. In the
classification module, the feature $\mathcal{F}^{e}$ is led into a global
average pooling layer and a fully connected layer for classification with the
category $C$ to obtain a better semantic representation for localization.
Inside the ASPP module, the feature $\mathcal{F}^{e}$ is squeezed to
$\mathcal{F}^{d}$ for transfer to the reconstruction module.

###  3.3 Reconstruction Module

\begin{overpic}[width=424.94574pt]{images/BiRef.pdf}
\put(4.0,4.0){\footnotesize{$\mathcal{I}$}}
\put(1.0,2.5){\footnotesize{1024$\times$1024}}
\put(6.0,11.0){\scriptsize{Adaptatively Crop}}
\put(9.5,5.0){\footnotesize{$\\{\mathcal{P}_{k=1}^{N}\\}$}}
\put(12.0,15.5){\footnotesize{$\mathcal{F}_{i}^{d+}$}}
\put(42.0,10.0){\footnotesize{$\mathcal{F}_{i}^{\theta}$}}
\put(49.5,9.0){\footnotesize{$\mathcal{F}_{i}^{d^{\prime}}$}}
\put(51.2,5.0){\footnotesize{$\mathcal{M}_{i}$}}
\put(75.3,6.9){\footnotesize{$G_{i}^{m}$}}
\put(74.2,3.0){\footnotesize{$G_{i}^{gt}$}}
\put(69.0,3.2){\footnotesize{$\hat{G}_{i}$}}
\put(63.0,2.5){\footnotesize{$\mathcal{F}_{i}^{G}$}}
\put(70.8,15.6){\footnotesize{$\mathcal{M}_{i}$}}
\put(55.0,15.6){\footnotesize{$\mathcal{A}_{i}^{G}$}}
\put(48.2,17.7){\footnotesize{$\mathcal{F}_{i-1}^{d}$}}
\put(6.5,0.7){\footnotesize{Inward Reference}}
\put(30.5,0.7){\footnotesize{Reconstruction Block}}
\put(62.0,0.7){\footnotesize{Outward Reference}} \end{overpic} Figure 4:
Pipeline of the proposed bilateral reference blocks. The source images at the
original scale are combined with decoder features as the inward reference and
fed into the reconstruction block, where deformable convolutions with
hierarchical receptive fields are employed. The aggregated features are then
used to predict the gradient maps in the outward reference. Gradient-aware
features are then turned into the attention map to act on the original
features.

The setting of the receptive field (RF) has been a challenge of HR
segmentation. Small RFs lead to inadequate context information to locate the
right target on a large background, whereas large RFs often result in
insufficient feature extraction in detailed areas. To achieve balance, we
propose the reconstruction block (RB) in each BiRef block as a replacement for
the vanilla residual blocks. In RB, we employ deformable convolutions [3] with
hierarchical receptive fields (_i.e._ , 1$\times$1, 3$\times$3, 7$\times$7)
and an adaptive average pooling layer to extract features with RFs of various
scales. These features extracted by different RFs are then concatenated as
$\mathcal{F}_{i}^{\theta}$, followed by a 1$\times$1 convolution layer and a
batch normalization layer to generate the output feature of RM
$\mathcal{F}_{i}^{d^{\prime}}$. In the reconstruction module, the squeezed
feature $\mathcal{F}^{d}$ is fed into the BiRef block for the feature
$\mathcal{F}_{4}^{d}$. With $\mathcal{F}_{4}^{l}$, the first BiRef block
predicts coarse maps, which are then reconstructed into higher-resolution
versions through the following BiRef blocks. Following [24], the output
feature of each BiRef block $\mathcal{F}_{i}^{d}$ is added with its lateral
feature $\mathcal{F}_{i}^{l}$ of the LM at each stage,  _i.e._ ,
$\\{\mathcal{F}_{i}^{d+}=\textit{Upsample}\uparrow{}(\mathcal{F}_{i}^{d}+\mathcal{F}_{i}^{l}),i=4,3,2,1\\}$.
Meanwhile, all BiRef blocks generate intermediate predictions
$\\{\mathcal{M}_{i}\\}_{i=4}^{1}$ by multi-stage supervision, with resolutions
in ascending order. Finally, the last decoding feature $\mathcal{F}_{1}^{d+}$
is passed through an 1$\times$1 convolution layer to obtain the final
predicted maps $\mathcal{M}\in{}\mathbb{R}^{N\times{}1\times{}H\times{}W}$.

###  3.4 Bilateral Reference

In DIS, HR training images are very important for deep models to learn details
and perform highly accurate segmentation. However, most segmentation models
follow previous works [32, 24] to design the network architecture in an
encoder-decoder structure with down-sampling and up-sampling, respectively.
Besides, due to the large size of the input, concentrating on the target
objects becomes more challenging. To deal with these two main problems, we
propose _bilateral reference_ , consisting of an inward reference (InRef) and
an outward reference (OutRef), which is illustrated in Fig. 4. Inward
reference and outward reference play the roles of supplementing HR information
and drawing attention to areas with dense details, respectively.

In InRef, images $\mathcal{I}$ with original high resolution are cropped to
patches $\\{\mathcal{P}_{k=1}^{N}\\}$ of consistent size with the output
features of the corresponding decoder stage. These patches are stacked with
the original feature $\mathcal{F}_{i}^{d+}$ to be fed into the RM. Existing
methods with similar techniques either add $\mathcal{I}$ only at the last
decoding stage [28] or resize $\mathcal{I}$ to make it applicable with
original features in low resolution. Our inward reference avoids these two
problems through adaptive cropping and supplies the necessary HR information
at every stage.

In OutRef, we use gradient labels to draw more attention to areas of richer
gradient information, which is essential for the segmentation of fine
structures. First, we extract the gradient maps of the input images as
$G_{i}^{gt}$. Meanwhile, $\mathcal{F}_{i}^{\theta}$ is used to generate the
feature $\mathcal{F}_{i}^{G}$ to produce the predicted gradient maps
$\hat{G}_{i}$. With this gradient supervision, $\mathcal{F}_{i}^{G}$ is
sensitive to the gradient. It passes through a conv and a sigmoid layer and is
used to generate the gradient referring attention $\mathcal{A}_{i}^{G}$, which
is then multiplied by $\mathcal{F}_{i}^{d^{\prime}}$ to generate the output of
the BiRef block as $\mathcal{F}_{i-1}^{d}$.

Considering that the background may have non-target noise with a lot of
gradient information, we apply a masking strategy to alleviate the influence
of non-target areas. We perform morphological operations on intermediate
predictions $\mathcal{M}_{i}$ and use dilated $\mathcal{M}_{i}$ as a mask. The
mask is used to multiply the gradient map $G_{i}^{gt}$ to generate
$G_{i}^{m}$, where the gradients outside the mask area are removed.

###  3.5 Objective Function

In HR segmentation tasks, using only pixel-level supervision (BCE loss)
usually results in deterioration of detailed structural information in HR
data. Inspired by the great results in [29] which used a hybrid loss, we use
BCE, IoU, SSIM, and CE losses together to collaborate for the supervision on
the levels of pixel, region, boundary, and semantic, respectively. The final
objective function is a weighted combination of them and can be formulated as:

$\begin{split}{L}&={L}_{\text{pixel}}+{L}_{\text{region}}+{L}_{\text{boundary}}+{L}_{\text{semantic}}\\\
&=\lambda_{1}{L}_{\text{BCE}}+\lambda_{2}{L}_{\text{IoU}}+\lambda_{3}{L}_{\text{SSIM}}+\lambda_{4}{L}_{\text{CE}}\;,\end{split}$
(1)

where $\lambda_{1},~{}\lambda_{2},~{}\lambda_{3},$ and $\lambda_{4}$ are
respectively set to 30, 0.5, 10, and 5 to keep all the losses on the same
quantitative level at the beginning of the training. The full definition of
each individual loss can be found in the supplementary material.

###  3.6 HR DIS Specific Training Strategies

Due to the high cost of training models on HR data, we have explored training
tricks for HR segmentation tasks to improve performance and reduce training
costs. First, we found that our model converges relatively quickly in the
localization of targets and the segmentation of rough structures (measured by
F-measure [1], S-measure [6]) on DIS5K (_e.g._ , 200 epochs). However, the
performance in segmenting fine parts is still increasing after very long
training (_e.g._ , 400 epochs), which is reflected in metrics such as
$F_{\beta}^{\omega}$ and $HCE_{\gamma}$. Second, though long training can
easily achieve great results in terms of both structure and edges, it consumes
too much computation; we found that multi-stage supervision can dramatically
accelerate the learning on segmenting fine details and make the model achieve
similar performance as before but with only 30% training epochs. Third, we
also found that fine-tuning with only region-level losses can easily improve
the binarization of predicted results and those metric scores (_e.g._ ,
$F_{\beta}^{\omega}$, $E_{\phi}^{m}$, $HCE$) that are closer to practical use.
Finally, we used context feature fusion and image pyramid inputs on the
backbone, which are commonly used tricks to use deep models to process HR
images. In our experiments, these two modifications to the backbone achieved a
general improvement in DIS and similar HR segmentation tasks. For more
details, please refer to our supplementary material.

##  4 Experiments

###  4.1 Datasets

Training Sets. For the DIS task, we follow previous works [31, 48, 28] to use
DIS5K-TR as our training set in experiments. For the HRSOD task, we follow the
latest work [40] to set different combinations of HRSOD, UHRSD, and DUTS as
the training set. For the COD task, we follow the latest works [9, 15] to use
the concealed samples in CAMO-TR and COD10K-TR as the training set.

Test Sets. To obtain a complete evaluation of our BiRefNet, we tested it on
all test sets in DIS5K (DIS-TE1, DIS-TE2, DIS-TE3, and DIS-TE4). We also
conducted an evaluation of BiRefNet on the HRSOD test sets (DAVIS-S, HRSOD-TE,
and UHRSD-TE) and the COD test sets (CAMO-TE, COD10K-TE, and NC4K). Low-
resolution SOD test sets (DUTS-TE and DUT-OMRON) are additionally used for
supplementary experiments.

###  4.2 Evaluation Protocol

For a comprehensive evaluation, we employ the S-measure [6] ($S_{m}$),
max/mean/weighted F-measure [1]
($F_{\beta}^{x}/F_{\beta}^{m}/F_{\beta}^{\omega}$), max/mean E-measure [7]
($E_{\xi}^{x}/E_{\xi}^{m}$), mean absolute error (MAE), and relax HCE [31]
($HCE_{\gamma}$) to evaluate performance. Detailed descriptions of these
metrics can be found in the supplementary material.

###  4.3 Implementation Details

All images are resized to 1024$\times$1024 for training and testing. The
generated segmentation maps are resized (_i.e._ , bilinear interpolation) to
the original size of the corresponding ground-truth maps for evaluation.
Horizontal flip is the only data augmentation used in the training process.
The number of categories $C$ is set to 219, as given in DIS-TR. The proposed
BiRefNet is trained with Adam optimizer [18] for DIS/HRSOD/COD tasks for
800/120/120 epochs, respectively. The model is fine-tuned with the IoU loss
for the last 20 epochs. The initial learning rate is set to $10^{-5}$, with
$\beta_{1}=0.9$ and $\beta_{2}=0.99$. All experiments are implemented with
PyTorch [27] and run on four NVIDIA A100 GPUs. Batch size is set to $N$=4 for
each GPU during training.

###  4.4 Ablation Study

Table 1: Quantitative ablation studies of the proposed components in the proposed BiRefNet. The ablation studies of the proposed BiRefNet are conducted on the effectiveness of the proposed components, including reconstruction module (RM), inward reference (InRef), outward reference (OutRef), and their combinations. Modules | DIS-VD
---|---
RM | InRef | OutRef | $F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$
|  |  | .837 | .785 | .056 | .845 | .887 | 1204
✓ |  |  | .855 | .831 | .048 | .865 | .895 | 1167
| ✓ |  | .848 | .825 | .050 | .857 | .903 | 1152
✓ | ✓ |  | .869 | .834 | .041 | .886 | .912 | 1093
✓ |  | ✓ | .863 | .831 | .042 | .891 | .918 | 1106
| ✓ | ✓ | .861 | .839 | .044 | .881 | .911 | 1114
✓ | ✓ | ✓ | .889 | .851 | .038 | .900 | .924 | 1065
Table 2: Effectiveness of practical strategies for training high-resolution segmentation. The experimental comparison of the proposed several tricks for HR segmentation tasks is provided here, including context feature fusion (CFF), image pyramids input (IPT), regional loss fine-tuning (RLFT), and their combinations. The results are obtained by our final model. Modules | DIS-VD
---|---
CFF | IPT | RLFT | $F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$
|  |  | .889 | .851 | .038 | .900 | .924 | 1065
✓ |  |  | .893 | .856 | .038 | .904 | .928 | 1054
| ✓ |  | .895 | .857 | .037 | .904 | .927 | 1051
|  | ✓ | .890 | .861 | .036 | .899 | .932 | 1043
✓ | ✓ | ✓ | .897 | .863 | .036 | .905 | .937 | 1039

We study the effectiveness of each component (_i.e._ , RM and BiRef) and
practical strategies (_i.e._ , CFF, IPT, and RLFT) introduced for our BiRefNet
and conduct an investigation about their contributions to improved DIS
results. Quantitative results regarding each module and strategy are shown in
Tab. 1 and 2, respectively.

\begin{overpic}[width=486.91699pt]{images/qual_dis.pdf} \put(7.0,1.0){Image} \put(22.5,1.0){GT} \put(36.0,1.0){{Ours}} \put(48.0,1.0){UDUN~{}\cite[cite]{[\@@bibref{Number}{UDUN}{}{}]}} \put(62.0,1.0){IS-Net~{}\cite[cite]{[\@@bibref{Number}{DIS5K}{}{}]}} \put(76.0,1.0){U${}^{2}$Net~{}\cite[cite]{[\@@bibref{Number}{U-2-Net}{}{}]}} \put(90.0,1.0){HRNet~{}\cite[cite]{[\@@bibref{Number}{HRNet}{}{}]}} \put(1.0,50.0){\rotatebox{90.0}{DIS-TE1}} \put(1.0,39.0){\rotatebox{90.0}{DIS-TE2}} \put(1.0,28.0){\rotatebox{90.0}{DIS-TE3}} \put(1.0,17.0){\rotatebox{90.0}{DIS-TE4}} \put(1.0,5.5){\rotatebox{90.0}{DIS-VD}} \end{overpic} Figure 5: Qualitative comparisons of the proposed BiRefNet and previous methods on the DIS5K benchmark. The results of the previous methods are from [28], where all models are trained with images with 1024$\times$1024 resolution. Zoom in for a better view. Table 3: Quantitative comparisons between our BiRefNet and the state-of-the-art methods on DIS5K validation and test sets. “$\uparrow$” (“$\downarrow$”) means that the higher (lower) is better. We use the results provided in [28], where all methods take 1024$\times$1024 input. Methods | DIS-TE1 (500) | DIS-TE2 (500) | DIS-TE3 (500)
---|---|---|---
$F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$ | $F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$ | $F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$
BASNet19 [29] | .663 | .577 | .105 | .741 | .756 | 155 | .738 | .653 | .096 | .781 | .808 | 341 | .790 | .714 | .080 | .816 | .848 | 681
U2Net20 [30] | .701 | .601 | .085 | .762 | .783 | 165 | .768 | .676 | .083 | .798 | .825 | 367 | .813 | .721 | .073 | .823 | .856 | 738
HRNet20 [37] | .668 | .579 | .088 | .742 | .797 | 262 | .747 | .664 | .087 | .784 | .840 | 555 | .784 | .700 | .080 | .805 | .869 | 1049
PGNet22 [40] | .754 | .680 | .067 | .800 | .848 | 162 | .807 | .743 | .065 | .833 | .880 | 375 | .843 | .785 | .056 | .844 | .911 | 797
IS-Net22 [31] | .740 | .662 | .074 | .787 | .820 | 149 | .799 | .728 | .070 | .823 | .858 | 340 | .830 | .758 | .064 | .836 | .883 | 687
FP-DIS23 [48] | .784 | .713 | .060 | .821 | .860 | 160 | .827 | .767 | .059 | .845 | .893 | 373 | .868 | .811 | .049 | .871 | .922 | 780
UDUN23 [28] | .784 | .720 | .059 | .817 | .864 | 140 | .829 | .768 | .058 | .843 | .886 | 325 | .865 | .809 | .050 | .865 | .917 | 658
BiRefNet (Ours) | .866 | .829 | .036 | .889 | .917 | 115 | .906 | .876 | .031 | .913 | .943 | 283 | .920 | .888 | .029 | .918 | .951 | 617
Methods | DIS-TE4 (500) | DIS-TE (1-4) (2,000) | DIS-VD (470)
$F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$ | $F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$ | $F_{\beta}^{x}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $HCE_{\gamma}\downarrow$
BASNet19 [29] | .785 | .713 | .087 | .806 | .844 | 2852 | .744 | .664 | .092 | .786 | .814 | 1007 | .737 | .656 | .094 | .781 | .809 | 1132
U2Net20 [30] | .800 | .707 | .085 | .814 | .837 | 2898 | .771 | .676 | .082 | .799 | .825 | 1042 | .753 | .656 | .089 | .785 | .809 | 1139
HRNet20 [37] | .772 | .687 | .092 | .792 | .854 | 3864 | .743 | .658 | .087 | .781 | .840 | 1432 | .726 | .641 | .095 | .767 | .824 | 1560
PGNet22 [40] | .831 | .774 | .065 | .841 | .899 | 3361 | .809 | .746 | .063 | .830 | .885 | 1173 | .798 | .733 | .067 | .824 | .879 | 1326
IS-Net22 [31] | .827 | .753 | .072 | .830 | .870 | 2888 | .799 | .726 | .070 | .819 | .858 | 1016 | .791 | .717 | .074 | .813 | .856 | 1116
FP-DIS23 [48] | .846 | .788 | .061 | .852 | .906 | 3347 | .831 | .770 | .047 | .847 | .895 | 1165 | .823 | .763 | .062 | .843 | .891 | 1309
UDUN23 [28] | .846 | .792 | .059 | .849 | .901 | 2785 | .831 | .772 | .057 | .844 | .892 | 977 | .823 | .763 | .059 | .838 | .892 | 1097
BiRefNet (Ours) | .906 | .866 | .038 | .902 | .940 | 2830 | .900 | .865 | .034 | .906 | .938 | 961 | .897 | .863 | .036 | .905 | .937 | 1039
Table 4: Quantitative comparisons between our BiRefNet and the state-of-the-art methods in high-resolution and low-resolution SOD datasets. TR denotes the training set. To provide a fair comparison with existing methods, we train our BiRefNet with different combinations of training sets, where D, H, and U represent DUTS [38], HRSOD [44], and UHRSD [40], respectively. Test Sets | High-Resolution Benchmarks | Low-Resolution Benchmarks
---|---|---
Methods | TR | DAVIS-S (92) | HRSOD-TE (400) | UHRSD-TE (988) | DUTS-TE (5,019) | DUT-OMRON (5,168)
$S_{m}\uparrow$ | $F_{\beta}^{x}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $F_{\beta}^{x}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $F_{\beta}^{x}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $F_{\beta}^{x}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $F_{\beta}^{x}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $\mathcal{M}\downarrow$
LDF20 [39] | D | .922 | .911 | .947 | .019 | .904 | .904 | .919 | .032 | .888 | .913 | .891 | .047 | .892 | .898 | .910 | .034 | .838 | .820 | .873 | .051
HRSOD19 [44] | D,H | .876 | .899 | .955 | .026 | .896 | .905 | .934 | .030 | - | - | - | - | .824 | .835 | .885 | .050 | .762 | .743 | .831 | .065
DHQ21 [36] | D,H | .920 | .938 | .947 | .012 | .920 | .922 | .947 | .022 | .900 | .911 | .905 | .039 | .894 | .900 | .919 | .031 | .836 | .820 | .873 | .045
PGNet22 [40] | D | .935 | .936 | .947 | .015 | .930 | .931 | .944 | .021 | .912 | .931 | .904 | .037 | .911 | .917 | .922 | .027 | .855 | .835 | .887 | .045
PGNet22 [40] | D,H | .948 | .950 | .975 | .012 | .935 | .937 | .946 | .020 | .912 | .935 | .905 | .036 | .912 | .919 | .925 | .028 | .858 | .835 | .887 | .046
PGNet22 [40] | H,U | .954 | .957 | .979 | .010 | .938 | .945 | .946 | .020 | .935 | .949 | .916 | .026 | .859 | .871 | .897 | .038 | .786 | .772 | .884 | .058
BiRefNet (Ours) | D | .946 | .937 | .972 | .012 | .943 | .934 | .960 | .021 | .922 | .928 | .940 | .035 | .922 | .910 | .946 | .025 | .860 | .810 | .884 | .046
BiRefNet (Ours) | D,H,U | .973 | .978 | .992 | .005 | .960 | .962 | .979 | .011 | .952 | .960 | .971 | .016 | .941 | .944 | .969 | .016 | .881 | .845 | .908 | .035
Table 5: Comparison of BiRefNet with the state-of-the-art methods. As seen, BiRefNet performs much better than previous methods. Methods | CAMO (250) | COD10K (2,026) | NC4K (4,121)
---|---|---|---
$S_{m}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $F_{\beta}^{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $E_{\phi}^{x}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $F_{\beta}^{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $E_{\phi}^{x}\uparrow$ | $\mathcal{M}\downarrow$ | $S_{m}\uparrow$ | $F_{\beta}^{\omega}\uparrow$ | $F_{\beta}^{m}\uparrow$ | $E_{\phi}^{m}\uparrow$ | $E_{\phi}^{x}\uparrow$ | $\mathcal{M}\downarrow$
SINet20 [8] | .751 | .606 | .675 | .771 | .831 | .100 | .771 | .551 | .634 | .806 | .868 | .051 | .808 | .723 | .769 | .871 | .883 | .058
BGNet22 [34] | .812 | .749 | .789 | .870 | .882 | .073 | .831 | .722 | .753 | .901 | .911 | .033 | .851 | .788 | .820 | .907 | .916 | .044
ZoomNet22 [26] | .820 | .752 | .794 | .878 | .892 | .066 | .838 | .729 | .766 | .888 | .911 | .029 | .853 | .784 | .818 | .896 | .912 | .043
SINetv222 [9] | .820 | .743 | .782 | .882 | .895 | .070 | .815 | .680 | .718 | .887 | .906 | .037 | .847 | .770 | .805 | .903 | .914 | .048
HitNet23 [14] | .849 | .809 | .831 | .906 | .910 | .055 | .871 | .806 | .823 | .935 | .938 | .023 | .875 | .834 | .853 | .926 | .929 | .037
FSPNet23 [15] | .856 | .799 | .830 | .899 | .928 | .050 | .851 | .735 | .769 | .895 | .930 | .026 | .879 | .816 | .843 | .915 | .937 | .035
BiRefNet (Ours) | .907 | .888 | .900 | .954 | .959 | .030 | .907 | .858 | .870 | .951 | .957 | .017 | .915 | .890 | .903 | .952 | .958 | .023

Baseline. We provide a simple but strong encoder-decoder network as the
baseline for the DIS task. To capture better hierarchical features on various
scales, we chose the Swin transformer [25] as our backbone network. Then, to
obtain a better semantic representation in the DIS task, we divided the images
in DIS-TR into 219 classes according to their label names and added an
auxiliary classification head at the end of the encoder. In the decoder of the
baseline network, each decoder block is made up of two residual blocks [13].
All stages of the encoder and decoder are connected with an 1$\times{}$1
convolution, except the deepest stage, where an ASPP [2] block is used for
connectivity. With this setup, our baseline network has outperformed existing
DIS models in most metrics, as shown in Tab. 1 and Tab. 3.

Reconstruction Module. As shown in Tab. 1, our model gains an overall
improvement with the proposed RM. The RM provides multi-scale receptive fields
on the HR features for local details and overall semantics. It brings
$\sim$2.2% $F_{\beta}^{x}$ relative improvement with little extra
computational cost.

Bilateral Reference. We separately investigate the effectiveness of the inward
reference (InRef, with source images) and the outward reference (OutRef, with
gradient labels) in BiRef. InRef supplemented lossless HR information
globally, while OutRef drew more attention to the fine-detail parts to achieve
higher precision in those areas. As shown in Tab. 1, they work jointly to
bring about 2.9% $F_{\beta}^{x}$ relative improvement to BiRefNet. Finally, RM
and BiRef are combined to achieve 6.2% $F_{\beta}^{x}$ relative improvement.

Training Strategies. As shown in Tab. 2, the proposed strategies improve
performance from different perspectives. CCF and IPT improve overall
performance, while RLFT specifically improves precision in edge details, which
is reflected in metrics such as $F_{\beta}^{\omega}$ and $HCE_{\gamma}$.

###  4.5 State-of-the-Art Comparison

To validate the general applicability of our method, we conduct extensive
experiments on four tasks,  _i.e._ , high-resolution dichotomous image
segmentation (DIS), high-resolution salient object detection (HRSOD),
concealed object detection (COD), and salient object detection (SOD). We
compare our proposed BiRefNet with all the latest task-specific models on
existing benchmarks.

Quantitative Results. Tab. 3 shows a quantitative comparison between the
proposed BiRefNet and previous state-of-the-art methods. Our BiRefNet
outperforms all previous methods in widely used metrics. The complexities of
DIS-TE1$\sim$DIS-TE4 are in ascending order. The metrics for structure
similarity (_e.g._ , $S_{\alpha}$, $E_{\phi}^{x}$) focus more on global
information. Pixel-level metrics, such as MAE ($M$), emphasize the precision
of details. Metrics based on mean values (_e.g._ , $E_{\phi}^{m}$,
$F_{\phi}^{m}$) better match the requirements of practical applications where
maps are thresholded. As seen in Tab. 3, our BiRefNet outperforms previous
methods not only on the accuracy of the global shape but also in the details
of the pixels. It is noteworthy that the results are better especially in
metrics that cater more to practical applications.

Additionally, our BiRefNet outperforms existing task-specific models on the
HRSOD and COD tasks. As shown in Tab. 4, BiRefNet achieved much higher
accuracy on both high-resolution and low-resolution SOD benchmarks. Compared
with the previous SOTA method [40], our BiRefNet achieved an average
improvement of 2.6% $S_{m}$. Furthermore, as shown in Tab. 5, in the COD task,
BiRefNet also shows a much better performance compared to the previous SOTA
models, with an average improvement of 7.4% $S_{m}$ on the three widely used
COD benchmarks. These results show the remarkable generalization ability of
our BiRefNet to extend to similar HR tasks.

For a clearer illustration of the generalizability and powerful performance of
BiRefNet, we provide a radar picture shown in the supplementary material,
where our model is compared with all the best models for different tasks.

\begin{overpic}[width=462.07266pt]{images/qual_cod.pdf}
\par\put(0.7,40.5){\rotatebox{90.0}{Tiny}}
\put(0.7,29.5){\rotatebox{90.0}{Slim}}
\put(0.7,16.5){\rotatebox{90.0}{Occluded}}
\put(0.7,6.0){\rotatebox{90.0}{Multiple}} \put(6.5,1.0){Input}
\put(21.5,1.0){GT} \put(35.0,1.0){{Ours}} \put(49.0,1.0){HitNet}
\put(62.0,1.0){FSPNet} \put(75.0,1.0){ZoomNet} \put(89.0,1.0){SINetv2}
\end{overpic} Figure 6: Visual comparisons of the proposed BiRefNet and other
competitors on COD10K benchmark. Samples with different challenges are
provided here to show the superiority of BiRefNet from different perspectives.

Qualitative Results. Fig. 5 shows segmentation maps produced by the most
competitive existing DIS models and the proposed BiRefNet. As the results
show, we provide samples of all test sets and one validation set. BiRefNet
outperforms the previous DIS methods from two perspectives,  _i.e._ , the
location of target objects and the more accurate segmentation of the details
of the objects. For example, in the samples of DIS-TE4 and DIS-TE2, there are
neighboring distractors that attract the attention of other models to produce
false positives. On the contrary, our BiRefNet eliminates the distractors and
accurately segments the target. In samples of DIS-TE3 and DIS-VD, BiRefNet
shows much greater performance in precisely segmenting areas where fine
details are rich. Compared with previous methods, our BiRefNet can clearly
segment slim shapes and curved edges better. The progress on these two fronts
shows the power of BiRefNet in the DIS task.

We also provide a qualitative comparison on the COD task. Fig. 6 shows hard
samples with different challenges. For example, in the row of the occluded
frog, the area of the frog is divided by the branch that covers it. Even
though, our BiRefNet can accurately segment the scattered fragments almost the
same as the GT map. In contrast, in the results of the other methods,
fragments are difficult to find all, let alone to provide precise segmentation
maps. For tiny and slim objects, BiRefNet shows better ability to find the
right target. In the sample with multiple objects, our BiRefNet shows
superiority in finding all concealed objects.

##  5 Potential Applications

We envisagte that generated fine segmentation maps have potential to be
utilized in various practical applications, _e.g._ :

Application #1 Crack Detection. The quality of walls is important for the
health of the architecture [49]. However, segmentation models trained on
commonly used datasets (_e.g._ , COCO [23]) can only segment regular
foreground objects. The proposed BiRefNet trained on the DIS5K dataset is more
aware of the fine details and can also segment targets with higher shape
complexities. As shown in Fig. 7, our BiRefNet can accurately find cracks in
the walls and help maintain when to repair them.

\begin{overpic}[width=208.67871pt]{images/app_crack.pdf} \end{overpic} Figure
7: Application #1. Crack detection for the maintenance of architecture health.
\begin{overpic}[width=208.67871pt]{images/app_bgRm.pdf} \end{overpic} Figure
8: Application #2. Highly accurate object extraction in high-resolution
natural images.

Application #2 Highly Accurate Object Extraction. Foreground object extraction
and background removal have been popular applications in recent years.
However, commonly seen methods fail to generate high-quality results when
target objects have too high shape complexities [30, 29] or need manual
guidance (_e.g._ , scribble, point, and coarse mask) for more accurate
segmentation [21, 4]. The proposed BiRefNet trained on DIS5K can generate
results with much higher resolution and segment thin threads at the hair level
without a mask, as shown in Fig. 8. On the basis of such refined results,
there may be numerous successful downstream applications in the future.

##  6 Conclusions

This work proposes a BiRefNet framework equipped with a bilateral reference,
which can perform dichotomous image segmentation, high-resolution (HR) salient
object detection, and concealed object detection in the same framework. With
the comprehensive experiments conducted, we find that unscaled source images
and a focus on regions of rich information are vital to generating fine and
detailed areas in HR images. To this end, we propose the bilateral reference
to fill in the missing information in the fine parts (inward reference) and
guide the model to focus more on regions with richer details (outward
reference). This significantly improves the model’s ability to capture tiny-
pixel features. To alleviate the high training cost of HR data training, we
also provide various practical tricks to deliver higher-quality prediction and
faster convergence. Competitive results on 13 benchmarks demonstrate
outstanding performance and strong generalization ability of our BiRefNet. We
also show that the techniques of BiRefNet can be transferred and used in many
practical applications. We hope that the proposed framework can encourage the
development of unified models for various tasks in the future.

## References

  * Achanta et al. [2009] Radhakrishna Achanta, Sheila Hemami, Francisco Estrada, and Sabine Susstrunk.  Frequency-tuned salient region detection.  In _CVPR_ , 2009.
  * Chen et al. [2018] Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam.  Encoder-decoder with atrous separable convolution for semantic image segmentation.  In _ECCV_ , 2018.
  * Dai et al. [2017] Jifeng Dai, Haozhi Qi, Yuwen Xiong, Yi Li, Guodong Zhang, Han Hu, and Yichen Wei.  Deformable convolutional networks.  In _ICCV_ , 2017.
  * Dai et al. [2022] Linhui Dai, Xiang Song, Xiaohong Liu, Chengqi Li, Zhihao Shi, Martin Brooks, and Jun Chen.  Enabling trimap-free image matting with a frequency-guided saliency-aware network via joint learning.  _IEEE TMM_ , 25:4868–4879, 2022.
  * Deng et al. [2023] Xinhao Deng, Pingping Zhang, Wei Liu, and Huchuan Lu.  Recurrent multi-scale transformer for high-resolution salient object detection.  In _ACM MM_ , 2023.
  * Fan et al. [2017] Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, and Ali Borji.  Structure-measure: A new way to evaluate foreground maps.  In _ICCV_ , 2017.
  * Fan et al. [2018] Deng-Ping Fan, Cheng Gong, Yang Cao, Bo Ren, Ming-Ming Cheng, and Ali Borji.  Enhanced-alignment measure for binary foreground map evaluation.  In _IJCAI_ , 2018.
  * Fan et al. [2020] Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, Ming-Ming Cheng, Jianbing Shen, and Ling Shao.  Camouflaged object detection.  In _CVPR_ , 2020.
  * Fan et al. [2022] Deng-Ping Fan, Ge-Peng Ji, Ming-Ming Cheng, and Ling Shao.  Concealed object detection.  _IEEE TPAMI_ , 44(10):6024–6042, 2022.
  * Fan et al. [2023a] Deng-Ping Fan, Ge-Peng Ji, Peng Xu, Ming-Ming Cheng, Christos Sakaridis, and Luc Van Gool.  Advances in deep concealed scene understanding.  _VI_ , 1(1):16, 2023a.
  * Fan et al. [2023b] Deng-Ping Fan, Jing Zhang, Gang Xu, Ming-Ming Cheng, and Ling Shao.  Salient objects in clutter.  _IEEE TPAMI_ , 45(2):2344–2366, 2023b.
  * Grosky and Jain [1986] William I Grosky and Ramesh Jain.  A pyramid-based approach to segmentation applied to region matching.  _IEEE TPAMI_ , 8(5):639–650, 1986.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In _CVPR_ , 2016.
  * Hu et al. [2023] Xiaobin Hu, Shuo Wang, Xuebin Qin, Hang Dai, Wenqi Ren, Donghao Luo, Ying Tai, and Ling Shao.  High-resolution iterative feedback network for camouflaged object detection.  In _AAAI_ , 2023.
  * Huang et al. [2023] Zhou Huang, Hang Dai, Tian-Zhu Xiang, Shuo Wang, Huai-Xin Chen, Jie Qin, and Huan Xiong.  Feature shrinkage pyramid for camouflaged object detection with transformers.  In _CVPR_ , 2023.
  * Ji et al. [2023] Ge-Peng Ji, Deng-Ping Fan, Yu-Cheng Chou, Dengxin Dai, Alexander Liniger, and Luc Van Gool.  Deep gradient learning for efficient camouflaged object detection.  _MIR_ , 20(1):92–108, 2023.
  * Kim et al. [2022] Taehun Kim, Kunhee Kim, Joonyeong Lee, Dongmin Cha, Jiho Lee, and Daijin Kim.  Revisiting image pyramid structure for high resolution salient object detection.  In _ACCV_ , 2022.
  * Kingma and Ba [2015] Diederik P. Kingma and Jimmy Ba.  Adam: A method for stochastic optimization.  In _ICLR_ , 2015.
  * Lai et al. [2017] Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang.  Deep laplacian pyramid networks for fast and accurate super-resolution.  In _CVPR_ , 2017.
  * Lai et al. [2018] Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang.  Fast and accurate image super-resolution with deep laplacian pyramid networks.  _IEEE TPAMI_ , 41(11):2599–2613, 2018.
  * Li et al. [2022] Jizhizi Li, Jing Zhang, Stephen J Maybank, and Dacheng Tao.  Bridging composite and real: towards end-to-end deep image matting.  _IJCV_ , 130(2):246–266, 2022.
  * Li et al. [2023] Xiaofei Li, Jiaxin Yang, Shuohao Li, Jun Lei, Jun Zhang, and Dong Chen.  Locate, refine and restore: A progressive enhancement network for camouflaged object detection.  In _IJCAI_ , 2023.
  * Lin et al. [2014] Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick.  Microsoft coco: Common objects in context.  In _ECCV_ , 2014.
  * Lin et al. [2017] Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie.  Feature pyramid networks for object detection.  In _CVPR_ , 2017.
  * Liu et al. [2021] Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo.  Swin transformer: Hierarchical vision transformer using shifted windows.  In _ICCV_ , 2021.
  * Pang et al. [2022] Youwei Pang, Xiaoqi Zhao, Tian-Zhu Xiang, Lihe Zhang, and Huchuan Lu.  Zoom in and out: A mixed-scale triplet network for camouflaged object detection.  In _CVPR_ , 2022.
  * Paszke et al. [2019] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  PyTorch: An imperative style, high-performance deep learning library.  _NeurIPS_ , 2019.
  * Pei et al. [2023] Jialun Pei, Zhangjun Zhou, Yueming Jin, He Tang, and Heng Pheng-Ann.  Unite-divide-unite: Joint boosting trunk and structure for high-accuracy dichotomous image segmentation.  In _ACM MM_ , 2023.
  * Qin et al. [2019] Xuebin Qin, Zichen Zhang, Chenyang Huang, Chao Gao, Masood Dehghan, and Martin Jagersand.  Basnet: Boundary-aware salient object detection.  In _CVPR_ , 2019.
  * Qin et al. [2020] Xuebin Qin, Zichen Zhang, Chenyang Huang, Masood Dehghan, Osmar R Zaiane, and Martin Jagersand.  U2-net: Going deeper with nested u-structure for salient object detection.  _PR_ , 106:107404, 2020.
  * Qin et al. [2022] Xuebin Qin, Hang Dai, Xiaobin Hu, Deng-Ping Fan, Ling Shao, et al.  Highly accurate dichotomous image segmentation.  In _ECCV_ , 2022.
  * Ronneberger et al. [2015] Olaf Ronneberger, Philipp Fischer, and Thomas Brox.  U-net: Convolutional networks for biomedical image segmentation.  In _MICCAI_ , 2015.
  * Shen et al. [2022] Tiancheng Shen, Yuechen Zhang, Lu Qi, Jason Kuen, Xingyu Xie, Jianlong Wu, Zhe Lin, and Jiaya Jia.  High quality segmentation for ultra high-resolution images.  In _CVPR_ , 2022.
  * Sun et al. [2022] Yujia Sun, Shuo Wang, Chenglizhao Chen, and Tian-Zhu Xiang.  Boundary-guided camouflaged object detection.  In _IJCAI_ , 2022.
  * Tang et al. [2021a] Chufeng Tang, Hang Chen, Xiao Li, Jianmin Li, Zhaoxiang Zhang, and Xiaolin Hu.  Look closer to segment better: Boundary patch refinement for instance segmentation.  In _CVPR_ , 2021a.
  * Tang et al. [2021b] Lv Tang, Bo Li, Yijie Zhong, Shouhong Ding, and Mofei Song.  Disentangled high quality salient object detection.  In _CVPR_ , 2021b.
  * Wang et al. [2020] Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, et al.  Deep high-resolution representation learning for visual recognition.  _IEEE TPAMI_ , 43(10):3349–3364, 2020.
  * Wang et al. [2017] Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan.  Learning to detect salient objects with image-level supervision.  In _CVPR_ , 2017.
  * Wei et al. [2020] Jun Wei, Shuhui Wang, Zhe Wu, Chi Su, Qingming Huang, and Qi Tian.  Label decoupling framework for salient object detection.  In _CVPR_ , 2020.
  * Xie et al. [2022] Chenxi Xie, Changqun Xia, Mingcan Ma, Zhirui Zhao, Xiaowu Chen, and Jia Li.  Pyramid grafting network for one-stage high resolution saliency detection.  In _CVPR_ , 2022.
  * Xu et al. [2017] Ning Xu, Brian Price, Scott Cohen, and Thomas Huang.  Deep image matting.  In _CVPR_ , 2017.
  * Yin et al. [2022] Bowen Yin, Xuying Zhang, Qibin Hou, Bo-Yuan Sun, Deng-Ping Fan, and Luc Van Gool.  Camoformer: Masked separable attention for camouflaged object detection.  _arXiv_ , 2022.
  * Yu et al. [2021] Qihang Yu, Jianming Zhang, He Zhang, Yilin Wang, Zhe Lin, Ning Xu, Yutong Bai, and Alan Yuille.  Mask guided matting via progressive refinement network.  In _CVPR_ , 2021.
  * Zeng et al. [2019] Yi Zeng, Pingping Zhang, Jianming Zhang, Zhe Lin, and Huchuan Lu.  Towards high-resolution salient object detection.  In _CVPR_ , 2019.
  * Zhao et al. [2017] Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia.  Pyramid scene parsing network.  In _CVPR_ , 2017.
  * Zhao et al. [2018] Hengshuang Zhao, Xiaojuan Qi, Xiaoyong Shen, Jianping Shi, and Jiaya Jia.  Icnet for real-time semantic segmentation on high-resolution images.  In _ECCV_ , 2018.
  * Zhong et al. [2022] Yijie Zhong, Bo Li, Lv Tang, Senyun Kuang, Shuang Wu, and Shouhong Ding.  Detecting camouflaged object in frequency domain.  In _CVPR_ , 2022.
  * Zhou et al. [2023] Yan Zhou, Bo Dong, Yuanfeng Wu, Wentao Zhu, Geng Chen, and Yanning Zhang.  Dichotomous image segmentation with frequency priors.  In _IJCAI_ , 2023.
  * Zou et al. [2018] Qin Zou, Zheng Zhang, Qingquan Li, Xianbiao Qi, Qian Wang, and Song Wang.  Deepcrack: Learning hierarchical convolutional features for crack detection.  _IEEE TIP_ , 28(3):1498–1512, 2018.