A Study on the Generality of Neural Network Structures for Monocular
  Depth Estimation
Monocular depth estimation has been widely studied, and significant
improvements in performance have been recently reported. However, most previous
works are evaluated on a few benchmark datasets, such as KITTI datasets, and
none of the works provide an in-depth analysis of the generalization
performance of monocular depth estimation. In this paper, we deeply investigate
the various backbone networks (e.g.CNN and Transformer models) toward the
generalization of monocular depth estimation. First, we evaluate
state-of-the-art models on both in-distribution and out-of-distribution
datasets, which have never been seen during network training. Then, we
investigate the internal properties of the representations from the
intermediate layers of CNN-/Transformer-based models using synthetic
texture-shifted datasets. Through extensive experiments, we observe that the
Transformers exhibit a strong shape-bias rather than CNNs, which have a strong
texture-bias. We also discover that texture-biased models exhibit worse
generalization performance for monocular depth estimation than shape-biased
models. We demonstrate that similar aspects are observed in real-world driving
datasets captured under diverse environments. Lastly, we conduct a dense
ablation study with various backbone networks which are utilized in modern
strategies. The experiments demonstrate that the intrinsic locality of the CNNs
and the self-attention of the Transformers induce texture-bias and shape-bias,
respectively.
Jinwoo Bae, Kyumin Hwang, Sunghoon Im
# A Study on the Generality of Neural Network Structures for Monocular Depth Estimation

###### Abstract

Monocular depth estimation has been widely studied, and significant improvements in performance have been recently reported. However, most previous works are evaluated on a few benchmark datasets, such as KITTI datasets, and none of the works provide an in-depth analysis of the generalization performance of monocular depth estimation. In this paper, we deeply investigate the various backbone networks (_e.g_CNN and Transformer models) toward the generalization of monocular depth estimation. First, we evaluate state-of-the-art models on both in-distribution and out-of-distribution datasets, which have never been seen during network training. Then, we investigate the internal properties of the representations from the intermediate layers of CNN-/Transformer-based models using synthetic texture-shifted datasets. Through extensive experiments, we observe that the Transformers exhibit a strong shape-bias rather than CNNs, which have a strong texture-bias. We also discover that texture-biased models exhibit worse generalization performance for monocular depth estimation than shape-biased models. We demonstrate that similar aspects are observed in real-world driving datasets captured under diverse environments. Lastly, we conduct a dense ablation study with various backbone networks which are utilized in modern strategies. The experiments demonstrate that the intrinsic locality of the CNNs and the self-attention of the Transformers induce texture-bias and shape-bias, respectively.

 Monocular depth estimation, Out-of-Distribution, Generalization, Transformer

## 1 Introduction

Monocular depth estimation (MDE) is widely utilized for spatial recognition technologies such as autonomous driving [1, 2, 3] or AR/VR [4, 5] because of its portability and cost-effectiveness. Various MDE processes have achieved remarkable progress over the past decade. Most previous works [6, 7, 8, 9, 10, 11, 12, 13] concentrate on boosting performance using limited benchmark datasets, especially the KITTI dataset [14]. However, these works have not provided a deep investigation into what MDE networks have learned. This means that they cannot guarantee the model's behavior is correct. To examine the interpretability of MDE networks, one previous work [15] employs a target network for MDE model analysis. Another approach [16] uses a synthetic dataset, which contains the changes in image contents (e.g., camera pose). However, universality questions about other networks still remain because of the fixed specific network [6] in certain experimental conditions.

Recently, several works [17, 18, 19, 20] aim to analyze interpretability, taking inspiration from convolutional neural networks (CNNs) whose designs are based on human visual processes [21, 22]. Then, how can humans efficiently extract and recognize important information from complex scenes? Compared to other cues like texture or color, the biological vision system considers the object's shape to be the single most important visual cue [23]. This allows humans, including little kids, to easily distinguish an object from a line drawing or a silhouette image. Many researchers believe that CNN would also behave similarly [24, 25, 26]. However, in contrast to belief, recent studies [27, 28, 18, 20] have discovered that CNNs are heavily predisposed to recognize textures rather than shapes. CNN-based models accurately assign labels to images even when the shapes of the structures are disturbed [29, 30]. On the other hand, CNN models are unable to accurately predict labels in a texture-removed image with a well-preserved shape [31]. Transformers [32] achieve outstanding performance on various computer vision tasks [33, 34, 35] and have attracted much attention. Moreover, many works [36, 37, 27, 38] reveal that Transformers have a strong shape

Fig. 1: **Analysis on the generality of state-of-the-art models and modernized network structures. The \(x\)-axis is the CKA similarity indicating whether the network is shape biased or texture biased. The \(y\)-axis shows Absolute Relative error where a lower number is better performance. We use the synthetic texture-shifted datasets described in Sec. 4.5.**
bias notwithstanding the lack of a spatial locality, compared to CNNs. Due to its strong shape bias, Transformer is considered more robust than a CNN and more similar to human cognitive processes [39].

Then, how does this observation affect the MDE? We hypothesize two things. First, the network's generality will differ depending on the texture-/shape-bias. To verify the generality, we evaluate state-of-the-art MDE models trained on KITTI [14] using five public depth datasets (SUN3D [40], RGBD [41], MVS [42], Scenes11 [42], and ETH3D [43]). We also conduct experiments on six different texture-shifted datasets, including three synthetic texture-shifted datasets (Watercolor, Pencil-sketch, Style-transfer) and three real-world texture-shifted datasets (Oxford Robotcar [44], Rainy Cityscapes [45], Foggy Cityscapes [46]). Through these experiments, we confirm that texture-bias is vulnerable to generalization while shape-bias shows robust generalization performance. Second, the texture-/shape-bias are related to the intrinsic properties of CNN and Transformer structures. The modernized Transformer-based model [47, 48] imitates the intrinsic locality inductive bias of the CNNs. The modernized CNN-based model [49, 50] is designed to mimic the self-attention of the Transformer. We finally conduct ablation studies on the generalization performance, and the texture-/shape-bias of various modernized backbone structures that originate from a specific design for each CNN and Transformer (_e.g._, locality and self-attention) such as RegionViT [47], Twins [48], ConvNeXt [49] and SLAK [50].

This paper extends our previous work in [51] which proposes a new self-supervised monocular depth estimation network adopting Transformers [52]. The Transformer-based network shows the generalization performance on various environments on depth estimation. While the short version [51] only addresses self-supervised MDE models, the extended version deals with full MDE models, including both self-supervised and supervised methods. Moreover, we deeply analyze the reason why the proposed network achieves better-generalized performance rather than CNN-based models by comparing the performance, and intermediate-layer feature similarity [53, 54] on various texture-shifted datasets. As shown in Fig. 1, we observe that the Transformer structure has more shape-biased properties than the CNN structure, which has texture-biased properties. It enables the Transformer-based models to achieve better generalization performance than the CNN-based models. We also experiment extensively on modern backbone architectures (_e.g._, ConvNeXt [49], RegionViT [47]) to investigate the origin of the texture-/shape-biased properties. Through these extensive experiments, we observe that the intrinsic locality of CNNs induces texture-biased characteristics, while the self-attention mechanism, the base layer of Transformers, induces shape-biased properties.

## 2 Related Work

### _Self-supervised monocular depth estimation_

Self-supervised depth estimation methods [55, 56, 57, 9, 12, 56, 11] simultaneously train depth and motion network by imposing photometric consistency loss between the target and source images warped by the predicted depth and motion. SfMLearner [55] first proposes a depth and ego-motion estimation pipeline without the ground truth depth and motion. Monodepth2 [8] presents a minimum reprojection loss to handle occlusions, a full-resolution multi-scale sampling method to reduce visual artifacts, and an auto-masking loss to ignore outlier pixels. PackNet-SfM [9] introduces packing and unpacking blocks that leveraged 3D convolutions to learn the dense appearance and geometric information in real-time. HR-Depth [12] analyzes the reason for the inaccurate depth prediction in large gradient regions and designed a skip connection to extract representative features in high resolution.

### _Supervised monocular depth estimation_

Supervised methods [58, 59, 10, 13] use a ground truth depth acquired from RGB-D cameras or LiDAR sensors for supervision in training. They also estimate depth maps given a single image as input. BTS [58] adopts a local planar guidance layer to densely encoded features to preserve local detail and create depth map sharpness at multi-stages in the decoder. AdaBins [10] estimate the depth by linear combinations of bin centers that are adaptively decided per image. The bin building block divides the depth range of the image into bins. LapDepth [59] employs a Laplacian pyramid at the multi-level upscaling encoder to preserve local detail, such as a boundary. It also trains stably by utilizing weight standardization. GLPDepth [60] proposes a hierarchical transformer encoder to capture the global context of images and a selective feature fusion module to connect multi-scale local features and global context information at the decoder. The feature fusion module helps the decoder become more powerful, even if the decoder is lightweight. DepthFormer [13] proposes leveraging the transformer's effective attention mechanism and the spatial inductive bias of the CNN to capture long-range correlation. It also uses a hierarchical aggregation and heterogeneous interaction module to enhance the affinity of the network.

### _Vision Transformers_

Recently, Transformers [52] has shown promise for solving computer vision tasks such as image classification [32, 61], object detection [33], and dense prediction. [62, 63, 11, 64]. ViT [32] employs a Transformers architecture on fixed-size image patches for image classification for the first time. DeiT [61] utilizes Knowledge distinction on ViT architecture, showing good performance only with the ImageNet dataset. DETR [33] proposes the direct set prediction approach, which simplifies the object detection pipeline, based on a CNN-Transformer network and bipartite matching. Some works [11, 63] have employed Transformers for monocular depth estimation in a supervised manner. DPT [63] introduces a dense prediction using a Transformer as the basic computational building block of the encoder. These works show generalized performance, but they require a large number of training datasets captured in diverse environments with ground truth depth maps. TransDepth [11] utilizes multi-scale information to capture local level details. Previous works lack studies on whether models behave as intended on another domain dataset. The work [65] aggregates the attention map between a single frame
and cross frames to refine the attraction map to improve performance. The works [11, 65] only focus on improving performance on benchmark datasets.

### _Modernized Architecture_

Despite the tremendous success of Transformers for vision tasks, a Transformer requires a large model and data size to achieve state-of-the-art performance. Recently, many works [47, 48, 49, 50, 66] utilize local attention on the Transformer to alleviate problems. The Swin Transformer [66] designs a pyramid structure network differently from a vision transformer [32], which is an isotropic structure. It achieves state-of-the-art performance in classification tasks with local attention using the sliding window strategy. Twins [48] employs global attention for the non-overlapping region and local attention to perform better under limited conditions. RegionViT [47] shows state-of-the-art performance on several visual tasks using regional-to-local attention, which alleviates the weakness of the standard attention mechanism. ConvNeXt [49] modernizes convolution neural layers such as depth-wise convolution and utilizes techniques such as Patchify stem and achieves competitive performance with the Transformer. SLaK [50] attempts to design the network with an extremely large kernel size (_e.g._, \(51\times 51\)) to leverage the sparsity that is observed from the human visual system. SLaK [50] repeats the Prune-and-Grow step in training to optimize the sparse kernels.

## 3 Method

This section describes MonoFormer, which is an encoder-decoder structure with a multi-level feature fusion module for self-supervised monocular depth estimation described in Sec. 3.1. An Attention Connection Module (ACM) learns the channel and position attentions in Sec. 3.2. A Feature Fusion Decoder (FFD) adaptively fuses the encoder features with the attention maps in Fig. 2.

### _Transformer-based Encoder_

MonoFormer [51] is composed of a CNN and Transformer for an image encoder. The encoder employs ResNet50 [67] as the CNN backbone (\(E(\theta)\) in Fig. 2), and \(L\) number of Transformers. MonoFormer sets the \(L\) to 4. The encoder is used to extract a feature map \(F\in\mathbb{R}^{C\times H\times W}\) from an input image \(I\), and the map is divided into \(N=(\frac{H}{16}\times\frac{W}{16})\) number of patches \(p_{n}\in\mathbb{R}^{C\times 16\times 16}\), which is utilized as the input of the first Transformer layer. Following the work [63], MonoFormer additionally use a special token \(t_{s}\). MonoFormer input the patch tokens \(p_{n},\ n\in\{1,...,N\}\) and the special token \(t_{s}\) with a learnable linear projection layer \(E\) as follows:

\[Z_{0}=[t_{s};\ p_{1}E;\ p_{2}E;\...\ ;\ p_{N}E], \tag{1}\]

where \(Z_{0}\) is the latent embedding vector. The CNN-Transformer encoder comprises a Multi-head Self-Attention (MSA) layer, a Multi-Layer Perceptron (MLP) layer, and Layer Norm (LN) layers. The MLP is built with GELU non-linearity [68]. The LN is used before each block, and residual connections are used after every block. Self-Attention (SA) at each layer \(I\in\{1,...,L\}\) is processed with the learnable parameters \(W^{m}_{Q},W^{m}_{K},W^{m}_{V}\in\mathbb{R}^{C\times d}\) of {query, key, value} weight matrices, given the embedding vector \(Z_{l}\in\mathbb{R}^{N\times C}\) as follows:

\[\begin{split}\text{SA}^{m}_{l-1}=\text{softmax}(\frac{Q^{m}_{l-1 }(K^{m}_{l-1})^{\text{T}}}{\sqrt{d}})V^{m}_{l-1},\ m\in\{1,...,M\},\\ \text{Q}^{m}_{l-1}=Z_{l-1}W^{m}_{Q},\ K^{m}_{l-1}=Z_{l-1}W^{m}_{K},\ V^{m}_{l-1}=Z_{l-1}W^{m}_{V},\end{split} \tag{2}\]

where \(M\) and \(d\) are the number of SA blocks and the dimension of the self-attention block, which is the same as the dimension of the weight matrices, respectively. The Multi-head Self-Attention (MSA) consists of the \(M\) number of SA blocks with the learnable parameters of weight matrices \(W\in\mathbb{R}^{Md\times C}\) as follows:

\[\begin{split}\text{MSA}_{l-1}=Z_{l-1}+\text{concat}(\text{SA}^{1} _{l-1};\ \text{SA}^{2}_{l-1};\ \dots;\ \text{SA}^{M}_{l-1})W,\\ Z_{l}=\text{MLP}(\text{LN}(\text{MSA}_{l-1}))+\text{MSA}_{l-1}. \end{split} \tag{3}\]

Fig. 2: **Overall Architecture of MonoFormer (MF-ours).**
This Transformer layer is repeated \(L\) times with unique learnable parameters. The outputs of the Transformers \(\{Z_{1},...,Z_{L}\}\) are utilized as the input of the following layers ACM and FFD. The hybrid encoder can be replaced with the other backbones. We evaluate all the performance of our depth estimation networks with various backbones by changing the encoder to the modernized backbone structures, such as ViT [32], RegionViT [47], Twins [48], ConvNeXt [49] and SLaK [50] as shown in Fig. 3.

### _Attention Connection Module (ACM)_

The skip connection module of MonoFormer, ACM, that extracts global context attention and a semantic presentation from the given features \(Z_{l},\ l\in\{1,...,L\}\). The skip connection, widely utilized for dense prediction tasks [69], helps keep the fine detail by directly transferring the spatial information from the encoder to the decoder. However, because of its simplicity, it is challenging for the naive skip connection method to preserve local detail like object boundaries [70]. To address the issue, The ACM is proposed that extracts attention weight from the spatial and channel domains inspired by [71] It consists of position attention, channel attention modules, and a fusion block that gathers important information from two attentions. The position attention module produces a position attention map \(A^{p}_{l}\in\mathbb{R}^{C\times N}\) as follows:

\[A^{p}_{l}=\text{softmax}(Q^{p}_{l}(K^{p}_{l})^{\text{T}})V^{p}_{l}, \tag{4}\]

where \(Q^{p}_{l}\), \(K^{p}_{l}\) and \(V^{p}_{l}\) are the query, key, and value matrices computed by passing \(Z_{l}\) through a single convolutional layer. The channel attention module directly calculate the channel attention map \(A^{c}_{l}\in\mathbb{R}^{C\times N}\) by computing the gram matrix of \(Z_{l}\) as follows:

\[A^{c}_{l}=\text{softmax}(Z_{l}Z^{\text{T}}_{l}). \tag{5}\]

The position attention map \(A^{p}_{l}\) and channel attention map \(A^{c}_{l}\) enhance the feature representation by capturing long-range context and exploiting the inter-dependencies between each channel map, respectively. These two attention maps are utilized in the following section, which highlights the importance of the features.

### _Feature Fusion Decoder (FFD)_

The FFD gets the encoder features \(Z_{l}\), the attention maps \(A^{p}_{l},\ A^{c}_{l}\), and the output feature \(X_{L}\) of the last Transformer layer passed through a Residual convolutional layer. The features \(X_{L-l+1},\ l\in\{1,...,L\}\) are fused through the decoder with a single Convolutional layer (Conv) and Channel Normalization (CN) with learnable parameters \(\alpha,\beta\) and \(\gamma\) as follows:

\[\begin{split} X_{L-l}=\hat{X}_{L-l}[1+\tanh(\gamma(\text{CN}( \alpha||\hat{X}_{L-l}||_{2}+\beta)],\\ \hat{X}_{L-l}=\text{Conv}(w_{p}A^{p}_{l}Z_{l}+w_{c}A^{c}_{l}Z_{l }+Z_{l})+X_{L-l+1},\end{split} \tag{6}\]

where \(w_{p}\) and \(w_{c}\) are the learnable parameters that determine the importance of the position and channel attentions [72]. The parameter \(\alpha\) works so that each channel can learn about each other individually, and \(\gamma\) and \(\beta\) control the activation channel-wisely following the work in [73]. Through this process, the FFD can assemble the local detailed semantic representation and the global context from the fused features to produce the fine depth map.

### _Training loss and implementation detail_

We train both depth and motion networks using photometric consistency (L2 loss and SSIM loss) and edge-aware smoothness losses following the best practices of self-supervised monocular depth estimation [55, 9, 8]. We set the weight for SSIM, L2 photometric, and smoothness losses as \(0.85\), \(0.15\) and \(0.001\), respectively. We use 7 convolution layers for 6-DoT camera pose estimation following the work in [55]. We implement our framework on PyTorch and train it on 4 Titan RTX GPUs. We use the Adam optimizer [74] with \(\beta_{1}=0.9\) and \(\beta_{2}=0.999\). Our model is trained for 50 epochs with a batch size of 8. The learning rates for depth and pose network are \(2\times 10^{-5}\) and \(5\times 10^{-4}\), respectively. We will release the source code, the trained weights and the datasets once the paper is accepted.

## 4 Experimental Results

We conduct extensive experiments to investigate the generalization performance of various network structures and the effect of texture-/shape-bias for monocular depth estimation. First, we evaluate state-of-the-art KITTI-trained models, including various modernized backbone architectures on the KITTI dataset and various depth benchmark datasets. Evaluations are conducted on the KITTI, an in-distribution dataset in Sec. 4.2, and the other depth datasets, out-of-distribution datasets in Sec. 4.4. We also conduct an ablation study on the MonoFormer in Sec. 4.3. Then, we investigate the texture-bias and shape-bias of the MDE models

Fig. 3: **Illustrations of various modernized architectures.**
using both the self-supervised and supervised methods with texture-shifted datasets in Sec. 4.5. We analyze the depth estimation performance and the feature representation of each model on the texture-shifted datasets, which are synthetically generated. Lastly, we demonstrate the intrinsic properties of the CNN-based and Transformer-based networks are observed in the real-world texture-shifted datasets captured from different driving environments, such as the weather changes and illumination changes in Sec. 4.6.

### _Competitive methods and evaluation setups_

We conduct extensive experiments to compare the performances of the monocular depth estimation methods. For self-supervised setting, we compare our work, called MF-Ours [51], with state-of-the-art methods, Monodepth2 [8], PackNet-SfM [9], SGDepth [56], R-MSFM [75]. We also evaluate state-of-the-art supervised methods, BTS [58], AdaBins [10], TransDepth [11], DepthFormer [13], and GLPDepth [60]. We note that some Transformer-based models, such as MF-Ours, TransDepth, and DepthFormer, use both CNNs and Transformers for their encoders. We also analyze the performance of various modernized architectures such as RegionViT [47], Twins [48], ConvNeXt [49] and SLaK [50]. We replace the encoder of MF-Ours with these modernized backbones, whose names are defined as MF-(backbone). Its taxonomy is described in Tab. I. We illustrate these basic network structures of the modernized CNN and Transformer architectures in Fig. 3. eTransformer-based models [47, 48] add the locality by using local attention to compensate for the shortcomings of the ViT (_e.g._, necessity of large dataset). CNN-based architectures [49, 50] aim to extract global information while utilizing the intrinsic locality inductive bias. They also imitate the self-attention of the Transformer using improved strategies such as large kernel size, patchify stem [49], and Layer Norm [76].

We train all networks using the KITTI Eigen split [14, 77] consisting of 39,810 training and 4,424 validation data whose size is \(640\times 192\). All experiments in this paper are conducted with this KITTI-trained model whose results are reported in Sec. 4.2-4.6. Following a previous work [9], we remove around 5% of the total data for training to address the infinite-depth problems that commonly occur in dynamic scenes. We use typical error and accuracy metrics for depth, absolute relative (Abs Rel), square relative (Sq Rel), root-mean-square-error (RMSE), its log (RMSElog), and the ratio of inliers following the work [9].

### _Evaluation on the in-distribution dataset_

We conduct the evaluations with an in-distribution dataset, where the KITTI-trained models are evaluated on 697 KITTI test data. The qualitative results in Fig. 4 show that our self-supervised method precisely preserves object boundaries. This indicates that the encoder captures global context and informative local features and transfers them to the decoder

\begin{table}
\begin{tabular}{l l l} \hline \hline Models & Supervision & Methods \\ \hline CNN-based & Self-supervised & Monodepth2 [8], PackNet-SfM [9] \\  & R-MSFM [75], MF-ConvNeXt [49], SLaK [50] \\ \hline \multirow{2}{*}{_Transformer-based_} & Supervised & BTS [58], AdaBins [10] \\ \cline{2-3}  & Self-supervised & MF-ViT [32], RegionViT [47], Twins [48], Ours [51] \\ \hline Supervised & TransDepth [11], DepthFormer [13], GLPDepth [60] \\ \hline \hline \end{tabular}
\end{table} TABLE I: **Taxonomy of MDE methods w.r.t backbones.**

\begin{table}
\begin{tabular}{l l l l l l l l l} \hline \hline Method & Train & Model & Mean-square & Mean-square & Mean-square & Mean-square & Mean-square & Mean-square \\ \hline Monodepth2 & M & CNN & 0.115 & 0.204 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.205 & 0.401 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.205 & 0.401 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.208 & 0.401 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.206 & 0.401 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.206 & 0.401 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.206 & 0.400 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.206 & 0.400 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.206 & 0.400 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & CNN & 0.111 & 0.206 & 0.400 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & TransDepth & 0.111 & 0.206 & 0.400 & 0.100 & 0.000 & 0.000 & 0.000 \\ PathNet & M & TransDepth & 0.110 & 0.300 & 0.400 & 0.100 & 0.000 & 0.000 & 0.000 \\ \hline \hline BTS & D & CNN & 0.200 & 0.200 & 0.200 & 0.200 & 0.500 & 0.500 & 0.500 & 0.500 \\ Ash Ashuna & D & CNN & 0.050 & 0.100 & 2.300 & 0.000 & 0.000 & 0.000 & 0.000 & 0.000 \\ PathNet & D & TransDepth & 0.400 & 0.200 & 0.200 & 0.000 & 0.000 & 0.000 & 0.000 \\ PathNet & D & TransDepth & 0.400 & 0.400 & 0.100 & 0.100 & 0.000 & 0.000 & 0.000 \\ \hline \hline \end{tabular}
\end{table} TABLE II: **Quantitative results on the in-distribution dataset.** M is Monocular images, and D is GT depth. Trans means Transformer. **Bold** is the best performance.

Fig. 4: **Depth map results on the in-distribution dataset. MonoFormer (MF-Ours) is from our previous work [51].**
for pixel-wise prediction. The quantitative results in Tab. II show that MF-Ours outperforms all competitive methods. We also evaluate the performance of models with the other backbones, MF-(ConvNeXt/SLaK/RegionViT/Twins/ViT). We employ only the encoder part of our method as a backbone without changing other parts in this experiment. Supervised methods outperform all self-supervised methods regardless of backbones. MF-Ours and DepthFormer achieve the best performance among the self-supervised methods and supervised methods on the in-distribution datasets, respectively. Overall, the performance gap among all the competitive methods is marginal for the in-distribution dataset, but the Transformer-based models generally outperform the CNN-based models.

### _Ablation study_

In this section, we conduct the ablation study on MF-Ours proposed in our previous work [51]. We use the same models and datasets for the experiments used in Sec. 4.2.

#### 4.3.1 Effectiveness of the proposed modules.

We conduct an ablation study to demonstrate the effectiveness of the proposed modules, ACM and FFD in Tab. III. The baseline is DPT [63]. The models with only the ACM module or FFD module marginally improve the depth estimation performance due to the absence of proper attention map fusions. On the other hand, our MonoFormer with both ACM and FFD significantly improves the performance. The results show the proposed model achieves the best performance in all measurements. The qualitative comparison in Fig. 5 shows that the model with both ACM and FFD keeps clearer object boundaries, even a small car in far depth.

#### 4.3.2 Visualization of attention maps.

We visualize the attention maps from the lower to higher layers of Transformers. As shown in Fig. 6, the encoder in the shallow layer extracts local region features. The deeper the layer, the more global shape contexts are extracted. Another observation is that ACM captures more detailed attention at different depths of the encoder features. FFD enhances the encoder features by fusing them with the attention map from ACM. The fused feature captures features from coarse to fine details. These experiments show that our model is capable of accurate pixel-wise prediction as it secures adequate local details.

#### 4.3.3 The number of encoder and decoder layers.

We compare the performance of MF-Ours according to the number of encoder and decoder layers in Tab. IV. Each layer has ACM and FFD modules. We find out that the model with four transformer layers achieves the best performance. The results are slightly degraded with the MF-Ours with 3 or 5 layers. Therefore, we set \(L\) as four for MF-Ours in all experiments in this paper.

### _Evaluation on out-of-distribution datasets_

We compare the generalization performance of all the competitive models using public depth datasets captured at common indoor environments, including office workspaces, meeting rooms, and kitchen areas (RGBD [78], SUN3D [40]), man-made indoor and outdoor environments (MVS [42], ETH3D [43]), and synthetic scenes from graphics tools (Scenes11 [42]). Interestingly, both qualitative and quantitative results in Fig. 7 and Tab. 5-7 show that Transformer-based models better generalization performance even in the out-of-distribution datasets that have never been seen during network training. Meanwhile, CNN-based models predict unreliable depth maps that have lost the detail of object boundaries and produce significant errors in texture-less regions, such as the wall of a building. The experiments demonstrate that Transformers are more robust than CNNs for environmental changes. Another interesting observation is that MF-ConvNeXt generally outperforms all the other CNN-based models and produces depth results comparable to other Transformer-based models. On the other hand, MF-RegionViT fails to estimate depth accurately, even though Transformer-based. A detailed analysis of why Transformers and MF-ConvNeXt show better generalization performance is provided in the following section.

\begin{table}
\begin{tabular}{c c c c c} \hline \# of layers & Abs Rel \(\downarrow\) & RMSE \(\downarrow\) & RMSE\(\downarrow\) & \(\delta<1.25\uparrow\) \\ \hline \(L=2\) & 0.148 & 5.327 & 0.810 & 0.939 \\ \(L=3\) & 0.112 & 4.745 & 0.881 & **0.962** \\ \(L=4\) & **0.104** & **4.580** & 0.873 & **0.962** \\ \(L=5\) & 0.111 & 4.692 & **0.884** & **0.962** \\ \hline \end{tabular}
\end{table} TABLE IV: **Ablation study on the number of layers.**

Fig. 5: **Results of MF-Ours with/without ACM and FFD.**

\begin{table}
\begin{tabular}{c c c c c} \hline  & Abs Rel \(\downarrow\) & Seq Rel \(\downarrow\) & RMSE\(\downarrow\) & \(\delta<1.25\uparrow\) \\ \hline baseline & 0.113 & 0.899 & 4.783 & 0.189 & 0.882 \\ +ACM & 0.113 & 0.879 & 4.820 & 0.189 & 0.879 \\ +FP & 0.112 & 0.860 & 4.803 & 0.186 & 0.879 \\ +ACM\&FFD & **0.104** & **0.846** & **4.580** & **0.183** & **0.891** \\ \hline \end{tabular}
\end{table} TABLE III: **Ablation study on ACM and FFD.**

Fig. 6: **Visualization of attention and feature maps. The left column from the second row is the maps from shallow layers, whereas the right is the maps from deep layers.**
[MISSING_PAGE_POST]
### _Analysis of texture-/shape-bias on state-of-the-art methods and various backbone networks_

In this section, we verify the intrinsic properties of CNNs and Transformers that lead to the robustness of depth estimation to environmental changes. We hypothesize that CNNs and transformers identify texture and shape information as key visual cues for depth estimation, respectively, inspired by the work [20, 28]. Thus, we synthetically generate the texture-shifted datasets in Sec. 4.5.1. Then, we validate the texture-/shape-bias of the model by comparing the performance of the competitive methods on the generated datasets in Sec. 4.5.2. Finally, we analyze the internal representation of each neural network structure by measuring centered kernel alignment (CKA) in Sec. 4.5.3.

#### 4.5.1 Texture-shifted datasets generation

In general, the texture is defined as an image's spatial color or pixel intensity pattern [79, 80, 81]. Inspired by [18], we use three different texture shift strategies to investigate the impact of textures on the inference process in depth: texture smoothing (Watercolor), texture removal (Pencil-sketch), and texture transfer (Style-transfer). The generated images and the corresponding results of each model are shown in Fig. 8. The first two images are watercolored images, the middle two images and the last two images are pencil-sketch and style-transfered images, respectively. The following is a summary of the image generation:

\begin{tabular}{l l} _Watercolor_ & We smooth the texture details from original images while preserving the color cues using cv2. stylization. The image looks like a watercolor pictures. \\ _Pencil-sketch_ & We remove both textures and color from original images using cv2.pencilSketch. The image seems like a sketch drawn with pencils. \\ _Style-transfer_ & We apply a new texture to the original image (context) by utilizing other images (style) using a style transfer [82]. The textures of the original images are changed. \\ \end{tabular}

#### 4.5.2 Evaluation on synthetic texture-shifted datasets

We compare the performance of all the competitive methods and the modernized backbones on the synthetic texture-shifted datasets, the same as the experiments in Sec. 4.4. The qualitative and quantitative results are shown in Fig. 8 and Tab. 8, respectively. As seen in Sec. 4.4, both the Transformer-based models produce better depth maps than pure CNN-based methods regardless of supervised or self-supervised methods. In particular, the depth results from CNN-based models are unrecognizable with the strong texture-shifted datasets, especially the style-transfered data. We also observe that MF-ConvNeXt shows a tolerable depth map on texture shift datasets and has lower errors than other CNN-based models, although it is purely CNN-based. These experiments support our two findings observed in Sec. 4.4. One is that networks whose encoder consists of Transformers are generally robust to texture changes. However, MF-ConvNeXt and MF-RegionViT show different aspects from the CNN-based and Transformer-based models, which are the respective backbone models. In the following section, we deeply analyze the intermediate feature representations of all backbones to verify the reason for these observations.

\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{Datasets} & Train & Backbone & \multicolumn{5}{c}{Multi-task 50,184} & \multicolumn{5}{c}{Multi-task 50,184} & \multicolumn{5}{c}{Multi-task 50,184} & \multicolumn{5}{c}{Multi-task 50,184} & \multicolumn{5}{c}{Multi-task 50,184} \\ \hline \hline KnowledgePh2 & M & CNN & 15GP & **0.763** & 0.532 & 0.532 & 0.552 & 0.567 \\ PackNeXt & M & CNN & 2.068 & 0.870 & 0.320 & 0.310 & 0.500 & 0.674 \\ RASHp & M & CNN & 1.272 & 0.276 & 0.341 & 0.280 & 0.494 & 0.846 \\ RASHp & M & CNN & 1.827 & 0.639 & 0.284 & 0.350 & 0.506 & 0.718 \\ RASHp & M & CNN & 1.837 & 0.320 & 0.295 & 0.505 & 0.800 \\ RASHp & M & CNN & 1.846 & 0.240 & 0.263 & 0.263 & 0.263 & 0.263 \\ \hline \hline \end{tabular} 
\begin{tabular}{l l} _Style-transfer_ & We apply a new texture to the original image (context) by utilizing other images (style) using a style transfer [82]. The textures of the original images are changed. \\ \end{tabular}
\end{table} TABLE
\begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{Datasets} & \multicolumn{8}{c}{Datasets} & \multicolumn{8}{c}{Datasets} & \multicolumn{8}{c}{Datasets} & \multicolumn{8}{c}{Datasets} \\ \cline{2-13}  & Abs Rel & Sq Rel & RMSE \(\delta\) \(<\) 1.25 & \(\delta\) \(<\) 1.25\({}^{\circ}\) & \(\delta\) \(<\) 1.25\({}^{\circ}\) & Abs Rel & Sq Rel & RMSE \(\delta\) \(<\) 1.25 & \(\delta\) \(<\) 1.25\({}^{\circ}\) & Abs Rel & Sq Rel & RMSE \(\delta\) \(<\) 1.25 & \(\delta\) \(<\) 1.25\({}^{\circ}\) & \(\delta\) \(<\) 1.25\({}^{\circ}\) \\ \hline Monodepft2 & 0.170 & 1.345 & 6.175 & 0.750 & 0.900 & 0.960 & 0.196 & 1.522 & 6.232 & 0.691 & 0.898 & 0.962 & 0.247 & 2.440 & 7.825 & 0.647 & 0.828 & 0.920 \\ PacikNet-SM & 0.174 & 1.346 & 6.343 & 0.742 & 0.906 & 0.961 & 0.204 & 1.569 & 5.686 & 0.837 & 0.888 & 0.888 & 0.897 & 0.205 & 1.608 & 6.778 & 0.678 & 0.948 \\ R-MSFM6 & 0.194 & 1.613 & 7.173 & 0.696 & 0.876 & 0.943 & 0.217 & 6.098 & 6.719 & 0.647 & 0.872 & 0.951 & 0.294 & 2.788 & 8.599 & 0.521 & 0.770 & 0.894 \\ MF-ConvNet & 0.195 & 1.175 & 5.957 & 0.774 & 0.921 & 0.969 & 0.176 & 1.280 & 5.682 & 0.728 & 0.947 & 0.965 & 0.219 & 1.722 & 7.241 & 0.634 & 0.861 & 0.942 \\ MF-SLK & 0.170 & 1.272 & 6.316 & 0.755 & 0.907 & 0.961 & 0.268 & 2.208 & 0.842 & 0.533 & 0.811 & 0.915 & 0.242 & 1.889 & 7.423 & 0.648 & 0.855 & 0.934 \\ \hline MF-ViT & 0.152 & 1.196 & 5.686 & 0.799 & 0.852 & 0.973 & 0.774 & 1.311 & 5.707 & 0.756 & 0.920 & 0.987 & 0.816 & 1.359 & 0.623 & 0.758 & 0.959 \\ MF-RegionViT & 0.179 & 1.341 & 6.309 & 0.732 & 0.902 & 0.998 & 0.260 & 2.021 & 7.941 & 0.954 & 0.832 & 0.939 & 0.422 & 0.704 & 0.947 & 0.885 & 0.927 \\ MF-Twins & 0.166 & 1.295 & 6.585 & 0.751 & 0.911 & 0.963 & 0.205 & 1.630 & 7.512 & 0.655 & 0.871 & 0.949 & 0.232 & 1.974 & 0.845 & 0.599 & 0.839 & 0.931 \\ MF-ConvNet & **0.140** & **1.205** & **5.465** & **0.815** & **0.986** & **0.975** & **0.151** & **1.284** & **5.175** & **0.756** & **0.934** & **0.976** & **0.275** & **1.370** & **4.145** & **0.728** & **0.906** & **0.982** \\ \hline BTS & 0.225 & 4.960 & 5.039 & 0.599 & 0.753 & 0.844 & 0.247 & 2.180 & 7.965 & 0.503 & 0.745 & 0.871 & 0.824 & 3.233 & 9.300 & 0.512 & 0.674 & 0.770 \\ AdaIns & 0.301 & 2.993 & 8.761 & 0.415 & 0.642 & 0.782 & 0.165 & 1.133 & 8.512 & 0.721 & 0.889 & 0.957 & 0.317 & 3.500 & 0.944 & 0.416 & 0.626 & 0.751 \\ \hline TianDepth & 0.111 & 0.728 & 4.866 & 0.835 & 0.905 & 0.955 & 0.136 & 0.987 & 0.724 & 0.775 & 0.926 & 0.975 & 0.146 & 1.127 & 5.970 & 0.761 & 0.911 & 0.964 \\ DepthFormer & 0.135 & 1.102 & 5.853 & 0.783 & 0.905 & 0.955 & 0.162 & 1.290 & 6.498 & 0.713 & 0.885 & 0.953 & 0.274 & 2.766 & 3.566 & 0.501 & 0.691 & 0.801 \\ GLFDDepth & **0.099** & **0.573** & **4.212** & **0.872** & **0.860** & **0.986** & **0.092** & **0.813** & **3.950** & **0.874** & **0.970** & **0.992** & **0.099** & **0.513** & **3.950** & **0.874** & **0.970** & **0.992** \\ \hline \hline \end{tabular}
\end{table} TABLE VIII: **Quantitative results on synthetic texture-shifted (Watercolor, Pencil-sketch, Style-transfer) datasets.**

Fig. 8: **Depth map results on synthetic texture-shifted (Watercolor, Pencil-sketch, Style-transfer) datasets.**
#### 4.5.3 Analysis on feature representation from backbones

To analyze the internal properties of CNNs and Transformers, we employ the centered kernel alignment (CKA), which is the similarity measure of internal representations of neural networks by following previous works [53, 54, 83]. The CKA is widely used to analyze feature representations of neural networks [53, 84], because of its invariant properties to the orthogonal transformation of representations and isotropic scaling [54]. We freeze the encoder \(E\) of the depth networks trained on the original KITTI dataset. We extract features \(z_{a},z_{b}\) from the last layer of the encoder by passing the original image \(\mathbf{I}_{a}\) and texture-shifted image \(\mathbf{I}_{b}\), respectively. Then, the CKA is computed given \(K=z_{a}z_{a}^{T}\) and \(L=z_{b}z_{b}^{T}\) with \(m\) number of samples as follows:

\[CKA(K,L)=\frac{HSIC(K,L)}{\sqrt{HSIC(K,K)HSIC(L,L)}}, \tag{7}\] \[HSIC(K,L)=(KHLH)/(m-1)^{2}, \tag{8}\]

where \(H\) is the centering matrix \(H_{n}=I_{n}-\frac{1}{n}\mathbf{1}^{T}\). The overall process is illustrated in Fig. 10. We use 697 image pairs from the KITTI Eigen dataset and their corresponding texture-shifted datasets. We compute the CKA with three types of texture-shifted datasets, including watercolor, pencil-sketch, and style-transfer, as shown in Fig. 9.

The results of feature similarity show a similar aspect of the quantitative results on synthetic texture-shifted datasets in Tab. VIII. MF-Ours and GLPDepth extract the most similar features from the original and texture-shifted datasets in the self-supervised and supervised methods, respectively. Generally, Transformer-based models (MF-ViT, MF-Twins, GLPDepth, MF-Ours, TransDepth, DepthFormer) extract features of texture-shifted images similar to the original images. On the other hand, most CNN-based models (Monodepth2, PackNet-SfM, R-MSFM6, BTS, AdaBins) extract inconsistent features from original and texture-shifted images. The results are consistently observed regardless of the types of texture changes. It demonstrates that the robustness to environmental changes comes from consistent feature extraction from encoders. These results also support our observations in Sec. 4.5.2 that the Transformers have a strong shape-bias whereas CNNs have a strong texture bias. Moreover, models with shape-biased representation show better generalization performance than models with texture bias for monocular depth estimation. We also observe that the pure Transformer-based methods (MF-ViT, MF-Twins, GLPDepth) and CNN-Transformer hybrid methods (MF-Ours, TransDepth, DepthFormer) show similar properties. The experiments provide us with some interesting observations on modernized network structures.

First, it is noteworthy that MF-ConvNeXt, one of the CNN-based models, shows a relatively higher CKA similarity than the other CNN-based models. Even the similarity of the models is on par with Transformer-based models. ConvNeXt consists of a patchify stem, and a depth-wise convolution [85] with a large receptive field that mimics the self-attention mechanism using Transformer's macro design [49]. The design intuition of ConvNeXt makes that ConvNeXt can focus on global information, similar to self-attention. Because of the modernized macro design, MF-ConvNeXt consisting of only convolutional layers is robust to texture-shifted datasets and has a strong shape-bias.

Second, unlike other Transformer-based models, MF-RegionViT is sensitive to texture changes, similar to CNN-based models. It is highly biased toward texture information rather than shape information, even though it is a transformer-based model. RegionViT [47] employs region-to-local attention, which is getting interaction between local regions. It improves the locality of the networks through local attention by capturing fine-grained spatial information. We believe that the strong locality makes MF-RegionViT rely heavily on spatial information.

Based on these experiments, we find that a self-attention module capturing global information, strengthen the shape-bias of the network. We also observe that the locality from local attention or the intrinsic property of CNNs improves the texture-bias of the network. In terms of network design,

Fig. 10: Illustration of CKA [53] similarity computation.

Fig. 9: **Box-and-Whisker plot of CKA similarity of all competitive methods on three different synthetic texture-shifted datasets.** The blue and red tones indicate CNN-based and Transformer-based models, respectively. The circle and triangle indicate self-supervised and supervised methods, respectively.
[MISSING_PAGE_POST]
a self-attention mechanism is ultimately key to generalization. Additionally, the locality of the network weakens generality, even if it helps to optimize in the training process.

### _Evaluation on real-world texture-shifted datasets_

In addition, we conduct experiments on practical texture changes that can occur in real driving scenarios to demonstrate their applicability to the real-world. We evaluate all models using practical texture-shifted datasets consisting of the night (Oxford RobotCar [44]), foggy (Foggy CityScapes [46]), and rainy driving scenes (Rainy CityScapes [45]). Overall, the experiment demonstrates that the properties of CNNs and Transformers identified in Sec. 4.5 are similarly observed in real-world texture-shifted scenarios. Quantitative and qualitative results and detailed analyzes for each network are described in the following paragraphs.

The depth map results of each model are shown in Fig. 11. Similar to the experiments in Sec. 4.5.2, Transformer-based models show plausible depth maps, except MF-RegionViT. On the other hand, CNN-based models fail to estimate depth even for objects such as a car in scenes similar to the training dataset, with the exception of MF-ConvNeXt. The quantitative results in Tab. IX also show that CNN-based models have higher errors than Transformer-based models. MF-SLaK is no different from other CNN models, but MF-ConvNeXt shows low errors and feasible depth maps by mimicking Transformers, as shown in Fig. 11. MF-Twins show similar results to other Transformer-based models because only a weak locality is added compared to MF-RegionViT. However, MF-RegionViT indicates texture-biased results like CNN-based models due to its strong locality.

The results on rainy and foggy datasets (Rainy and Foggy CityScapes) also indicate similar aspects to the night scenes. Transformer-based models estimate the plausible depth and distinguish even tiny objects such as bollards. On the contrary, CNN-based models predict incorrect depth maps. In particular, CNN-based models exhibit significant errors in areas such as roads despite similar texture to training datasets. MF-ConvNeXt and MF-Twins also infer plausible depth maps regardless of changes in the weather environment, but MF-RegionViT and MF-SLaK estimate bizarre depth maps. In these experiments, we observe that textural shifts like weather and illumination changes confuse CNN-based models. We also find that CNN-based models are affected by changes in texture as well as loss of texture information, which can be a fatal problem in real-world driving applications.

## 5 Conclusion

In this paper, we have proposed a self-supervised monocular depth estimation method called MonoFormer (MF-Ours). More importantly, we have presented an in-depth analysis of the generalization performance of various modernized backbone structures as well as state-of-the-art self-supervised and supervised methods for monocular depth estimation using various datasets. The in-distribution datasets are used to compare common performance on the KITTI benchmark, while the out-of-distribution and texture-shifted datasets are used to compare the generalization performance. We deeply analyze the properties of the features extracted from each model using the synthetic texture-shifted datasets. Finally, we demonstrate the applicability of the model in real-world scenarios of changing environments using day-night, foggy and rainy datasets.

Through extensive experiments, we observe that MF-Ours and GLPDepth have the best generalization performance among all self-supervised and supervised methods, respectively. The supervised method, GLPDepth, achieves the best generalization performance among all monocular depth estimation methods. More interestingly, we provide three important observations about the generality of monocular depth estimation. First, CNN-based models heavily rely on texture information to recognize scenes and objects, while Transformer-based models use shape information to a greater degree for the monocular depth estimation task. Second, texture-biased representations result in poor generalization performance for environmental changes such as differences in illumination and weather. In contrast, shape-biased representations are more robust to such texture-shifted environments. Lastly, ConvNeXt and RegionViT are CNN-based and Transformer-based, respectively, but have properties different from those of the backbone structure. It shows that the texture-bias and shape-bias are not the intrinsic properties of CNNs and Transformers, respectively. Instead, the intrinsic locality of CNNs induces texture-biased characteristics, while the self-attention mechanism, the base layer of Transformers, induces shape-biased properties.

We believe our observations provide valuable insights into best practices in network design not only for monocular depth estimation but also for a variety of dense prediction tasks such as semantic segmentation, depth completion, normal estimation, etc. Based on these observations, the best way to design generalized networks for dense prediction is to utilize multi-self-attention (MSA) from Transformers and supplement the local information loss of global self-attention by using weak local attention as an auxiliary.

\begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{Datasets}} & \multicolumn{5}{c}{Dataset RobotCar} & \multicolumn{5}{c}{Foggy CityScapes} & \multicolumn{5}{c}{Foggy CityScapes} \\ \cline{2-13} \cline{13-13} \multicolumn{1}{c}{} & Abs Rel & Std & KLR & Std & KLR & Std & KLR & Std & KLR & Std & KLR & Std & KLR & Std & KLR & Std & KLR & Std \\ \hline MonoFormer & 1.03 & **0.04** & **0.19** & 0.204 & 0.40 & 0.564 & 0.48 & 0.080 & 0.109 & 0.514 & 0.720 & 0.853 & 1.215 & 1.294 & 1.299 & 0.138 & 0.134 & 0.489 \\ ParaNet-Soft & 1.00 & 0.05 & **0.19** & 0.193 & 0.78 & 0.536 & 0.457 & 0.095 & 0.104 & 0.545 & 0.782 & 0.873 & 1.515 & 1.711 & 1.699 & 0.244 & 0.446 \\ R-ASFMG & 0.951 & 0.043 & 0.160 & 0.235 & 0.362 & 0.469 & 0.445 & 0.082 & 0.124 & 0.442 & 0.607 & 0.890 & 1.310 & 0.308 & 1.097 & 0.171 & 0.315 & 0.441 \\ Our-ConvN & **0.947** & **0.029** & **0.19** & **0.310** & 0.546 & 0.293 & 0.353 & 0.020 & 0.483 & 0.564 & 0.992 & **0.836** & **0.396** & **0.473** & **0.729** & **0.800** \\ MF-SLaK & 0.916 & 0.041 & 0.160 & 0.239 & 0.456 & 0.623 & 0.839 & 0.151 & 0.285 & 0.299 & 0.440 & 0.652 & 0.856 & 0.510 & 0.460 & 0.356 & 0.611 & 0.822 \\ \hline MF-V & 0.035 & **0.029** & **0.19** & 0.235 & 0.434 & 0.402 & 0.368 & 0.038 & 0.026 & 0.991 & 0.394 & 0.947 & 0.830 & 0.981 & 0.866 & 0.285 & 0.479 \\ MF-RegionViT & 1.067 & 0.041 & 0.160 & 0.290 & 0.456 & 0.623 & 0.640 & 0.088 & 0.167 & 0.337 & 0.645 & 0.782 & 0.708 & 0.773 & 0.805 & 0.251 & 0.840 & 0.676 \\ MF-Series & 0.831 & **0.027** & **0.199** & 0.300 & 0.516 & 0.686 & 0.350 & 0.209 & 0.687 & 0.648 & 0.842 & 0.920 & 0.756 & 0.825 & 0.839 & 0.285 & 0.356 & 0.688 \\ M-Ours & 0.869 & **0.020** & **0.19** & 0.322 & **0.355** & 0.691 & 0.316 & **0.225** & 0.284 & 0.874 & **0.879** & **0.298** & 0.780 & 0.707 & 0.732 & 0.233 & 0.574 & 0.738 \\ \hline BIS & 0.070 & 0.296 & 0.150 & 0.340 & 0.604 & 0.725 & 1.092 & 0.524 & 0.117 & 0.372 & 0.716 & 0.794 & 1.366 & 1.822 & 0.710 & 0.399 & 0.252 & 0.756 \\ AdaLars & 0.677 & 0.039 & 0.160 & 0.371 & 0.640 & 0.721 & 1.441 & 0.534 & 0.158 & 0.316 & 0.467 & 0.744 & 1.299 & 1.452 & 0.466 & 0.330 & 0.585 & 0.771 \\ \hline \hline \multicolumn{1}{c}{ImageNet} & **0.531** & **0.329** & **0.350** & 0.558 & 0.768 & 0.631 & 0.075 & **0.809** & 0.831 & 0.797 & 0.880 & 1.781 & 1.030 & 0.793 & 0.885 & 0.698 & 0.771 \\ DepthFormer & 0.550 & **0.039** & **0.19** & 0.420 & 0.629 & 0.756 & 0.669 & 0.083 & 0.803 & 0.846 & 0.794 & 0.857 & 1.039 & 1.016 & 0.816 & 0.473 & 0.690 & 0.764 \\ GLPDepth & 0.538 & **0.379** & **0.199** & 0.422 & **0.673** & **0.780** & **0.594** & **0.070** & 0.301 & 0.561 & **0.799** & **0.872** & **1.030** & **0.528** & **0.560** & **0.513** & **0.742** & **0.822** \\ \hline \hline \end{tabular}
\end{table} TABLE IX: **Quantitative results on real-world texture-shifted (RobotCar, Foggy and Rainy CityScapes) datasets.**