Note that the above overall loss function is differentiable. The gradient of the $ \ell_{VN}$ loss can be easily computed as Eq.  and Eq.  are both differentiable.

Experiments
===========

In this section, we conduct several experiments to compare ours against state-of-the-art methods. We evaluate our methods on two datasets, NYUD-V2 and KITTI.

Datasets
--------

**NYUD-V2.** The NYUD-V2 dataset consists of 464 different indoor scenes, which are further divided into 249 scenes for training and 215 for testing. We randomly sample 29K images from the training set to form NYUD-Large. Note that DORN uses the whole training set, which is significantly larger than that what we use. Apart from the whole dataset, there are officially annotated 1449 images (NYUD-Small), in which 795 images are split for training and others are for testing. In the ablation study, we use the NYUD-Small data.

**KITTI.** The KITTI dataset contains over 93K outdoor images and depth maps with the resolution around $1240\times374$. All images are captured on driving cars by stereo cameras and a Lidar. We test on 697 images from 29 scenes split by Eigen [@eigen2014depth], validate on 888 images, and train on about 23488 images from the remaining 32 scenes.

Implementation Details
----------------------

