depth map $D_{pred}$. From the $D_{pred}$, the 3D scene point cloud $P_{pred}$ can be reconstructed. The ground truth point cloud $P_{gt}$ is reconstructed from $D_{gt}$.

We enforce two types of supervision for training the network. We firstly follow standard monocular depth prediction methods to enforce pixel-wise depth supervision over $D_{pred}$ with $D_{gt}$. With the reconstructed point clouds, we then align the spatial relationship between the $P_{pred}$ and the $P_{gt}$ using the proposed *virtual normal*. When the network is well trained, we not only obtain accurate depth map but also high-quality point clouds. From the reconstructed point clouds, other 3D features can be directly calculated, such as the surface normal.

High-order Geometric Constraints
--------------------------------

![Illustration of fitting point clouds to obtain the local surface normal. The directions of the surface normals is fitted with different sampling sizes on a real point cloud (a). Because of noise, the surface normals vary significantly. (b) compares the angular difference between surface normals computed with different sample sizes in Mean Difference Error. The error can vary significantly. []{data-label="fig:fit plane"}](./Fig/Fitting_normal_errors){width="45.00000%"}

**Surface Normal.** The surface normal is an important ‘local’ feature for many point-cloud based applications such as registration [@rusu2008aligning] and object detection [@hinterstoisser2011multimodal; @gupta2014learning]. It appears 