the continuous conditional random field (CRF) to exploit consistency information of neighbouring pixels. CRF establishes a pair-wise constraint for local regions. Furthermore, several high-order constraints are investigated. Chen [@chen2018rethinking] applied the generative adversarial training to lead the network to learn a context-aware and patch-level loss automatically. Note that most of these methods directly work with the depth, instead of in the 3D space.

![image](./Fig/pipeline){width=".82\textwidth"}

\[fig:architecture\]

**Surface Normal.** Surface normal is an important geometry information for 3D scene understanding. Several data-driven methods [@eigen2015predicting; @eigen2014depth; @fouhey2013data; @fouhey2014unfolding; @wang2015designing; @zeisl2014discriminatively] have achieved promising results. Eigen  [@eigen2015predicting] proposed a CNN with different output channels to directly predict depth map, surface normal and semantic labels. Bansal  [@bansal2016marr] proposed a two-stream network to predict the surface normal first, which is further joined with the input image to learn the pose. Note that most of these methods formulate surface normal prediction and depth prediction as multiple different tasks.

Our Method
==========

Our approach resolves the monocular depth prediction and reconstructs the high-quality scene 3D point cloud from the predicted depth at the same time. The pipeline is illustrated in Fig. \[fig:architecture\].

We take an RGB image $I_{in}$ as the input of an encoder-decoder network and predict the 