on scene.

Meanwhile, as a higher-level task, structured scene parsing has also attracted much attention. A pioneering work was proposed by Tu et al. [@tu2005image], in which they mainly focused on faces and texture patterns by a Bayesian inference framework. In [@DBLP:ImageParsing-Attribute], Han et al. proposed to hierarchically parse the indoor scene images by developing a generative grammar model. An extended study also explored the more complex outdoor environment in [@DBLP:3Dparsing-liu]. A hierarchical model was proposed in [@zhu2012recursive] to represent the image recursively by contextualized templates at multiple scales, and rapid inference was realized based on dynamic programming. Ahuja et al. [@ahuja2008connected] developed a connected segmentation tree for object and scene parsing. Some other related works [@silberman2012indoor][@gupta2013perceptual] investigated the approaches for RGB-D scene understanding, and achieved impressive results. Among these works, the hierarchical space tiling (HST) proposed by Wang et al. [@J:Space-Tiling], which was applied to quantize the huge and continuous scene configuration space, seemed to be the most related one to ours. It adopted the weakly supervised learning associated the text (*i.e.* nouns and adjectives) to optimize the structure of the parsing graph. But the authors didn’t introduce the relations between objects into their method. In terms of the 