are extensively evaluated under challenging scenarios. In particular, on PASCAL VOC 2012, our generated semantic segmentations are more favorable than those by other weakly-supervised scene labeling methods. Moreover, we propose a dedicated dataset for facilitating further research on scene parsing, which contains more than 5000 scene images of 33 categories with elaborative annotations for semantic object label maps, scene hierarchy and inter-object relations.

The remainder of this paper is organized as follows. Section \[sec:Related\_Work\] provides a brief review of the related work. Then we introduce the CNN-RsNN model in Section \[CNN-RNN Architecture\] and follow with the model training algorithm in Section \[sec:training\]. The experimental results and comparisons are presented in Section \[sec:experiments\]. Section \[sec:conclusion\] concludes the paper and presents some outlook for future work.

Related Work {#sec:Related_Work}
============

Scene understanding has been approached through many recognition tasks such as image classification, object detection, and semantic segmentation. In current research, a myriad of different methods focus on what general scene type the image shows (classification) [@DBLP:VisualAttr][@DBLP:Multi-Class][@DBLP:fine-grained], what objects and their locations are in a scene (semantic labeling or segmentation) [@DBLP:Seg-03][@DBLP:Seg-09][@DBLP:Seg-15][@tighe2014scene]. These methods, however, ignore or over simplified the compositional representation of objects and fail to gain a deeper and structured understanding 