Portrait segmentation has gained more and more attractions in recent years due to the popularity of selfie images. Compared to general semantic segmentation problems, portrait segmentation focuses on facial areas with higher requirements especially over the boundaries. To improve the performance of portrait segmentation, we propose a boundary-sensitive deep neural network (BSN) for better accuracy among the portrait boundaries. BSN introduces three novel techniques. First, an individual boundary-sensitive mask is proposed by dilating the contour line and assigning the boundary pixels with multi-class labels. Second, a global boundary-sensitive mask is employed as a position sensitive prior to further constrain the overall shape of the segmentation map. Third, we train a boundary-sensitive attribute classifier jointly with the segmentation network to reinforce the network with semantic boundary shape information. We have evaluated BSN on the state-of-the-art public portrait segmentation datasets, i.e., the PFCN dataset, as well as the portrait images collected from other three popular image segmentation datasets: COCO, COCO-Stuff, and PASCAL VOC. Our method achieves the superior quantitative and qualitative performance over state-of-the-arts on the evaluated datasets, especially obtains better visualization effect on the portrait boundary region.
In recent years, convolutional neural network (CNN) based methods have been successfully applied to semantic segmentation. In 2014, Long al @cite_20 introduced the end-to-end Fully Convolutional Networks (FCN) which takes a natural image as input and performs dense pixel-wise predictions to generate a segmentation map of the same size as the input image. Fully connected layers are removed from this network to preserve the spatial information and deconvolutional layers are proposed for up-sampling to recover the full image size. This paradigm popularized the CNN based method and was quickly adopted by subsequent approaches. In traditional CNN architectures, pooling layer was introduced to increase the receptive field as the network goes deeper. However, it also decreases the resolution of feature map. Yu al. @cite_22 proposed the dilated convolutional layer to replace the pooling layer, which allows for increasing the size of the receptive field without losing resolution in feature maps. Chen al. @cite_11 proposed the DeepLab system which passes multiple rescaled input images to different network branches in parallel and combines the features maps with max operation at the end.
@cite_22 @cite_20 @cite_11
2963840672 1903029394 2412782625
Abstract: State-of-the-art models for semantic segmentation are based on adaptations of convolutional networks that had originally been designed for image classification. However, dense prediction and image classification are structurally different. In this work, we develop a new convolutional network module that is specifically designed for dense prediction. The presented module uses dilated convolutions to systematically aggregate multi-scale contextual information without losing resolution. The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage. We show that the presented context module increases the accuracy of state-of-the-art semantic segmentation systems. In addition, we examine the adaptation of image classification networks to dense prediction and show that simplifying the adapted network can increase accuracy. Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, exceed the state-of-the-art in semantic segmentation. Our key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet [20], the VGG net [31], and GoogLeNet [32]) into fully convolutional networks and transfer their learned representations by fine-tuning [3] to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional network achieves state-of-the-art segmentation of PASCAL VOC (20 relative improvement to 62.2 mean IU on 2012), NYUDv2, and SIFT Flow, while inference takes less than one fifth of a second for a typical image. In this work we address the task of semantic image segmentation with Deep Learning and make three main contributions that are experimentally shown to have substantial practical merit. First , we highlight convolution with upsampled filters, or ‘atrous convolution’, as a powerful tool in dense prediction tasks. Atrous convolution allows us to explicitly control the resolution at which feature responses are computed within Deep Convolutional Neural Networks. It also allows us to effectively enlarge the field of view of filters to incorporate larger context without increasing the number of parameters or the amount of computation. Second , we propose atrous spatial pyramid pooling (ASPP) to robustly segment objects at multiple scales. ASPP probes an incoming convolutional feature layer with filters at multiple sampling rates and effective fields-of-views, thus capturing objects as well as image context at multiple scales. Third , we improve the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models. The commonly deployed combination of max-pooling and downsampling in DCNNs achieves invariance but has a toll on localization accuracy. We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF), which is shown both qualitatively and quantitatively to improve localization performance. Our proposed “DeepLab” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task, reaching 79.7 percent mIOU in the test set, and advances the results on three other datasets: PASCAL-Context, PASCAL-Person-Part, and Cityscapes. All of our code is made publicly available online.