Visual navigation based on a topological map using only cheap cameras as external sensors has potential to solve these problems ; road-following and intersection recognition can enable robust navigation . This paper proposes a novel scheme for intersection recognition using results of semantic segmentation , which has a high affinity for vision-based road-following strongly depending on semantic segmentation . The proposed scheme mainly composed of mode filtering for a segmented image and similarity computation like the Hamming distance showed that good accuracy for the Tsukuba-Challenge 2018 dataset constructed by the authors : perfect results were obtained for more than half intersections included in the dataset . Rather than using a single downstream classifier , the fused deep neural network ( F - DNN + SS ) method uses a derivation of the Faster R - CNN framework fusing multiple parallel classifiers including Resnet and Googlenet using soft - rejection , and further incorporates pixel - wise semantic segmentation in a post - processing manner to suppress background proposals . Simultaneous Detection & Segmentation RCNN ( SDS - RCNN ) improves object detection by using semantic segmentation as a strong cue , infusing the segmentation masks on top of shared feature maps as a reinforcement to the pedestrian detector . Recently , an active detection model ( ADM ) based on multi - layer feature representations , executes sequences of coordinate transformation actions on a set of initial bounding - box proposals to deliver accurate prediction of pedestrian locations , and achieve a more balanced detection performance for different scale pedestrian instances on the Caltech benchmark . A shortcoming of this method is that , although they reason globally about instances , they also can not handle occlusions . Arnab combine an object detector with a semantic segmentation module using a CRF model . By considering the image holistically it can handle occlusions and produce more precise segmentations . section : B.2 Training our proposed network B.2.1 Training the category - level segmentation module We initialise our semantic segmentation network with the COCO pre - trained ResNet - 101 weights provided by [ reference ] . Training is first performed on the Pascal VOC 2012 training set using the extra annotations from [ reference ] , which combine to a total of 9012 training images . Care is taken to ensure that all images from the Pascal Person - Parts test set is excluded from this training set . Experiments on rice dataset from the Chinese spacelab TG-2 demonstrate the impressing segmentation performance of our TASCN , especially for complex background interference and uneven illumination . In addition , both the quantitative evaluation ( Mean IoU ) and qualitative visual results show that our TASCN outperforms the state-of-the-art methods , including FCN , U-Net , SegNet , PSPNet , DeepLabv3 + and DANet . In conclusion , a novel TASCN is proposed to meet the increasing need for fine plant parsing in plant research . Then , the SegNet-VGG is acquired by fine-tuned transfer learning VGG16 , using the first 13 layers of VGG16 as the encoder of the SegNet and combined the original decoder of the SegNet . By adjusting the input size of the Unet-VGG , SegNet-VGG , and PSPNet , the preprocessed data can be correctly fed to the three network models . The three models are used as the basic trainer to train and segment the datasets . In this paper , we propose a pixel-level image encryption method based on semantic segmentation . We use a PSPNet model trained on Cityscapes dataset to gain the coordinate of each pixels in ROI , then we encrypt these coordinates with a novel chaotic encryption system we proposed . In this system , we improved the scrambling-diffusion architecture . The application of the coarse ground truth ( GT ) annotations of these datasets without detriment to the accuracy of semantic image segmentation ( by the mean intersection over union — mIoU ) could simplify and speedup the dataset preparation and model fine tuning before its practical application . Here the results of the comparative analysis for semantic segmentation accuracy obtained by PSPNet deep learning architecture are presented for fine and coarse annotated images from Cityscapes dataset . Two scenarios were investigated : scenario 1—the fine GT images for training and prediction , and scenario 2—the fine GT images for training and the coarse GT images for prediction . At the same time , the semantic segmentation is added to the SLAM system to generate a semantic map with semantic information , which can enrich the understanding of the mobile carrier to the environment and obtain high-level perception . In this paper , we combine the visual SLAM system ORB-SLAM2 and PSPNet semantic segmentation network , and propose a PSPNet-SLAM system , which uses optical flow and semantic segmentation to detect and eliminate dynamic points to achieve dynamic scenes semantic SLAM . We performed experiments on the TUM RGB-D dataset . For the models pre-trained on Cityscapes dataset , the inference time was equal in the limits of standard deviation , but the segmentation accuracy was different for various cities and stereo channels even . The distributions of accuracy ( mean intersection over union - mIoU ) values for each city and channel are asymmetric , long-tailed , and have many extreme outliers , especially for PSPNet network in comparison to ICNet network . Some statistical properties of these distributions ( skewness , kurtosis ) allow us to distinguish these two networks and open the question about relations between architecture of deep learning networks and statistical distribution of the predicted results ( mIoU here ) . Moreover , we build a Welding Spot Quality Inspection Dataset taken from real assembly line . Compared with other pipelines , including U-net , MaskRCNN , and PSPNet , CU-net shows a significant superiority in both processing speed and detection accuracy . The results of template-based method and TCV heuristic algorithms have shown high computational efficiency and ensured inspection accuracy . An Unmanned Aerial Vehicle ( UAV ) is used to obtain high-resolution RGB images of a Brazilian coffee farm . The proposed methodology uses Convolutional Neural Networks ( CNN ) with U-Net and PSPNet architectures to classify areas into two classes : pests and non-pests . Results demonstrate the viability of the proposed methodology , with an average F-measure of 0.69 for the U-Net architecture with the image resolution 640 × 480 . Compared with real-time segmentation models such as BiSeNet , our model achieves higher accuracy at comparable speed on the Cityscapes Dataset , enabling the application in speed-demanding tasks such as street-scene understanding for autonomous driving . Furthermore , our ShelfNet achieves 79.0 % mIoU on Cityscapes Dataset with ResNet34 backbone , outper-forming PSPNet and BiSeNet with large backbones such as ResNet101 . Through extensive experiments , we validated the superior performance of ShelfNet . The proposed method achieves 65.7 % mean intersection over union ( mIoU ) on the Cityscapes test set with only 8.4 G floating-point operations ( FLOPs ) on $ 1024\times 2048 $ inputs . Furthermore , by re-training the existing PSPNet and DeepLabV3 models on our synthetic dataset , we obtained an average 2 % mIoU improvement on small objects . In this paper , we extend the use of deep learning to remote sensing image classification to large geographical regions , and explore a way to make deep learning classifiers transferable for different regions . We take Jingjinji region and Henan province in China as the study areas , and choose FCN , ResNet , and PSPNet as classifiers . The models are trained by different proportions of training samples from Jingjinji region . Scene parsing is challenging for unrestricted open vocabulary and diverse scenes . In this paper , we exploit the capability of global context information by different - region - based context aggregation through our pyramid pooling module together with the proposed pyramid scene parsing network ( PSPNet ) . Our global prior representation is effective to produce good quality results on the scene parsing task , while PSPNet provides a superior framework for pixel - level prediction . section : Image Cascade Network We start by analyzing computation time budget of different components on the high performance segmentation framework PSPNet with experimental statistics . Then we introduce the image cascade network ( ICNet ) as illustrated in Fig . Specially , for DeepLab over e-ophtha EX , the F-score increases 5.2 percentage points , and the area under the SE-PPV curve ( AUC ) increases 10.6 percentage points . Moreover , the AUC increases more than 4 percentage points over IDRiD dataset for both DeepLab and FCRN . The source code of bin loss is available at : https://github.com/guomugong/bin_loss . Semantic image segmentation is an essential part of remote sensing image processing because accurate understanding of the ground information is the first step in obtaining useful knowledge of surface coverage . The popular semantic segmentation convolutional neural network model ( DeepLab v3 + ) can not effectively use attention information , resulting in coarse segmentation boundaries . In this work , a new type of bottleneck using attention information which can extract semantic information and more abundant features from images is proposed . Finding suggest that using all available images , regardless of season , is preferred . Data augmentations were found to greatly increase performance for both networks , with DeepLab v3 + achieving up to 81 % recall and 69 % precision . However , caution should be taken when applying augmentations which modify spectral values of pixels . DeepLab is an algorithm based on the fully convolutional networks . However , DeepLab algorithm still has room for improvement , and we design three improved methods : ( 1 ) the global context structure module , ( 2 ) highly efficient decoder module , and ( 3 ) multi-scale feature fusion module . The experimental results show that the three improved methods that we proposed in this paper can make the model obtain more expressive features and improve the accuracy of the algorithm . Eyes with florid disease are particularly challenging to segment , as the anatomy is often highly distorted from normal . In this context , we propose an end-to-end machine learning method consisting of near perfect detection of retinal fluid using random forest classifier and an efficient DeepLab algorithm for quantification and labeling of the target fluid compartments . In particular , we achieve an average Dice score of 86.23 % with reference to manual delineations made by a trained expert . By using atrous convolution , DeepLab network 's filters can observe greater receptive field without reducing the feature map dimension , therefore this structure keeps global and position information . Consequently , we compromises above mentioned methods , The optical flow estimation is carried out on the image processed by the CLAHE method , and the accurate segmentation results are obtained by using the DeepLab network . Experiments show good performance of our method . Segmentation of dermoscopic images is the first step in this process , thus accuracy is crucial . Although techniques utilizing convolutional neural networks have been used in the past for lesion segmentation , we present a solution employing the recently published DeepLab 3 , an atrous convolution method for image segmentation . Although the results produced by this run are not ideal , with a mean Jaccard index of 0.498 , we believe that with further adjustments and modifications to the compatibility with the DeepLab code and with training on more powerful processing units , this method may achieve better results in future trials . We demonstrate the utility of adversarially tuned scene generation on two real world benchmark datasets ( CityScapes and CamVid ) for traffic scene semantic labeling with a deep convolutional net ( DeepLab ) . We obtained performance improvements by 2.28 and 3.14 points on the IoU metric between the DeepLab models trained on simulated sets prepared from the scene generation models before and after tuning to CityScapes and CamVid respectively . This paper follows up on our prior publication on the application of AI and deep learning in the evaluation of BE.1 2 In our initial publications , we developed a computer-aided diagnosis ( CAD ) model and demonstrated promising performance scores in the classification and segmentation domains during BE assessment.1 2 However , these results were achieved on optimal endoscopic images , which may not mirror the real-life situation sufficiently . To enable the seamless integration of AI-based image classification into the clinical workflow , our previous system was developed further to increase the speed of image analysis for classification and the resolution of the dense prediction , which shows the color-coded spatial distribution of cancer probabilities.1 2 Still based on deep convolutional neural nets ( CNNs ) and a residual net ( ResNet ) architecture with DeepLab V.3 + , a state-of-the-art encoder – decoder network was adapted.3 To transfer the endoscopic livestream to our AI system , a capture card ( Avermedia , Taiwan ) was plugged to the endoscopic monitor . Online supplementary video 1 shows the setting of AI-based BE evaluation in the endoscopy room of the University Hospital Augsburg ( figure 1 ) . We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field ( CRF ) , which is shown both qualitatively and quantitatively to improve localization performance . Our proposed “ DeepLab ” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task , reaching 79.7 percent mIOU in the test set , and advances the results on three other datasets : PASCAL-Context , PASCAL-Person-Part , and Cityscapes . All of our code is made publicly available online . On LUNA16 , the two-stage framework attained a sensitivity of 96.4 % , outperforming other recent models in the literature , including deep models . Finally , we show that adopting a transfer learning approach , particularly , the DeepLab model weights of the first stage of the framework , to infer binary ( malignant-benign ) labels on the Kaggle dataset for pulmonary nodules achieves a classification accuracy of 95.66 % , which represents approximately 4 % improvement over the recent literature . We also elaborate on implementation details and share our experience on training our system . The proposed ` DeepLabv3 ' system significantly improves over our previous DeepLab versions without DenseCRF post-processing and attains comparable performance with other state-of-art models on the PASCAL VOC 2012 semantic image segmentation benchmark . As shown in Tab . [ reference ] ( a ) , incorporating the fully connected CRF to our model ( denoted by DeepLab - CRF ) yields a substantial performance boost , about 4 % improvement over DeepLab . We note that the work of krahenbuhl2011efficient improved the result of TextonBoost shotton2009textonboost to , which makes the improvement we report here ( from to ) all the more impressive . As shown in Tab . [ reference ] ( a ) , adding the multi - scale features to our DeepLab model ( denoted as DeepLab - MSc ) improves about performance , and further incorporating the fully connected CRF ( denoted as DeepLab - MSc - CRF ) yields about 4 % improvement . Finally , we employ kernel size and input stride = 12 , and further change the filter sizes from 4096 to 1024 for the last two layers . Interestingly , the resulting model , DeepLab - CRF - LargeFOV , matches the performance of the expensive DeepLab - CRF - 7x7 . At the same time , it is times faster to run and has significantly fewer parameters ( 20.5 M instead of 134.3 M ) . Submission to ICLR 2015 . Introduces the model DeepLab - CRF , which attains the performance of on PASCAL VOC 2012 test set . paragraph : v2 Rebuttal for ICLR 2015 . Adds the model DeepLab - MSc - CRF , which incorporates multi - scale features from the intermediate layers . DeepLab - MSc - CRF yields the performance of on PASCAL VOC 2012 test set . To obtain high - resolution predictions , a number of recent methods focus on refining the low - resolution prediction to obtain high resolution prediction . DeepLab - CRF performs bilinear upsampling of the prediction score map to the input image size and apply the dense CRF method to refine the object boundary by leveraging the color contrast information . CRF - RNN extends this approach by implementing recurrent layers for end - to - end learning of the dense CRF and the FCNN network . The network was trained for 60 K iterations . We now compare the accuracy of our front - end module to the FCN - 8s design of and the DeepLab network of . For FCN - 8s and DeepLab , we evaluate the public models trained by the original authors on VOC - 2012 . The vanilla base network without using any module is also included as a baseline ( referred to as “ NonAdapt ” ) . Two different semantic segmentation networks are used , DeepLab V2 and the PSP Net , where the former one uses VGG - 16 as the backbone network , and the latter one uses ResNet - 101 . The results of all variants of our methods based on two different models are summarized in Talbe [ reference ] . In particular , the “ frozen ” method refers to freezing the first a few layers when training the segmentation model , and “ source distillation ” refers to replacing real images with synthetic images for distillation . For all methods , the DeepLab network is deployed due to its efficiency . All other experimental settings are identical with previous experiments . Predictions from all branches are fused together to produce the final segmentation output . Please refer to the DeepLab - ASPP - L model ( A trous S patial P yramid P ooling , with L arge atrous rates ) from for details . Pose Estimation .
Visual navigation based on a topological map using only cheap cameras as external sensors has potential to solve these problems ; road-following and intersection recognition can enable robust navigation . This paper proposes a novel scheme for intersection recognition using results of semantic segmentation , which has a high affinity for vision-based road-following strongly depending on semantic segmentation . The proposed scheme mainly composed of mode filtering for a segmented image and similarity computation like the Hamming distance showed that good accuracy for the Tsukuba-Challenge 2018 dataset constructed by the authors : perfect results were obtained for more than half intersections included in the dataset .
Rather than using a single downstream classifier , the fused deep neural network ( F - DNN + SS ) method uses a derivation of the Faster R - CNN framework fusing multiple parallel classifiers including Resnet and Googlenet using soft - rejection , and further incorporates pixel - wise semantic segmentation in a post - processing manner to suppress background proposals . Simultaneous Detection & Segmentation RCNN ( SDS - RCNN ) improves object detection by using semantic segmentation as a strong cue , infusing the segmentation masks on top of shared feature maps as a reinforcement to the pedestrian detector . Recently , an active detection model ( ADM ) based on multi - layer feature representations , executes sequences of coordinate transformation actions on a set of initial bounding - box proposals to deliver accurate prediction of pedestrian locations , and achieve a more balanced detection performance for different scale pedestrian instances on the Caltech benchmark .
A shortcoming of this method is that , although they reason globally about instances , they also can not handle occlusions . Arnab combine an object detector with a semantic segmentation module using a CRF model . By considering the image holistically it can handle occlusions and produce more precise segmentations .
section : B.2 Training our proposed network B.2.1 Training the category - level segmentation module We initialise our semantic segmentation network with the COCO pre - trained ResNet - 101 weights provided by [ reference ] . Training is first performed on the Pascal VOC 2012 training set using the extra annotations from [ reference ] , which combine to a total of 9012 training images . Care is taken to ensure that all images from the Pascal Person - Parts test set is excluded from this training set .
Experiments on rice dataset from the Chinese spacelab TG-2 demonstrate the impressing segmentation performance of our TASCN , especially for complex background interference and uneven illumination . In addition , both the quantitative evaluation ( Mean IoU ) and qualitative visual results show that our TASCN outperforms the state-of-the-art methods , including FCN , U-Net , SegNet , PSPNet , DeepLabv3 + and DANet . In conclusion , a novel TASCN is proposed to meet the increasing need for fine plant parsing in plant research .
Then , the SegNet-VGG is acquired by fine-tuned transfer learning VGG16 , using the first 13 layers of VGG16 as the encoder of the SegNet and combined the original decoder of the SegNet . By adjusting the input size of the Unet-VGG , SegNet-VGG , and PSPNet , the preprocessed data can be correctly fed to the three network models . The three models are used as the basic trainer to train and segment the datasets .
In this paper , we propose a pixel-level image encryption method based on semantic segmentation . We use a PSPNet model trained on Cityscapes dataset to gain the coordinate of each pixels in ROI , then we encrypt these coordinates with a novel chaotic encryption system we proposed . In this system , we improved the scrambling-diffusion architecture .
The application of the coarse ground truth ( GT ) annotations of these datasets without detriment to the accuracy of semantic image segmentation ( by the mean intersection over union — mIoU ) could simplify and speedup the dataset preparation and model fine tuning before its practical application . Here the results of the comparative analysis for semantic segmentation accuracy obtained by PSPNet deep learning architecture are presented for fine and coarse annotated images from Cityscapes dataset . Two scenarios were investigated : scenario 1—the fine GT images for training and prediction , and scenario 2—the fine GT images for training and the coarse GT images for prediction .
At the same time , the semantic segmentation is added to the SLAM system to generate a semantic map with semantic information , which can enrich the understanding of the mobile carrier to the environment and obtain high-level perception . In this paper , we combine the visual SLAM system ORB-SLAM2 and PSPNet semantic segmentation network , and propose a PSPNet-SLAM system , which uses optical flow and semantic segmentation to detect and eliminate dynamic points to achieve dynamic scenes semantic SLAM . We performed experiments on the TUM RGB-D dataset .
For the models pre-trained on Cityscapes dataset , the inference time was equal in the limits of standard deviation , but the segmentation accuracy was different for various cities and stereo channels even . The distributions of accuracy ( mean intersection over union - mIoU ) values for each city and channel are asymmetric , long-tailed , and have many extreme outliers , especially for PSPNet network in comparison to ICNet network . Some statistical properties of these distributions ( skewness , kurtosis ) allow us to distinguish these two networks and open the question about relations between architecture of deep learning networks and statistical distribution of the predicted results ( mIoU here ) .
Moreover , we build a Welding Spot Quality Inspection Dataset taken from real assembly line . Compared with other pipelines , including U-net , MaskRCNN , and PSPNet , CU-net shows a significant superiority in both processing speed and detection accuracy . The results of template-based method and TCV heuristic algorithms have shown high computational efficiency and ensured inspection accuracy .
An Unmanned Aerial Vehicle ( UAV ) is used to obtain high-resolution RGB images of a Brazilian coffee farm . The proposed methodology uses Convolutional Neural Networks ( CNN ) with U-Net and PSPNet architectures to classify areas into two classes : pests and non-pests . Results demonstrate the viability of the proposed methodology , with an average F-measure of 0.69 for the U-Net architecture with the image resolution 640 × 480 .
Compared with real-time segmentation models such as BiSeNet , our model achieves higher accuracy at comparable speed on the Cityscapes Dataset , enabling the application in speed-demanding tasks such as street-scene understanding for autonomous driving . Furthermore , our ShelfNet achieves 79.0 % mIoU on Cityscapes Dataset with ResNet34 backbone , outper-forming PSPNet and BiSeNet with large backbones such as ResNet101 . Through extensive experiments , we validated the superior performance of ShelfNet .
The proposed method achieves 65.7 % mean intersection over union ( mIoU ) on the Cityscapes test set with only 8.4 G floating-point operations ( FLOPs ) on $ 1024\times 2048 $ inputs . Furthermore , by re-training the existing PSPNet and DeepLabV3 models on our synthetic dataset , we obtained an average 2 % mIoU improvement on small objects .
In this paper , we extend the use of deep learning to remote sensing image classification to large geographical regions , and explore a way to make deep learning classifiers transferable for different regions . We take Jingjinji region and Henan province in China as the study areas , and choose FCN , ResNet , and PSPNet as classifiers . The models are trained by different proportions of training samples from Jingjinji region .
Scene parsing is challenging for unrestricted open vocabulary and diverse scenes . In this paper , we exploit the capability of global context information by different - region - based context aggregation through our pyramid pooling module together with the proposed pyramid scene parsing network ( PSPNet ) . Our global prior representation is effective to produce good quality results on the scene parsing task , while PSPNet provides a superior framework for pixel - level prediction .
section : Image Cascade Network We start by analyzing computation time budget of different components on the high performance segmentation framework PSPNet with experimental statistics . Then we introduce the image cascade network ( ICNet ) as illustrated in Fig .
Specially , for DeepLab over e-ophtha EX , the F-score increases 5.2 percentage points , and the area under the SE-PPV curve ( AUC ) increases 10.6 percentage points . Moreover , the AUC increases more than 4 percentage points over IDRiD dataset for both DeepLab and FCRN . The source code of bin loss is available at : https://github.com/guomugong/bin_loss .
Semantic image segmentation is an essential part of remote sensing image processing because accurate understanding of the ground information is the first step in obtaining useful knowledge of surface coverage . The popular semantic segmentation convolutional neural network model ( DeepLab v3 + ) can not effectively use attention information , resulting in coarse segmentation boundaries . In this work , a new type of bottleneck using attention information which can extract semantic information and more abundant features from images is proposed .
Finding suggest that using all available images , regardless of season , is preferred . Data augmentations were found to greatly increase performance for both networks , with DeepLab v3 + achieving up to 81 % recall and 69 % precision . However , caution should be taken when applying augmentations which modify spectral values of pixels .
DeepLab is an algorithm based on the fully convolutional networks . However , DeepLab algorithm still has room for improvement , and we design three improved methods : ( 1 ) the global context structure module , ( 2 ) highly efficient decoder module , and ( 3 ) multi-scale feature fusion module . The experimental results show that the three improved methods that we proposed in this paper can make the model obtain more expressive features and improve the accuracy of the algorithm .
Eyes with florid disease are particularly challenging to segment , as the anatomy is often highly distorted from normal . In this context , we propose an end-to-end machine learning method consisting of near perfect detection of retinal fluid using random forest classifier and an efficient DeepLab algorithm for quantification and labeling of the target fluid compartments . In particular , we achieve an average Dice score of 86.23 % with reference to manual delineations made by a trained expert .
By using atrous convolution , DeepLab network 's filters can observe greater receptive field without reducing the feature map dimension , therefore this structure keeps global and position information . Consequently , we compromises above mentioned methods , The optical flow estimation is carried out on the image processed by the CLAHE method , and the accurate segmentation results are obtained by using the DeepLab network . Experiments show good performance of our method .
Segmentation of dermoscopic images is the first step in this process , thus accuracy is crucial . Although techniques utilizing convolutional neural networks have been used in the past for lesion segmentation , we present a solution employing the recently published DeepLab 3 , an atrous convolution method for image segmentation . Although the results produced by this run are not ideal , with a mean Jaccard index of 0.498 , we believe that with further adjustments and modifications to the compatibility with the DeepLab code and with training on more powerful processing units , this method may achieve better results in future trials .
We demonstrate the utility of adversarially tuned scene generation on two real world benchmark datasets ( CityScapes and CamVid ) for traffic scene semantic labeling with a deep convolutional net ( DeepLab ) . We obtained performance improvements by 2.28 and 3.14 points on the IoU metric between the DeepLab models trained on simulated sets prepared from the scene generation models before and after tuning to CityScapes and CamVid respectively .
This paper follows up on our prior publication on the application of AI and deep learning in the evaluation of BE.1 2 In our initial publications , we developed a computer-aided diagnosis ( CAD ) model and demonstrated promising performance scores in the classification and segmentation domains during BE assessment.1 2 However , these results were achieved on optimal endoscopic images , which may not mirror the real-life situation sufficiently . To enable the seamless integration of AI-based image classification into the clinical workflow , our previous system was developed further to increase the speed of image analysis for classification and the resolution of the dense prediction , which shows the color-coded spatial distribution of cancer probabilities.1 2 Still based on deep convolutional neural nets ( CNNs ) and a residual net ( ResNet ) architecture with DeepLab V.3 + , a state-of-the-art encoder – decoder network was adapted.3 To transfer the endoscopic livestream to our AI system , a capture card ( Avermedia , Taiwan ) was plugged to the endoscopic monitor . Online supplementary video 1 shows the setting of AI-based BE evaluation in the endoscopy room of the University Hospital Augsburg ( figure 1 ) .
We overcome this by combining the responses at the final DCNN layer with a fully connected Conditional Random Field ( CRF ) , which is shown both qualitatively and quantitatively to improve localization performance . Our proposed “ DeepLab ” system sets the new state-of-art at the PASCAL VOC-2012 semantic image segmentation task , reaching 79.7 percent mIOU in the test set , and advances the results on three other datasets : PASCAL-Context , PASCAL-Person-Part , and Cityscapes . All of our code is made publicly available online .
On LUNA16 , the two-stage framework attained a sensitivity of 96.4 % , outperforming other recent models in the literature , including deep models . Finally , we show that adopting a transfer learning approach , particularly , the DeepLab model weights of the first stage of the framework , to infer binary ( malignant-benign ) labels on the Kaggle dataset for pulmonary nodules achieves a classification accuracy of 95.66 % , which represents approximately 4 % improvement over the recent literature .
We also elaborate on implementation details and share our experience on training our system . The proposed ` DeepLabv3 ' system significantly improves over our previous DeepLab versions without DenseCRF post-processing and attains comparable performance with other state-of-art models on the PASCAL VOC 2012 semantic image segmentation benchmark .
As shown in Tab . [ reference ] ( a ) , incorporating the fully connected CRF to our model ( denoted by DeepLab - CRF ) yields a substantial performance boost , about 4 % improvement over DeepLab . We note that the work of krahenbuhl2011efficient improved the result of TextonBoost shotton2009textonboost to , which makes the improvement we report here ( from to ) all the more impressive .
As shown in Tab . [ reference ] ( a ) , adding the multi - scale features to our DeepLab model ( denoted as DeepLab - MSc ) improves about performance , and further incorporating the fully connected CRF ( denoted as DeepLab - MSc - CRF ) yields about 4 % improvement .
Finally , we employ kernel size and input stride = 12 , and further change the filter sizes from 4096 to 1024 for the last two layers . Interestingly , the resulting model , DeepLab - CRF - LargeFOV , matches the performance of the expensive DeepLab - CRF - 7x7 . At the same time , it is times faster to run and has significantly fewer parameters ( 20.5 M instead of 134.3 M ) .
Submission to ICLR 2015 . Introduces the model DeepLab - CRF , which attains the performance of on PASCAL VOC 2012 test set . paragraph : v2 Rebuttal for ICLR 2015 . Adds the model DeepLab - MSc - CRF , which incorporates multi - scale features from the intermediate layers . DeepLab - MSc - CRF yields the performance of on PASCAL VOC 2012 test set .
To obtain high - resolution predictions , a number of recent methods focus on refining the low - resolution prediction to obtain high resolution prediction . DeepLab - CRF performs bilinear upsampling of the prediction score map to the input image size and apply the dense CRF method to refine the object boundary by leveraging the color contrast information . CRF - RNN extends this approach by implementing recurrent layers for end - to - end learning of the dense CRF and the FCNN network .
The network was trained for 60 K iterations . We now compare the accuracy of our front - end module to the FCN - 8s design of and the DeepLab network of . For FCN - 8s and DeepLab , we evaluate the public models trained by the original authors on VOC - 2012 .
The vanilla base network without using any module is also included as a baseline ( referred to as “ NonAdapt ” ) . Two different semantic segmentation networks are used , DeepLab V2 and the PSP Net , where the former one uses VGG - 16 as the backbone network , and the latter one uses ResNet - 101 . The results of all variants of our methods based on two different models are summarized in Talbe [ reference ] .
In particular , the “ frozen ” method refers to freezing the first a few layers when training the segmentation model , and “ source distillation ” refers to replacing real images with synthetic images for distillation . For all methods , the DeepLab network is deployed due to its efficiency . All other experimental settings are identical with previous experiments .
Predictions from all branches are fused together to produce the final segmentation output . Please refer to the DeepLab - ASPP - L model ( A trous S patial P yramid P ooling , with L arge atrous rates ) from for details . Pose Estimation .
Intersection Recognition Using Results of Semantic Segmentation for Visual Navigation Small-scale Pedestrian Detection Based on Somatic Topology Localization and Temporal Feature Aggregation Semantic Instance Segmentation with a Discriminative Loss Function Holistic, Instance-Level Human Parsing Two-level attention and score consistency network for plant segmentation Automated polyp segmentation for colonoscopy images: A method based on convolutional neural networks and ensemble learning. Pixel Level Image Encryption Based on Semantic Segmentation Impact of Ground Truth Annotation Quality on Performance of Semantic Image Segmentation of Traffic Conditions Dynamic Scene Semantics SLAM Based on Semantic Segmentation Performance Evaluation of Deep Learning Networks for Semantic Segmentation of Traffic Stereo-Pair Images Real-Time CU-Net-Based Welding Quality Inspection Algorithm in Battery Production Analysis of nematodes in coffee crops at different altitudes using aerial images ShelfNet for Fast Semantic Segmentation Small Object Augmentation of Urban Scenes for Real-Time Semantic Segmentation Large-scale Landsat image classification based on deep learning methods Pyramid Scene Parsing Network ICNet for Real-Time Semantic Segmentation on High-Resolution Images Bin loss for hard exudates segmentation in fundus images Remote sensing semantic segmentation with convolution neural network using attention mechanism Optimization of Convolutional Neural Networks for Hedgerow Object Detection in Very High-Resolution Satellite Images Image semantic segmentation with an improved fully convolutional network Classification and Quantification of Retinal Cysts in OCT B-Scans: Efficacy of Machine Learning Methods Segmentation of Underwater Object in Videos Skin Lesion Segmentation Using Atrous Convolution via DeepLab v3 Adversarially Tuned Scene Generation Real-time use of artificial intelligence in the evaluation of cancer in Barrett’s oesophagus DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs A Two-Stage Framework for Automated Malignant Pulmonary Nodule Detection in CT Scans Rethinking Atrous Convolution for Semantic Image Segmentation Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation Multi-Scale Context Aggregation by Dilated Convolutions ROAD: Reality Oriented Adaptation for Semantic Segmentation of Urban Scenes Revisiting Unreasonable Effectiveness of Data in Deep Learning Era
116647544d4cd34929b8eca2b98e49460a9a67c8 5f1981d60b797050daea8a2c44add7fdd28f36ca 3fedd49241110b165b18221f46cd2b49d20a464e a11e093fa3466c5934749e1b9697de7800d78bac 32ae8f6d612500909920290774158b6e3a5ad34d 2b3751e0145fc72ba9acdf93c4c8aa81cdc6980f 53f3e676a1e7adc2b727a992a819352435365104 e2072af41bc2d944df1350453c92dc36ee712686 b2f888f2fd9afd497ca8d184893694d84cff3d06 9b8aac79ac9409ce232e4000fa4f20738e6956dd a74e6d5a0ce3ebf0ccc0186c0e8b8c2c2250a53c 795ed7600c0e707faa646e475f5bf378b718d3f0 f36fca9220fc0e23b8af6137ed55b4d58ed85892 88469ee8199b9082f279698509f7d06d9839808e c4fb7959455ab7cc511be108409becba48084072 6c6ebff31f91773b351a5f532f1536d7e68d7b8e a60f05edf9d794dd7f44feb35ffc85af43a9b14e bcdef2759701fd7490073583084da7eef4d2c2ba 1031eea8f62623195bc633f0ffab937f19bc3138 dc3cd4e110b526cb59bd7527d540120c5fae77ce 552a91bf2acacba136845c592ea9c119d3faddb3 cab372bc3824780cce20d9dd1c22d4df39ed081a 5313def35451285d8c6cbd9481d93583a5b25d65 ee4a012a4b12d11d7ab8c0e79c61e807927a163c
["Computer Science"] ["Engineering","Computer Science"] ["Physics"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science","Engineering"] ["Computer Science"] ["Computer Science"] ["Mathematics","Medicine","Computer Science"] ["Computer Science"] ["Computer Science","Mathematics"] ["Computer Science"] ["Computer Science"] ["Computer Science"] ["Computer Science","Medicine"] ["Engineering"] ["Computer Science"] ["Computer Science"] ["Medicine"] ["Computer Science","Medicine"] ["Computer Science","Medicine"] ["Computer Science"]
2019 2018 2017 2017 2020 2019 2018 2019 2020 2018 2020 2019 2018 2020 2019 2017 2018 2020 2019 2020 2020 2019 2018 2018 2017 2020 2018 2020 2017 2015 2016 2016 2018 2017
ICVS ArXiv ArXiv BMVC CompSysTech EUSIPCO 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ECCV EMBC CVPR ICLR 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) ICLR 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition 2017 IEEE International Conference on Computer Vision (ICCV)
https://api.semanticscholar.org/116647544d4cd34929b8eca2b98e49460a9a67c8 https://www.semanticscholar.org/paper/3a28fe49e7a856ddd60d134696a891ed7bca5962 https://www.semanticscholar.org/paper/1f8f0abfe4689aa93f2f6cc7ec4fd4c6adc2c2d6 https://www.semanticscholar.org/paper/52fedf3276c7eb67041a454848dbd94b0df3ab9c https://api.semanticscholar.org/5f1981d60b797050daea8a2c44add7fdd28f36ca https://api.semanticscholar.org/3fedd49241110b165b18221f46cd2b49d20a464e https://api.semanticscholar.org/a11e093fa3466c5934749e1b9697de7800d78bac https://api.semanticscholar.org/32ae8f6d612500909920290774158b6e3a5ad34d https://api.semanticscholar.org/2b3751e0145fc72ba9acdf93c4c8aa81cdc6980f https://api.semanticscholar.org/53f3e676a1e7adc2b727a992a819352435365104 https://api.semanticscholar.org/e2072af41bc2d944df1350453c92dc36ee712686 https://api.semanticscholar.org/b2f888f2fd9afd497ca8d184893694d84cff3d06 https://api.semanticscholar.org/9b8aac79ac9409ce232e4000fa4f20738e6956dd https://api.semanticscholar.org/a74e6d5a0ce3ebf0ccc0186c0e8b8c2c2250a53c https://api.semanticscholar.org/795ed7600c0e707faa646e475f5bf378b718d3f0 https://www.semanticscholar.org/paper/1031a69923b80ad01cf3fbb703d10757a80e699b https://www.semanticscholar.org/paper/35fe3dd3350c32467030884337dde10d5e20ff99 https://api.semanticscholar.org/f36fca9220fc0e23b8af6137ed55b4d58ed85892 https://api.semanticscholar.org/88469ee8199b9082f279698509f7d06d9839808e https://api.semanticscholar.org/c4fb7959455ab7cc511be108409becba48084072 https://api.semanticscholar.org/6c6ebff31f91773b351a5f532f1536d7e68d7b8e https://api.semanticscholar.org/a60f05edf9d794dd7f44feb35ffc85af43a9b14e https://api.semanticscholar.org/bcdef2759701fd7490073583084da7eef4d2c2ba https://api.semanticscholar.org/1031eea8f62623195bc633f0ffab937f19bc3138 https://api.semanticscholar.org/dc3cd4e110b526cb59bd7527d540120c5fae77ce https://api.semanticscholar.org/552a91bf2acacba136845c592ea9c119d3faddb3 https://api.semanticscholar.org/cab372bc3824780cce20d9dd1c22d4df39ed081a https://api.semanticscholar.org/5313def35451285d8c6cbd9481d93583a5b25d65 https://api.semanticscholar.org/ee4a012a4b12d11d7ab8c0e79c61e807927a163c https://www.semanticscholar.org/paper/39ad6c911f3351a3b390130a6e4265355b4d593b https://www.semanticscholar.org/paper/4cef5476f9da50c1a8fefdcb7114863966f61d67 https://www.semanticscholar.org/paper/7f5fc84819c0cf94b771fe15141f65b123f7b8ec https://www.semanticscholar.org/paper/0301963f73e8ace21653c0bed0e5feee128f140e https://www.semanticscholar.org/paper/8760bc7631c0cb04e7138254e9fd6451b7def8ca