Supervised deep learning methods require cleanly labeled large-scale datasets, but collecting such data is difficult and sometimes impossible. There exist two popular frameworks to alleviate this problem: semi-supervised learning and robust learning to label noise. Although these frameworks relax the restriction of supervised learning, they are studied independently. Hence, the training scheme that is suitable when only small cleanly-labeled data are available remains unknown. In this study, we consider learning from bi-quality data as a generalization of these studies, in which a small portion of data is cleanly labeled, and the rest is corrupt. Under this framework, we compare recent algorithms for semi-supervised and robust learning. The results suggest that semi-supervised learning outperforms robust learning with noisy labels. We also propose a training strategy for mixing mixup techniques to learn from such bi-quality data effectively. Learning from imperfect data is essential for applying machine learning, especially data-hungry deep learning, to real-world problems. One approach to handling this problem is semi-supervised learning (SSL), where training data consist of a small amount of labeled data and a large amount of unlabeled data. Another approach is robust learning to label noise (RLL), wherein all data are labeled, but some of them are mislabeled.SSL leverages large unlabeled data to improve the performance of supervised learning on a limited number of labeled data. In the context of deep SSL, one effective method is to train neural networks to maintain consistency for a small perturbation of unlabeled inputs BID7 ; BID10 ; BID11 ). BID8 refers these methods as consistency regularization.In the RLL setting, learners need to enhance their performance using corrupted labels and avoid the performance deterioration caused by such data. This requirement is particularly important for deep neural networks because they have ample capacity to remember whole samples even if their labels are completely random BID0 BID14 ). To tackle this problem, some methods use a small amount of clean data to estimate noise transition matrix BID12 ; BID2 ) or to learn to select possibly correctly-labeled samples BID4 ; BID3 ).Although both SSL and RLL aim to alleviate the limited-data problem, they have been studied independently and evaluated using different benchmarks. However, if only a small amount of clean data is available, they can be regarded as similar problems. In such as situation, can RLL outperform SSL under the same settings? This question was our initial motivation to unify these two lines of research.In this paper, we introduce a generalization of SSL and RLL, based on the concept of trusted data BID1 ; BID2 ) in the literature of RLL. More precisely , we assumed that some labels are guaranteed to be clean, and the rest are noisy. The two learning frameworks can be unified by controlling the ratio of corrupted labels to all labels and the noisiness of label corruption.Using the shared evaluation procedure in BID8 , we compared recent SSL and RLL algorithms using image classification task and found that the existing RLL methods using a small amount of clean data cannot outperform SSL under this setting. This finding suggests that such RLL algorithms cannot use noisy labels effectively. Therefore, it is necessary to adaptively use SSL and RLL in a data-driven manner. As a baseline learning algorithm , we propose combining the mixup losses for SSL BID11 ) and RLL BID15 ); the results obtained are comparable to those of SSL-and RLL-specific methods and indicate the effective use of useful information from noisy labels. In this paper, we introduce a novel framework of weakly supervised learning by unifying SSL and RLL, which have been independently studied. To handle this problem, we propose to mix mixup for SSL and RLL. This method empirically works well and achieves competitive results with semisupervised and robust learning specific methodologies.In addition, our experiments indicate that the performance of some RLL with trusted data might be inferior to that of SSL under identical settings. This result suggests that the existing RLL methods cannot effectively exploit the information which should be extracted from noisy labels.Our proposed method does not use the estimated quality; instead some hyperparameters are introduced. The use of quality estimation may ease hyperparameter tuning, but is still an open question.
We propose to compare semi-supervised and robust learning to noisy label under a shared setting The authors propose a strategy based on mixup for training a model in a formal setting that includes the semi-supervised and the robust learning tasks as special cases.