<p>There might be several approaches to solving your problem.</p>

<p>First - it might not be a problem after all. If the mislabeled data accounts for a small part of your training set, it might not matter. Actually, there are some cases when adding mislabeled data or just random noise improves robustness and generalization power of your classifier.</p>

<p>Second - you might want to use the training set to train the classifier and then check the data points for which the classifier gave the incorrect classification. It is possible that the classifier was actually right and directs you to the incorrectly labeled data. This data can be subsequently manually checked if such a thing is possible.</p>

<p>Third - you can filter the data up front using methods like consensus filters. This article might be a good way to start your research on this topic: <a href="https://arxiv.org/pdf/1106.0219" rel="nofollow noreferrer">Identifying Mislabeled Training Data - C.E. Brody and M.A. Friedl</a>.</p>
Machine learning where labelling of training data might not be 100% accurate
<p>I have a dataset which consists of people who have diabetes, and who have not. Using this data, I want to train a model to calculate a risk probability for people with unknown diabetes status. I know that the majority of people who have not been diagnosed with diabetes in the training  do not have diabetes, but it is likely that some of these people may have undiagnosed diabetes.</p>

<p>This appears to present a catch 22 situation. I want to identify people who are at-risk, or potentially have undiagnosed diabetes, however I know some of the people in my training dataset are incorrectly labelled as not having diabetes because they have not yet been diagnosed. Has anyone encountered such a problem? Can one still proceed on the basis that there may be some incorrectly labelled data, if it only counts for a small percentage of the data?</p>