An example method is provided in according with one implementation of the present disclosure. The method comprises receiving a training dataset of cases, where each of a plurality of classes is associated with a set of labeled cases in the training dataset. The method also comprises defining a proper subset of classes in the training dataset, and training a first classifier model on the proper subset of classes in the training dataset. The method further comprises testing the first classifier model on at least one class in the training dataset that was excluded from the proper subset, and determining a performance measurement of the first classifier model.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method comprising, by at least one processor: receiving a training dataset of cases, wherein each of a plurality of classes is associated with a set of labeled cases in the training dataset; defining a proper subset of classes in the training dataset; training a first classifier model on the proper subset of classes in the training dataset; testing the first classifier model on at least one class in the training dataset that was excluded from the proper subset; and determining a performance measurement of the first classifier model. 2. The method claim 1, further comprising, by at least one processor: determining a performance measurement of at least a second classifier model; comparing the performance measurement of the first classifier model and the second classifier model; and selecting a best performing classifier model based on the comparison. 3. The method of claim 1, wherein defining the proper subset of classes is performed with random selection of classes. 4. The method of claim 1, wherein the training dataset includes at least three classes. 5. The method of claim 1, further comprising, by at least one processor: partitioning the classes in the training dataset into a plurality of groups. 6. The method of claim 5, further comprising, by at least one processor: partitioning the labeled cases within each class into a plurality of partitions. 7. The method of claim 6, further comprising, by at least one processor: selecting a partition of cases as a test partition; selecting a group of classes to be excluded from the proper subset of classes, wherein the excluded group represents classes that are to be unfamiliar to the classifier model; training the first classifier model on labeled cases of the training set that are not in the selected group and the selected test partition; testing the first classifier model on the test cases of the test partition, wherein classes in the excluded group of classes are considered the cases that should be considered unfamiliar to the classifier model; and determining a performance measurement of the first classifier model. 8. The method of claim 7, further comprising, by at least one processor: determining a performance measurement of the first classifier model for a plurality of folds; aggregating the performance measurement of the first classifier model from the plurality of folds; and outputting the aggregated performance measurement of the first classifier model. 9. A system comprising: a dataset engine to: receive a training dataset of cases, wherein each of a plurality of classes is associated with a set of labeled cases in the training dataset, and define a proper subset of classes in the training dataset; a testing engine to: train at least a first classifier model and a second classifier model on the proper subset of classes in the training dataset, and test the first classifier model and the second classifier model on at least one class in the training dataset that was excluded from the proper subset; and an analysis engine to: determine a performance measurement of the first classifier model and the second classifier model; compare the performance measurement of the first classifier model and the second classifier model; select a best performing classifier model based on the comparison. 10. The system of claim 9, wherein the testing engine is further to: partition the labeled cases within each class into a plurality of partitions; and partition the classes in the training dataset into a plurality of groups. 11. The system of claim 10, wherein the testing engine is further to: select a partition of cases as a test partition; select a group of classes to be excluded from the proper subset of classes, wherein the excluded group represents classes that are to be unfamiliar to the classifier model; train a classifier model on labeled cases of the training set that are not in the selected group and the selected test partition; test the classifier model on the test cases of the test partition, where classes in the excluded group of classes are considered the cases that should be considered unfamiliar of the classifier model; and determine a performance measurement of the classifier model. 12. The system of claim 11, wherein the analysis engine is further to: determine a performance measurement of the classifier model for a plurality of folds; aggregate the performance measurement of the classifier model from the plurality of folds; and output the aggregated performance measurement of the classifier model. 13. A non-transitory machine-readable storage medium encoded with instructions executable by at least one processor, the machine-readable storage medium comprising instructions to: receive a training dataset of cases, wherein each of a plurality of classes is associated with a set of labeled cases in the training dataset; define a proper subset of classes in the training dataset by using random selection of classes; train a first classifier model on the proper subset of classes in the training dataset; test the first classifier model on at least one class in the training dataset that was excluded from the proper subset; and determine a performance measurement of the first classifier model. 14. The non-transitory machine-readable storage medium of claim 13, further comprising instructions to: determine a performance measurement of at least a second classifier model; compare the performance measurement of the first classifier model and the second classifier model; and select a best performing model based on the comparison. 15. The non-transitory machine-readable storage medium of claim 13, further comprising instructions to: partition the labeled cases within each class into a plurality of partitions; partition the classes in the training dataset into a plurality of groups; select a partition of cases as a test partition; select a group of classes to be excluded from the proper subset of classes, wherein the excluded group represents classes that are to be unfamiliar to the classifier model; train the first classifier model on labeled cases of the training set that are not in the selected group and the selected test partition; test the first classifier model on the test cases of the test partition, where classes in the excluded group of classes are considered the cases that should be considered unfamiliar to the classifier model; and determine a performance measurement of the first classifier model.