Classifier generation methods are described in which features used in classification (e.g., mass spectral peaks) are selected, or deselected using bagged filtering. A development sample set is split into two subsets, one of which is used as a training set the other of which is set aside. We define a classifier (e.g., K-nearest neighbor, decision tree, margin-based classifier or other) using the training subset and at least one of the features (or subsets of two or more features in combination). We apply the classifier to a subset of samples. A filter is applied to the performance of the classifier on the sample subset and the at least one feature is added to a “filtered feature list” if the classifier performance passes the filter. We do this for many different realizations of the separation of the development sample set into two subsets, and, for each realization, different features or sets of features in combination. After all the iterations are performed the filtered feature list is used to either select features, or deselect features, for a final classifier.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method improving the functioning of a computer as a classifier by selecting or deselecting one or more features in a data set for generating the classifier, comprising the steps of: a) obtaining physical measurement data from a development set of samples and a class label for each of the samples in the development set, the physical measurement data comprising a feature value for a multitude of individual features; and, in a programmed computer performing steps b)-g): b) separating the data for the development set of samples into two subsets, one of which is used as a training set; c) defining a classifier using the training subset and at least one of the features; d) applying the classifier defined in step c) to the training subset; e) applying a filter to the performance of the classifier applied in step d), f) adding the at least one of the features used in step c) to a filtered feature list if the classifier performance passes the filter step e); g) repeating steps b), c), d), e) and f) for different realizations of the separation of the development set of samples into two subsets, and for different one or more features in the set of features; and h) using the filtered feature list to either select features or deselect features from the multitude of individual features for use in a final classifier generated from the development set of samples. 2. The method of claim 1, wherein the filter comprises a compound filter of two or more criteria separated by a logical operation. 3. The method of claim 2, wherein the filter comprises two classifier performance criteria separated by a logical AND operation and wherein at least one of the performance criteria is designed to deal with at least one of discrete classes, continuous variables, and censored time-to-event data. 4. The method of claim 1, wherein the samples comprise a set of blood-based samples and the measurement process comprises mass spectrometry. 5. The method of claim 1, wherein the samples comprise a set of samples from human patients and the measurement process comprises a genomic or proteomic assay. 6. The method of claim 1, wherein the samples comprise blood samples obtained from humans with liver disease. 7. The method of claim 1, wherein the filter comprises two classifier performance criteria separated by a logical AND operation and wherein one of the classifier performance criteria is classifier performance on a set of patient samples obtained from patients without cancer. 8. The method of claim 5, wherein the samples are obtained from patients with cancer, the assay is a genomic assay, and wherein the step h) is a selection of a set of features (genes) from the filtered feature list. 9. The method of claim 1, wherein the filter comprises two classifier performance criteria separated by a logical AND operation and wherein one of the classifier performance criteria is classification performance on a second set of samples separate from the development set of samples. 10. The method of claim 1, wherein the physical measurement process includes a first measurement process and a second measurement process different from the first measurement process. 11. The method of claim 1, wherein the first measurement process is mass spectrometry and the second measurement process is a genomic or proteomic assay. 12. The method of claim 1, wherein the separation of step b) is performed in a stratified manner based on a clinical consideration in the patient population from which the development set of samples is obtained. 13. The method of claim 1, wherein the filter includes a performance threshold of a hazard ratio between two classification groups. 14. The method of claim 1, wherein the samples are obtained from patients grouped in first and second treatment arm groups, and wherein the filter includes (1) classification performance in the form of a hazard ratio between two classes in the first treatment arm group, (2) classification performance in the form of a hazard ratio between the two classes in the second treatment arm group, and (3) a logical operator. 15. The method of claim 1, wherein the using step (h) comprises computing a weight for one or more features which pass the filtering step e). 16. The method of claim 15, wherein the weight is based on a diversity measure of the classifier defined in step c). 17. The method of claim 1, wherein the measurement method comprises mass spectrometry, the features comprise integrated intensity values at m/z ranges in a mass spectrum of the samples, and wherein the development set is separated into M different realizations of a training subset and a second subset, and wherein M is an integer greater than the number N of samples in the sample set. 18. The method of claim 17, wherein the number of features is greater than 100 and wherein in step (h) of the method of claim 1 the filtered feature list is used to select a subset of the greater than 100 features for the final classifier. 19. The method of claim 18, wherein in step (h) fewer than 10 features are selected. 20. The method of claim 17, wherein the number of features is greater than 100 and wherein in step (h) of the method of claim 1 the filtered feature list is used to deselect some of the greater than 100 features for use in the final classifier. 21. The method of claim 5, wherein the assay is an assay of mRNA expression levels. 22. The method of claim 21, and wherein the assay is of at least 1000 different genes, and wherein the step (h) fewer than 10 genes are selected for use in a final classifier. 23. The method of claim 21, wherein the samples are obtained from cancer patients. 24. A method of improving the functioning of a computer to generate a classifier, comprising the steps of: (a) obtaining a development set of samples; (b) conducting a physical measurement process on the development set of samples and obtaining measurement values; (c) supplying the measurement values to the computer; (d) with the computer conducting one or more pre-processing instructions on the measurement values; (e) creating a feature table for the samples in the form of a list of features and a measurement value for each of the features for each of the samples; (f) with the computer, conducting a bagged filtering process on the features of the feature table over many different realizations of a separation of the development set into subsets and over one or more sets of features to either identify a subset of features which have classification power or identify a subset of features which do not have classification power (“junky features”); and (g) defining a final classifier identifying (1) a subset of features as a result of the process (f), (2) parameters for a classification algorithm, and (3) a reference set for use with the classification algorithm comprising i) a set of samples from the development set, ii) their associated values for the set of features resulting from process (f) and iii) a class label for each of the samples in the reference set. 25. The method of claim 24, wherein the measurement process comprises mass spectrometry. 26. The method of claim 24, wherein the measurement process comprises a genomic or proteomic assay. 27. The method of claim 24, wherein the classification algorithm is in the form of a k-nearest neighbor classification algorithm. 28. The method of claim 24, wherein the final classifier is obtained from a classifier development exercise using a combination of mini-classifiers with drop-out regularization. 29. The method of claim 24, wherein the bagged filtering process (f) consists of steps b)-g) of claim 1. 30. A test method, comprising the steps of: (a) assaying a sample from a lung cancer patient for the expression level of a set of genes listed in Table 3, and (b) in a programmed computer comparing the expression levels to a reference set including expression levels of the same set of genes of step (a) from a multitude of other lung cancer patients with a classifier and generating a class label for the sample. 31. The test of claim 30, wherein the classifier is in the form of a k-nearest neighbor classifier. 32. A programmed computer comprising: a memory storing a the definition of filter in the form of at least one criterion for classifier performance, and instructions for a central processing unit of the computer for applying the filter to the performance of a classifier defined in the computer operating on many different realizations of a separation of a classifier development set into a training subset and a second subset and one or more measurement features of each member in the training subset; the memory further storing a filtered feature list listing the measurement features that pass the filter after execution of the instructions. 33. The programmed computer of claim 32, wherein the filter is in the form of a compound filter having two criteria for classifier performance and a logical operation. 34. The programmed computer of 33, wherein the classifier development set consists of measurement data of samples obtained from patients grouped in first and second treatment arm groups, and wherein the filter includes (1) classification performance in the form of a hazard ratio between two classes in the first treatment arm group, (2) classification performance in the form of a hazard ratio between the two classes in the second treatment arm group, and (3) a logical operator. 35. The programmed computer of claim 33, wherein the filter includes a classification performance criterion on a second set of samples other than the development set of samples. 36. The programmed computer of claim 33, wherein the measurement features comprise integrated intensity values at m/z ranges in a mass spectrum of each of the development set of samples, and wherein the development set is separated into M different realizations of a training subset and a second subset, and wherein M is an integer greater than the number N of samples in the development set of samples. 37. The method of claim 1, wherein in step c) the classifier is defined using two or more of the features in combination and wherein in step g) the process is repeated for each realization of the separation of the development set of samples into two subsets and, for each realization for all possible combinations of the two or more features. 38. The method of claim 24, wherein the one or more sets of features includes subsets of two or more of the features in combination.