A method and system are provided for combining models. The method includes forming, by a computer having a processor and a memory, model pairs from a model ensemble that includes a plurality of models. The method further includes comparing the model pairs based on sets of output results produced by the model pairs to provide comparison results. The method also includes constructing, by the computer, a combination model from at least one of the model pairs based on the comparison results. The comparing step is performed using user-generated set-based feedback.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method for combining models, comprising: forming, by a computer having a processor and a memory, model pairs from a model ensemble that includes a plurality of models; comparing the model pairs based on sets of output results produced by the model pairs to provide comparison results; and constructing, by the computer, a combination model from at least one of the model pairs based on the comparison results, wherein said comparing step is performed using user-generated set-based feedback. 2. The method of claim 1, where said comparing step comprises: displaying the sets of output results for the model pairs to a user; and receiving an indication from the user of which of the sets of output results provides best results for an intended purpose. 3. The method of claim 1, wherein said comparing step comprises receiving annotations from a user regarding the sets of output results produced by the model pairs, and wherein the method further comprising training, by the computer, an automated machine learning process with the annotations such the automated machine learning process reproduces applicable ones of the annotations for other model pairs. 4. The method of claim 3, wherein the automated machine learning process is trained with the annotations to enable the automated machine learning process to reproduce, without user intervention, the applicable ones of the annotations for the other model pairs. 5. The method of claim 3, wherein the annotations comprise win annotations and loss annotations from a model pairing evaluation scheme applied to the model pairs. 6. The method of claim 1, wherein said comparing step further comprises constructing a schedule of pairwise model comparisons whereby only the winners of each of the pairwise model comparisons are further compared. 7. The method of claim 1, wherein said constructing step further comprises assigning predefined scores to each of the plurality of models included in the ensemble based on wins and losses occurring in a model elimination process applied to the plurality of models. 8. The method of claim 7, wherein the model elimination process is a model combination elimination process that eliminates models from being combined with other models. 9. The method of claim 1, where said constructing step further comprises training an automated algorithm with win annotations and loss annotations produced by a user for the models in the models pairs, such that said automated algorithm can reproduce decisions made by the user. 10. The method of claim 1, wherein the plurality of models are machine learning models. 11. The method of claim 1, further comprising providing, based on the combination model, at least one of (i) a search result responsive to a search query, (ii) a response responsive to a question query, and (iii) a classification of an object responsive to a classification query. 12. The method of claim 1, further comprising constructing a subject matter classifier computing system using the combination model. 13. The method of claim 1, wherein said comparing step further comprises: computing a similarity measure between the model pairs based on one or more criterion; and splitting, based on the similarity, a fixed score between a winner and a loser in a given round in a model comparison process into two split scores, wherein the two split scores are used to determine the combination model constructed in said constructing step. 14. The method of claim 1, further comprising constructing a cognitive-based computer processing system using the combination model. 15. A non-transitory computer readable storage medium comprising a computer readable program for combining models, wherein the computer readable program when executed on a computer causes the computer to perform the steps of: forming model pairs from a model ensemble that includes a plurality of models; and constructing a combination model from at least one of the model pairs based on comparison results, wherein the comparison results are obtained by comparing the model pairs based on sets of output results produced by the model pairs using user-generated set-based feedback. 16. The non-transitory computer readable storage medium of claim 15, wherein said comparing step comprises receiving annotations from a user regarding the sets of output results produced by the model pairs, and wherein the method further comprising training, by the computer, an automated machine learning process with the annotations such the automated machine learning process reproduces applicable ones of the annotations for other model pairs. 17. The non-transitory computer readable storage medium of claim 16, wherein the automated machine learning process is trained with the annotations to enable the automated machine learning process to reproduce, without user intervention, the applicable ones of the annotations for the other model pairs. 18. The non-transitory computer readable storage medium of claim 15, wherein said constructing step further comprises assigning predefined scores to each of the plurality of models included in the ensemble based on wins and losses occurring in a model elimination process applied to the plurality of models, and wherein the model elimination process is a model combination elimination process that eliminates models from being combined with other models. 19. A system, comprising: a computer, having a processor and a memory, configured to: form model pairs from a model ensemble that includes a plurality of models; and construct a combination model from at least one of the model pairs based on comparison results, wherein the comparison results are obtained by comparing the model pairs based on sets of output results produced by the model pairs using user-generated set-based feedback. 20. The system of claim 19, wherein the computer is implemented as a server using a cloud computing configuration.