A method, system and computer-usable medium are disclosed for reducing labeled data imbalances when training an active learning system. The ratio of instances having positive labels or negative labels in a collection of labeled instances associated with an input category used for learning is determined. A first instance for annotation is selected from a collection of unlabeled instances if a first threshold for negative instances, and a first threshold confidence level of being a positive instance of the input category, have been met. A second instance for annotation is selected if a second threshold for positive instances, and a second threshold confidence level of being a negative instance of the input category, have been met. The first and second instances are respectively annotated with a positive and negative label and added to the collection of labeled instances, which are then used for training.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A computer-implemented method for active machine learning, comprising: determining a ratio of positive instances to negative instances in a collection of labeled instances associated with an input category, the positive instances and the negative instances being annotated to indicate whether an instance is a positive instance associated with the input category or a negative instance associated with the input category, the positive instances and the negative instances being used for learning by an active learning system; selecting a first instance for annotation from a collection of unlabeled instances if a first threshold for negative instances is met, the first instance meeting a first threshold confidence level of being a positive instance of the input category; and selecting a second instance for annotation from the collection of unlabeled instances if a second threshold for positive instances is met, the second instance meeting a second threshold confidence level of being a negative instance of the input category. 2. The method of claim 1, wherein: the first instance is an unlabeled instance having a highest confidence level of being a positive instance for the input category; and the second instance is an unlabeled instance having a highest confidence level of being a negative instance for the input category. 3. The method of claim 1, wherein: the first threshold is a ratio of a number of labeled negative instances divided by a number of labeled positive instances; and the second threshold is a ratio of a number of labeled positive instances divided by a number of labeled negative instances. 4. The method of claim 1, wherein: a classification of an unlabeled instance as a negative or a positive instance is performed by a semantic search technique. 5. The method of claim 4, wherein: the semantic search technique is performed by using distributional semantics and a search engine. 6. The method of claim 4, wherein: the classification is also performed in part by a machine learning based supervised classifier. 7. The method of claim 1, further comprising: annotating the first instance with a positive label, the annotated first instance added to the collection of labeled instances; and annotating the second instance with a negative label, the annotated second instance added to the collection of labeled instances. 8. The method of claim 1, further comprising: using the collection of annotated instances to train a machine learning system if a relatively equal number of positive instances and negative instances have been annotated. 9. A system comprising: a processor; a data bus coupled to the processor; and a computer-usable medium embodying computer program code, the computer-usable medium being coupled to the data bus, the computer program code used for active machine learning and comprising instructions executable by the processor and configured for: determining a ratio of positive instances to negative instances in a collection of labeled instances associated with an input category, the positive instances and the negative instances being annotated to indicate whether an instance is a positive instance associated with the input category or a negative instance associated with the input category, the positive instances and the negative instances being used for learning by an active learning system; selecting a first instance for annotation from a collection of unlabeled instances if a first threshold for negative instances is met, the first instance meeting a first threshold confidence level of being a positive instance of the input category; and selecting a second instance for annotation from the collection of unlabeled instances if a second threshold for positive instances is met, the second instance meeting a second threshold confidence level of being a negative instance of the input category. 10. The system of claim 7, wherein: the first instance is an unlabeled instance having a highest confidence level of being a positive instance for the input category; and the second instance is an unlabeled instance having a highest confidence level of being a negative instance for the input category. 11. The system of claim 7, wherein: the first threshold is a ratio of a number of labeled negative instances divided by a number of labeled positive instances; and the second threshold is a ratio of a number of labeled positive instances divided by a number of labeled negative instances. 12. The system of claim 7, wherein: a classification of an unlabeled instance as a negative or a positive instance is performed by a semantic search technique. 13. The system of claim 12, wherein: the semantic search technique is performed by using distributional semantics and a search engine. 14. The system of claim 12, wherein: the classification is also performed in part by a machine learning based supervised classifier. 15. The system of claim 7, further comprising: annotating the first instance with a positive label, the annotated first instance added to the collection of labeled instances; and annotating the second instance with a negative label, the annotated second instance added to the collection of labeled instances. 16. The system of claim 7, further comprising: using the collection of annotated instances to train a machine learning system if a relatively equal number of positive instances and negative instances have been annotated. 17. A non-transitory, computer-readable storage medium embodying computer program code, the computer program code comprising computer executable instructions configured for: determining a ratio of positive instances to negative instances in a collection of labeled instances associated with an input category, the positive instances and the negative instances being annotated to indicate whether an instance is a positive instance associated with the input category or a negative instance associated with the input category, the positive instances and the negative instances being used for learning by an active learning system; selecting a first instance for annotation from a collection of unlabeled instances if a first threshold for negative instances is met, the first instance meeting a first threshold confidence level of being a positive instance of the input category; and selecting a second instance for annotation from the collection of unlabeled instances if a second threshold for positive instances is met, the second instance meeting a second threshold confidence level of being a negative instance of the input category. 18. The non-transitory, computer-readable storage medium of claim 13, wherein: the first instance is an unlabeled instance having a highest confidence level of being a positive instance for the input category; and the second instance is an unlabeled instance having a highest confidence level of being a negative instance for the input category. 19. The non-transitory, computer-readable storage medium of claim 13, wherein: the first threshold is a ratio of a number of labeled negative instances divided by a number of labeled positive instances; and the second threshold is a ratio of a number of labeled positive instances divided by a number of labeled negative instances. 20. The non-transitory, computer-readable storage medium of claim 13, wherein: a classification of an unlabeled instance as a negative or a positive instance is performed by a semantic search technique. 21. The non-transitory, computer-readable storage medium of claim 20, wherein: the semantic search technique is performed by using distributional semantics and a search engine. 22. The non-transitory, computer-readable storage medium of claim 20, wherein: the classification is also performed in part by a machine learning based supervised classifier. 23. The non-transitory, computer-readable storage medium of claim 13, further comprising: annotating the first instance with a positive label, the annotated first instance added to the collection of labeled instances; and annotating the second instance with a negative label, the annotated second instance added to the collection of labeled instances. 24. The non-transitory, computer-readable storage medium of claim 13, further comprising: using the collection of annotated instances to train a machine learning system if a relatively equal number of positive instances and negative instances have been annotated. 25. The non-transitory, computer-readable storage medium of claim 13, wherein the computer executable instructions are deployable to a client system from a server system at a remote location. 26. The non-transitory, computer-readable storage medium of claim 13, wherein the computer executable instructions are provided by a service provider to a user on an on-demand basis.