A mechanism is provided in a data processing system for assessing question answering system performance. The mechanism receives question answering system results. The question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer. The question answering system is trained or tested using the ground truth questions and answers. The mechanism performs a matching operation comparing each question in the question answering system results to questions in the ground truth. A given question is determined to be on-topic or off-topic based on results of the matching operation. For a plurality of confidence threshold values, the mechanism determines a rightness or wrongness of each answer in the question answering system results. The mechanism generates performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong. The mechanism presents the performance statistics to the user via a user interface.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method, in a data processing system, for assessing question answering system performance, the method comprising: receiving, by the data processing system, question answering system results, wherein the question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer, and wherein the question answering system is trained or tested using the ground truth questions and answers; performing, by the data processing system, a matching operation comparing each question in the question answering system results to questions in the ground truth, wherein a given question is determined to be on-topic or off-topic based on results of the matching operation; for a plurality of confidence threshold values, determining, by the data processing system, a rightness or wrongness of each answer in the question answering system results; generating, by the data processing system, performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong; and presenting, by the data processing system, the performance statistics to the user via a user interface. 2. The method of claim 1, wherein receiving question answering system results comprises submitting each question from the ground truth questions and answers to the question answering system and receiving an response answer from the question answering system for each submitted question and a confidence value associated with each response answer. 3. The method of claim 1, wherein receiving question answering system results comprises receiving question answering system log results via a question answering system log application programming interface. 4. The method of claim 1, wherein determining a rightness or wrongness of each answer in the question answering system results comprises performing a matching operation comparing each given answer in the question answering system results to a corresponding ground truth answer. 5. The method of claim 1, wherein generating performance statistics comprises generating a hybrid Receiver Operating Characteristics (ROC) curve that plots probability of a true answer versus probability of a false answer for the plurality of confidence threshold values. 6. The method of claim 5, wherein presenting the performance statistics to the user comprises: presenting the hybrid ROC curve on the user interface; and responsive to receiving a user selected confidence threshold value, marking a location on the hybrid ROC curve corresponding to the selected confidence threshold. 7. The method of claim 6, wherein presenting the performance statistics to the user comprises responsive to receiving the user selected confidence threshold value, presenting an on-topic correct answer probability value and an off-topic incorrect answer probability value corresponding to the selected confidence threshold. 8. The method of claim 1, wherein generating performance statistics comprises generating a correctness curve that plots precision versus questions answered for the plurality of confidence threshold values. 9. The method of claim 8, wherein presenting the performance statistics to the user comprises: presenting the correctness curve on the user interface; and responsive to receiving a user selected confidence threshold value, marking a location on the correctness curve corresponding to the selected confidence threshold. 10. The method of claim 9, wherein presenting the performance statistics to the user comprises responsive to receiving the user selected confidence threshold value, presenting an on-topic precision value and an on-topic attempted percentage value corresponding to the selected confidence threshold. 11. A computer program product comprising a computer readable storage medium having a computer readable program stored therein, wherein the computer readable program, when executed on a computing device, causes the computing device to: receive question answering system results, wherein the question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer, and wherein the question answering system is trained or tested using the ground truth questions and answers; perform a matching operation comparing each question in the question answering system results to questions in the ground truth, wherein a given question is determined to be on-topic or off-topic based on results of the matching operation; for a plurality of confidence threshold values, determine a rightness or wrongness of each answer in the question answering system results; generate performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong; and present the performance statistics to the user via a user interface. 12. The computer program product of claim 11, wherein receiving question answering system results comprises submitting each question from the ground truth questions and answers to the question answering system and receiving an response answer from the question answering system for each submitted question and a confidence value associated with each response answer. 13. The computer program product of claim 11, wherein receiving question answering system results comprises receiving question answering system log results via a question answering system log application programming interface. 14. The computer program product of claim 11, wherein determining a rightness or wrongness of each answer in the question answering system results comprises performing a matching operation comparing each given answer in the question answering system results to a corresponding ground truth answer. 15. The computer program product of claim 11, wherein generating performance statistics comprises generating a hybrid Receiver Operating Characteristics (ROC) curve that plots probability of a true answer versus probability of a false answer for the plurality of confidence threshold values. 16. The computer program product of claim 15, wherein presenting the performance statistics to the user comprises: presenting the hybrid ROC curve on the user interface; and responsive to receiving a user selected confidence threshold value, marking a location on the hybrid ROC curve corresponding to the selected confidence threshold. 17. The computer program product of claim 16, wherein presenting the performance statistics to the user comprises responsive to receiving the user selected confidence threshold value, presenting an on-topic correct answer probability value and an off-topic incorrect answer probability value corresponding to the selected confidence threshold. 18. The computer program product of claim 11, wherein generating performance statistics comprises generating a correctness curve that plots precision versus questions answered for the plurality of confidence threshold values. 19. The computer program product of claim 18, wherein presenting the performance statistics to the user comprises: presenting the correctness curve on the user interface; and responsive to receiving a user selected confidence threshold value, marking a location on the correctness curve corresponding to the selected confidence threshold. 20. The computer program product of claim 19, wherein presenting the performance statistics to the user comprises responsive to receiving the user selected confidence threshold value, presenting an on-topic precision value and an on-topic attempted percentage value corresponding to the selected confidence threshold. 21. An apparatus comprising: a processor; and a memory coupled to the processor, wherein the memory comprises instructions which, when executed by the processor, cause the processor to: receive question answering system results, wherein the question answering system results comprise questions posed to the question answering system, answers returned by the question answering system for each question posed to the question answering system, and a confidence value for each answer, and wherein the question answering system is trained or tested using the ground truth questions and answers; perform a matching operation comparing each question in the question answering system results to questions in the ground truth, wherein a given question is determined to be on-topic or off-topic based on results of the matching operation; for a plurality of confidence threshold values, determine a rightness or wrongness of each answer in the question answering system results; generate performance statistics for the plurality of confidence threshold values based on whether each question is on-topic or off-topic and whether each answer is right or wrong; and present the performance statistics to the user via a user interface.