A method and associated systems for forecastable supervised labels and corpus sets for training a natural-language processing system. An NLP-training system asks an “oracle” expert to answer a predictive test question and, in response, receives from the oracle an answer, rationales for selecting that answer, and identifications of extrinsic natural-language sources of evidence that supports those rationales. The system retrieves updated versions of that evidence at a later time, and returns that updated evidence to the oracle. In response, the oracle returns an updated answer and rationales based on the updated evidence. The system then compares time-varying characteristics of the evidence in order to determine the relative contributions of each piece of evidence to the oracles' selections. Less relevant evidence is discarded and the remaining, optimized, evidence is forwarded to the NLP system to be used as training data.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. An NLP-training system comprising a processor, a memory coupled to the processor, and a computer-readable hardware storage device coupled to the processor, the storage device containing program code configured to be run by the processor via the memory to implement a method for forecastable supervised labels and corpus sets for training a natural-language processing system, the method comprising: the training system selecting an oracle, wherein an oracle is a human or computerized expert in a particular field of endeavor; the training system receiving from the oracle a first label and a first set of rationales, wherein the oracle selected the first label and the first set of rationales at a first time as a function of natural-language content stored at one or more extrinsic electronic sources, wherein the first label identifies a correct answer to a predictive question, wherein the first set of rationales identifies one or more reasons why the oracle selected the first label, and wherein answering the predictive question comprises predicting a future occurrence; the training system adding the natural-language content to a first set of corpora; the training system retrieving from the one or more extrinsic electronic sources, at a second time, a later version of the natural-language content; the training system creating a second set of corpora by adding to the first set of corpora the later version of the natural-language content; the training system communicating the second set of corpora to the oracle; the training system accepting from the oracle, in response to the communicating, a second label and a second set of rationales; and the training system eliminating less relevant datasets from the second set of corpora as a function of the relative relevance of each dataset of the second set of corpora to the oracle's selection of the second label and the second set of rationales. 2. The NLP-training system of claim 1, wherein the second label and the second set of rationales are, respectively, updated versions of the first label and the first set of rationales, and wherein the oracle selected the second label and the second set of rationales as a function of a change in the natural-language content that occurred between the first time and the second time. 3. The NLP-training system of claim 1, wherein a first corpus of the first set of corpora comprises a first set of datasets of a plurality of sets of datasets, and wherein every dataset of the first set of datasets comprises a subset of the natural-language content retrieved at the first time from a first source of the one or more extrinsic electronic sources. 4. The NLP-training system of claim 3, wherein all natural-language content stored in a corpus of the second set of corpora is derived from a single source of the one or more extrinsic electronic sources; wherein all content stored in a dataset of a corpus of the second set of corpora was derived at a same time from a same source of the one or more extrinsic electronic sources, wherein each document of a later dataset of the second set of corpora that was retrieved at the second time from a particular source of the one or more extrinsic electronic sources is a later version of a same document comprised by an earlier dataset of the second set of corpora that was retrieved from the particular source at the first time, and wherein the adding the later version of the natural-language content to the first set of corpora comprises adding each dataset retrieved at the second time from a specific source of the one or more extrinsic electronic sources to a corpus of the first set of corpora that comprises datasets retrieved at the first time from the specific source. 5. The NLP-training system of claim 3, wherein the eliminating further comprises: the system determining a relative contribution of each dataset comprised by the second set of corpora to the oracle's selection of the second label and the second rationales; and the system identifying a dataset comprised by the second set of corpora as being less relevant if that dataset's relative contribution falls below a threshold value. 6. The NLP-training system of claim 5, wherein the determining a relative contribution is performed as a function of a value of a lag variable, and wherein a lag variable measures a pattern of change in a time-varying characteristic of datasets of a corpus over a period of time. 7. The NLP-training system of claim 6, wherein the determining a relative contribution further comprises: the system transmitting to the oracle metadata identified by time-stamped values of the lag variable, wherein the time-stamped values identify characteristics of datasets that comprise natural-language content retrieved at the first time or retrieved at the second time; and the system receiving from the oracle, in response to the transmitting, an identification of a relative contribution of each dataset of the second set of corpora. 8. A method for forecastable supervised labels and corpus sets for training a natural-language processing system, the method comprising: an NLP-training system selecting an oracle, wherein an oracle is a human or computerized expert in a particular field of endeavor; the training system receiving from the oracle a first label and a first set of rationales, wherein the oracle selected the first label and the first set of rationales at a first time as a function of natural-language content stored at one or more extrinsic electronic sources, wherein the first label identifies a correct answer to a predictive question, wherein the first set of rationales identifies one or more reasons why the oracle selected the first label, and wherein answering the predictive question comprises predicting a future occurrence; the training system adding the natural-language content to a first set of corpora; the training system retrieving from the one or more extrinsic electronic sources, at a second time, a later version of the natural-language content; the training system creating a second set of corpora by adding to the first set of corpora the later version of the natural-language content; the training system communicating the second set of corpora to the oracle; the training system accepting from the oracle, in response to the communicating, a second label and a second set of rationales; and the training system eliminating less relevant datasets from the second set of corpora as a function of the relative relevance of each dataset of the second set of corpora to the oracle's selection of the second label and the second set of rationales. 9. The method of claim 8, wherein the second label and the second set of rationales are, respectively, updated versions of the first label and the first set of rationales, and wherein the oracle selected the second label and the second set of rationales as a function of a change in the natural-language content that occurred between the first time and the second time. 10. The method of claim 8, wherein a first corpus of the first set of corpora comprises a first set of datasets of a plurality of sets of datasets, and wherein every dataset of the first set of datasets comprises a subset of the natural-language content retrieved at the first time from a first source of the one or more extrinsic electronic sources. 11. The method of claim 10, wherein all natural-language content stored in a corpus of the second set of corpora is derived from a single source of the one or more extrinsic electronic sources; wherein all content stored in a dataset of a corpus of the second set of corpora was derived at a same time from a same source of the one or more extrinsic electronic sources, wherein each document of a later dataset of the second set of corpora that was retrieved at the second time from a particular source of the one or more extrinsic electronic sources is a later version of a same document comprised by an earlier dataset of the second set of corpora that was retrieved from the particular source at the first time, and wherein the adding the later version of the natural-language content to the first set of corpora comprises adding each dataset retrieved at the second time from a specific source of the one or more extrinsic electronic sources to a corpus of the first set of corpora that comprises datasets retrieved at the first time from the specific source. 12. The method of claim 10, wherein the eliminating further comprises: the system determining a relative contribution of each dataset comprised by the second set of corpora to the oracle's selection of the second label and the second rationales; and the system identifying a dataset comprised by the second set of corpora as being less relevant if that dataset's relative contribution falls below a threshold value. 13. The method of claim 12, wherein the determining a relative contribution is performed as a function of a value of a lag variable, and wherein a lag variable measures a pattern of change in a time-varying characteristic of datasets of a corpus over a period of time. 14. The method of claim 13, wherein the determining a relative contribution further comprises: the system transmitting to the oracle metadata identified by time-stamped values of the lag variable, wherein the time-stamped values identify characteristics of datasets that comprise natural-language content retrieved at the first time or retrieved at the second time; and the system receiving from the oracle, in response to the transmitting, an identification of a relative contribution of each dataset of the second set of corpora. 15. The method of claim 8, further comprising providing at least one support service for at least one of creating, integrating, hosting, maintaining, and deploying computer-readable program code in the computer system, wherein the computer-readable program code in combination with the computer system is configured to implement the selecting, receiving, adding, retrieving, creating, communicating, accepting, and eliminating. 16. A computer program product, comprising a computer-readable hardware storage device having a computer-readable program code stored therein, the program code configured to be executed by an NLP-training system comprising a processor, a memory coupled to the processor, and a computer-readable hardware storage device coupled to the processor, the storage device containing program code configured to be run by the processor via the memory to implement a method for forecastable supervised labels and corpus sets for training a natural-language processing system, the method comprising: the training system selecting an oracle, wherein an oracle is a human or computerized expert in a particular field of endeavor; the training system receiving from the oracle a first label and a first set of rationales, wherein the oracle selected the first label and the first set of rationales at a first time as a function of natural-language content stored at one or more extrinsic electronic sources, wherein the first label identifies a correct answer to a predictive question, wherein the first set of rationales identifies one or more reasons why the oracle selected the first label, and wherein answering the predictive question comprises predicting a future occurrence; the training system adding the natural-language content to a first set of corpora, wherein a first corpus of the first set of corpora comprises a first set of datasets of a plurality of sets of datasets, and wherein every dataset of the first set of datasets comprises a subset of the natural-language content retrieved at the first time from a first source of the one or more extrinsic electronic sources; the training system retrieving from the one or more extrinsic electronic sources, at a second time, a later version of the natural-language content; the training system creating a second set of corpora by adding to the first set of corpora the later version of the natural-language content; the training system communicating the second set of corpora to the oracle; the training system accepting from the oracle, in response to the communicating, a second label and a second set of rationales; and the training system eliminating less relevant datasets from the second set of corpora as a function of the relative relevance of each dataset of the second set of corpora to the oracle's selection of the second label and the second set of rationales. 17. The computer program product of claim 16, wherein the second label and the second set of rationales are, respectively, updated versions of the first label and the first set of rationales, and wherein the oracle selected the second label and the second set of rationales as a function of a change in the natural-language content that occurred between the first time and the second time. 18. The computer program product of claim 16, wherein all natural-language content stored in a corpus of the second set of corpora is derived from a single source of the one or more extrinsic electronic sources; wherein all content stored in a dataset of a corpus of the second set of corpora was derived at a same time from a same source of the one or more extrinsic electronic sources, wherein each document of a later dataset of the second set of corpora that was retrieved at the second time from a particular source of the one or more extrinsic electronic sources is a later version of a same document comprised by an earlier dataset of the second set of corpora that was retrieved from the particular source at the first time, and wherein the adding the later version of the natural-language content to the first set of corpora comprises adding each dataset retrieved at the second time from a specific source of the one or more extrinsic electronic sources to a corpus of the first set of corpora that comprises datasets retrieved at the first time from the specific source. 19. The computer program product of claim 16, wherein the eliminating further comprises: the system determining a relative contribution of each dataset comprised by the second set of corpora to the oracle's selection of the second label and the second rationales, wherein the determining a relative contribution is performed as a function of a value of a lag variable, and wherein a lag variable measures a pattern of change in a time-varying characteristic of datasets of a corpus over a period of time; and the system identifying a dataset comprised by the second set of corpora as being less relevant if that dataset's relative contribution falls below a threshold value. 20. The computer program product of claim 19, wherein the determining a relative contribution further comprises: the system transmitting to the oracle metadata identified by time-stamped values of the lag variable, wherein the time-stamped values identify characteristics of datasets that comprise natural-language content retrieved at the first time or retrieved at the second time; and the system receiving from the oracle, in response to the transmitting, an identification of a relative contribution of each dataset of the second set of corpora.