Testing machine learning sensors by adding obfuscated training data to test data, and performing real time model fit analysis on live network traffic to determine whether to retrain.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A process for improving a machine learning sensor for defending a computer network, comprising: obtaining samples of normal network traffic from said network; providing samples of cyber attacks from said network or from a repository of cyber attacks; classifying each of said samples as either said normal network traffic or said cyber attacks, to create ground truths for said samples; splitting said samples into a training set and a test set, with each of said sets containing samples of said normal network traffic and of said cyber attacks; using a model generating algorithm to generate a variety of models for distinguishing between said normal network traffic and said cyber attacks in said training set; obfuscating a portion of samples of said cyber attacks in said training set to create obfuscated attack samples; adding said obfuscated attack samples to said test set to form an enhanced test set; performing statistical analysis on performance of said models with said enhanced test set to determine intrusion detection capability error; selecting one of said models that optimizes a desired model parameter as an optimal model for distinguishing between said normal network traffic and said cyber attacks; and installing said optimal model in said sensor. 2. A process according to claim 1, further comprising: determining model fit of said sensor by applying anomaly detection, measuring similarity between live network traffic and said training set, and determining model overfit; assigning thresholds and aggregating results of said thresholds to identify model fit in real time on a scale of model fits; and activating model retraining based on said scale of model fits. 3. A process according to claim 1, wherein said intrusion detection capability error is the sum of the following: ratio of misclassified known attacks to total misclassified attacks, multiplied by test error of known attacks; ratio of misclassified unknown attacks to total misclassified attacks, multiplied by test error of unknown attacks; and ratio of misclassified obfuscated attacks to total misclassified attacks, multiplied by test error of obfuscated attacks, 4. A process for improving sensors using machine learning models for defending a computer network, comprising: installing said sensors in said computer network; determining model fit of said models by applying techniques of anomaly detection, measuring similarity between live network traffic and training data, and determining model overfit; assigning thresholds and aggregating results of said thresholds for each of said techniques, to identify model fit in real time on a scale of model fits; and activating model retraining based on said scale of model fits. 5. A process according to claim 4, wherein said determining of model fit is performed using a technique selected from the group consisting of chi-squared test, Kolmogorov-Smirnov test, coefficient of determination, and lack of fit sum of squares. 6. A process according to claim 4, wherein said determining model fit is performed using generalization complexity measures selected from the group consisting of Rademacher Complexity, Kolmogorov Complexity, and Vapnik Chervonenkis Entropy. 7. A process according to claim 4, wherein said assigning thresholds on a scale of model fits is performed using a scale of model fits in which: model fit is determined to be good if all three of said techniques are below said thresholds; model fit is determined to be deteriorating if one of said three is above said threshold; model fit is determined to be lacking if two of said three are above said threshold; model fit is determined to be extremely lacking if all of said three are above said threshold. 8. A process according to claim 7, wherein said activating of model retraining is based on the aggregate of said goodness of model fit measurements for all 3 of said techniques. 9. A process according to claim 7, wherein said activating of model retraining is based on goodness of model fit for only one of said techniques. 10. A process according to claim 1, wherein said selecting step is performed using a desired model parameter selected from the group consisting of intrusion detection capability error and test error. 11. A system, comprising; a network of computers; and a machine-learning based sensor deployed in said network, wherein said machine-learning based sensor has been trained by a process comprising: obtaining samples of normal network traffic from said network; providing samples of cyber attacks from said, network or from a repository of cyber .attacks; classifying each of said samples as either said normal network traffic or said cyber attacks, to create ground truths for said samples; splitting said samples info a training set and a test set, with each of said sets containing samples of said normal network, traffic and of said cyber attacks; using a model generating algorithm to generate a variety of models for distinguishing between said normal network traffic and said cyber attacks in said training set; obfuscating a portion of samples of said cyber attacks in said training set. to create obfuscated attack samples; adding said obfuscated attack samples to said test set to form an enhanced test set; performing statistical analysis on performance of said models with said enhanced test set to determine intrusion detection capability error; selecting one of said models that optimizes a desired model parameter as an optimal model for distinguishing between said normal network traffic and said cyber attacks. 12. A system according to claim 11, wherein said selecting step is performed using a desired model parameter selected from the group consisting of intrusion detection capability error and test error, 13. A computer program product stored in a computer readable medium for protecting a network, comprising: a machine learning based sensor that has been trained by a process comprising: obtaining samples of normal network traffic from said network; providing samples of cyber attacks from said network, or from a repository of cyber attacks; classifying each of said samples as either said normal network traffic or said cyber attacks, to create ground truths for said samples; splitting said samples into a training set and a test set, with each of said sets containing samples of said normal network traffic and of said cyber attacks; using a model generating algorithm to generate a variety of models for distinguishing between said normal network traffic and said cyber attacks in said training set; obfuscating a portion of samples of said cyber attacks in said training set to create obfuscated attack samples; adding said obfuscated attack samples to said test set to form an enhanced test set; performing statistical analysis on performance of said models with said enhanced test set to determine intrusion detection capability error; selecting one of said models that optimizes a desired model parameter as an optimal model for distinguishing between said normal network traffic and said cyber attacks. 14. A computer program product according to claim 12, wherein said selecting step is performed using a desired model parameter selected from the group consisting of intrusion detection capability error and test error. 15. A process for improving a machine learning sensor, comprising; obtaining samples of normal, data from a set of data containing normal and abnormal data; providing samples of abnormal data from said set or from a repository of abnormal data; classifying each of said samples as either said normal data or said abnormal data, to create ground truths for said samples; splitting said samples into a training set and a test set, with each of said sets containing samples of said normal data and of said abnormal data; using a model generating algorithm to generate a variety of models for distinguishing between said normal data and said abnormal data in said training set; obfuscating a portion of samples of said abnormal data in said training set to create obfuscated abnormal data samples; adding said obfuscated abnormal data samples to said test set to form an enhanced test set; performing statistical analysis on performance of said models with said enhanced, test set to determine intrusion detection capability error; selecting one of said models that optimizes a desired model parameter as an optimal model for distinguishing between said normal data and said abnormal data; and installing said optimal model in said sensor, 16. A system, comprising: a computer; and a machine-learning based sensor deployed in said computer, wherein said machine-learning based sensor has been trained by a process comprising; obtaining samples of normal data from a set of data containing normal and abnormal data; providing samples of abnormal data from said set or from a repository of abnormal data; classifying each of said samples as either said normal data or said abnormal data, to create ground truths for said samples; splitting said samples into a training set and a test set, with each of said sets containing samples of said normal data and of said abnormal data; using a model generating algorithm to generate a variety of models for distinguishing between said normal data and said abnormal data in said training set; obfuscating a portion of samples of said abnormal data in said training set to create obfuscated abnormal data samples; adding said obfuscated abnormal data samples to said test set to form an enhanced test set; performing statistical analysis on performance of said models with said enhanced test set to determine intrusion detection capability error; selecting one of said models that optimizes a desired model parameter as an optimal model for distinguishing between said normal data and said abnormal data; and installing said optimal model in said computer. 17. A computer program product stored in a computer readable medium for protecting a network, comprising: a machine learning based sensor that has been trained by a process comprising; obtaining samples of normal data from a set of data containing normal and abnormal data; providing samples of abnormal data from said set or from a repository of abnormal data; classifying each of said samples as either said normal data or said abnormal data, to create ground truths for said samples; splitting said samples into a training set and a. test set, with each of said sets containing samples of said normal data and of said abnormal data; using a model generating algorithm to generate a variety of models for distinguishing between said normal data and said abnormal data in said training set; obfuscating a portion of samples of said abnormal data iii said training set to create obfuscated abnormal data samples; adding said obfuscated abnormal data samples to said test set to form an enhanced test set; performing statistical analysis on performance of said models with said enhanced test set to determine intrusion detection capability error; selecting one of said models that optimizes a desired model parameter as an optimal model for distinguishing between said normal data and said abnormal data.