The technology disclosed relates to evolving a deep neural network based solution to a provided problem. In particular, it relates to providing an improved cooperative evolution technique for deep neural network structures. It includes creating blueprint structures that include a plurality of supermodule structures. The supermodule structures include a plurality of modules. The modules are neural networks. A first loop of evolution executes at the blueprint level. A second loop of evolution executes at the supermodule level. Further, multiple mini-loops of evolution execute at each of the subpopulations of the supermodules. The first loop, the second loop, and the mini-loops execute in parallel.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A computer-implemented system for cooperatively evolving a deep neural network structure that solves a provided problem when trained on a source of training data containing labeled examples of data sets for the problem, the system comprising: a memory storing a candidate supermodule genome database having a pool of candidate supermodules, each of the candidate supermodules identifying respective values for a plurality of supermodule hyperparameters of the supermodule, the supermodule hyperparameters including supermodule global topology hyperparameters identifying a plurality of modules in the candidate supermodule and module interconnects among the modules in the candidate supermodule, at least one of the modules in each candidate supermodule including a neural network, each candidate supermodule having associated therewith storage for an indication of a respective supermodule fitness value; the memory further storing a blueprint genome database having a pool of candidate blueprints for solving the provided problem, each of the candidate blueprints identifying respective values for a plurality of blueprint topology hyperparameters of the blueprint, the blueprint topology hyperparameters including a number of included supermodules, and interconnects among the included supermodules, each candidate blueprint having associated therewith storage for an indication of a respective blueprint fitness value; an instantiation module which instantiates each of at least a training subset of the blueprints in the pool of candidate blueprints, at least one of the blueprints being instantiated more than once, each instantiation of a candidate blueprint including identifying for the instantiation a supermodule from the pool of candidate supermodules for each of the supermodules identified in the blueprint; a training module which trains neural networks on training data from the source of training data, the neural networks are modules which are identified by supermodules in each of the blueprint instantiations, the training includes modifying submodules of the neural network modules in dependence upon back-propagation algorithms; an evaluation module which, for each given one of the blueprints in the training subset of blueprints: evaluates each instantiation of the given blueprint on validation data, to develop a blueprint instantiation fitness value associated with each of the blueprint instantiations, updates fitness values of all supermodules identified for inclusion in each instantiation of the given blueprint in dependence upon the fitness value of the blueprint instantiation, and updates a blueprint fitness value for the given blueprint in dependence upon the fitness values for the instantiations of the blueprint; a competition module which: selects blueprints for discarding from the pool of candidate blueprints in dependence upon their updated fitness values, and selects supermodules from the candidate supermodule pool for discarding in dependence upon their updated fitness values; a procreation module which: forms new supermodules in dependence upon a respective set of at least one parent supermodule from the pool of candidate supermodules, and forms new blueprints in dependence upon a respective set of at least one parent blueprint from the pool of candidate blueprints; and a solution harvesting module providing for deployment a selected one of the blueprints remaining in the candidate blueprint pool, instantiated with supermodules selected from the candidate supermodule pool. 2. The system of claim 1, wherein each supermodule in the pool of candidate supermodules further belongs to a subpopulation of the supermodules, wherein the blueprint topology hyperparameters of blueprints in the pool of candidate blueprints also identify a supermodule subpopulation for each included supermodule, wherein the instantiation module selects, for each supermodule identified in the blueprint, a supermodule from the subpopulation of supermodules which is identified by the blueprint, wherein the competition module, in selecting supermodules from the candidate supermodule pool for discarding in dependence upon their updated fitness values, does so in further dependence upon the subpopulation to which the supermodules belong, wherein the procreation module, in forming new supermodules in dependence upon a respective set of at least one parent supermodule from the pool of candidate supermodules, forms the new supermodules only in dependence upon parent supermodules which belong to the same subpopulation, and wherein the system is further configured to comprise a re-speciation module which re-speciates the supermodules in the pool of candidate supermodules into updated subpopulations. 3. The system of claim 2, wherein the competition module selects supermodules for discarding from the subpopulation with a same subpopulation identifier (ID). 4. The system of claim 1, further configured to comprise a control module which repeatedly invokes, for each of a plurality of generations, the training module, the evaluation module, the competition module, and the procreation module. 5. The system of claim 1, wherein the validation data is data previously unseen during training of a particular supermodule. 6. The system of claim 1, wherein a particular supermodule is identified in a plurality of blueprint instantiations, and wherein the evaluation module updates a supermodule fitness value associated with the particular supermodule in dependence of respective blueprint instantiation fitness values associated with each of the blueprint instantiations in the plurality. 7. The system of claim 6, wherein the supermodule fitness value is an average of the respective blueprint instantiation fitness values. 8. The system of claim 1, wherein the evaluation module assigns a supermodule fitness value to a particular supermodule if the supermodule fitness value is previously undetermined. 9. The system of claim 1, wherein the evaluation module, for a particular supermodule, merges a current supermodule fitness value with a previously determined supermodule fitness. 10. The system of claim 9, wherein the merging includes averaging. 11. The system of claim 1, wherein the evaluation module updates the blueprint fitness value for the given blueprint by averaging the fitness values for the instantiations of the blueprint. 12. The system of claim 1, wherein the supermodule hyperparameters further comprise module topology hyperparameters that identify a plurality of submodules of the neural network and interconnections among the submodules. 13. The system of claim 12, wherein crossover and mutation of the module topology hyperparameters during procreation includes modifying a number of submodules and/or interconnections among them. 14. A computer-implemented method of cooperatively evolving a deep neural network structure that solves a provided problem when trained on a source of training data containing labeled examples of data sets for the problem, the method including: storing a candidate supermodule genome database having a pool of candidate supermodules, each of the candidate supermodules identifying respective values for a plurality of supermodule hyperparameters of the supermodule, the supermodule hyperparameters including supermodule global topology hyperparameters identifying a plurality of modules in the candidate supermodule and module interconnects among the modules in the candidate supermodule, at least one of the modules in each candidate supermodule including a neural network, each candidate supermodule having associated therewith storage for an indication of a respective supermodule fitness value; storing a blueprint genome database having a pool of candidate blueprints for solving the provided problem, each of the candidate blueprints identifying respective values for a plurality of blueprint topology hyperparameters of the blueprint, the blueprint topology hyperparameters including a number of included supermodules, and interconnects among the included supermodules, each candidate blueprint having associated therewith storage for an indication of a respective blueprint fitness value; instantiating each of at least a training subset of the blueprints in the pool of candidate blueprints, at least one of the blueprints being instantiated more than once, each instantiation of a candidate blueprint including identifying for the instantiation a supermodule from the pool of candidate supermodules for each of the supermodules identified in the blueprint; training neural networks on training data from the source of training data, the neural networks are modules which are identified by supermodules in each of the blueprint instantiations, the training further includes modifying submodules of the neural networks in dependence upon back-propagation algorithms; for each given one of the blueprints in the training subset of blueprints, evaluating each instantiation of the given blueprint on validation data, to develop a blueprint instantiation fitness value associated with each of the blueprint instantiations, updating fitness values of all supermodules identified for inclusion in each instantiation of the given blueprint in dependence upon the fitness value of the blueprint instantiation, and updating a blueprint fitness value for the given blueprint in dependence upon the fitness values for the instantiations of the blueprint; selecting blueprints for discarding from the pool of candidate blueprints in dependence upon their updated fitness values; selecting supermodules from the candidate supermodule pool for discarding in dependence upon their updated fitness values; forming new supermodules in dependence upon a respective set of at least one parent supermodule from the pool of candidate supermodules; forming new blueprints in dependence upon a respective set of at least one parent blueprint from the pool of candidate blueprints; and deploying a selected one of the blueprints remaining in the candidate blueprint pool, instantiated with supermodules selected from the candidate supermodule pool. 15. The computer-implemented method of claim 14, wherein each supermodule in the pool of candidate supermodules further belongs to a subpopulation of the supermodules, wherein the blueprint topology hyperparameters of blueprints in the pool of candidate blueprints also identify a supermodule subpopulation for each included supermodule, wherein the instantiating includes selecting, for each supermodule identified in the blueprint, a supermodule from the subpopulation of supermodules which is identified by the blueprint, wherein, in selecting supermodules from the candidate supermodule pool for discarding in dependence upon their updated fitness values, doing so in further dependence upon the subpopulation to which the supermodules belong, wherein, in forming new supermodules in dependence upon a respective set of at least one parent supermodule from the pool of candidate supermodules, forming the new supermodules only in dependence upon parent supermodules which belong to the same subpopulation, and wherein re-speciating the supermodules in the pool of candidate supermodules into updated subpopulations. 16. The computer-implemented method of claim 14, further including repeatedly performing, for each of a plurality of generations, the training, the evaluating, the updating, the selecting, the forming, and the deploying. 17. The computer-implemented method of claim 14, wherein a particular supermodule is identified in a plurality of blueprint instantiations, and further including updating a supermodule fitness value associated with the particular supermodule in dependence of respective blueprint instantiation fitness values associated with each of the blueprint instantiations in the plurality. 18. A non-transitory computer readable storage medium impressed with computer program instructions to cooperatively evolve a deep neural network structure that solves a provided problem when trained on a source of training data containing labeled examples of data sets for the problem, the instructions, when executed on a processor, implement a method comprising: storing a candidate supermodule genome database having a pool of candidate supermodules, each of the candidate supermodules identifying respective values for a plurality of supermodule hyperparameters of the supermodule, the supermodule hyperparameters including supermodule global topology hyperparameters identifying a plurality of modules in the candidate supermodule and module interconnects among the modules in the candidate supermodule, at least one of the modules in each candidate supermodule including a neural network, each candidate supermodule having associated therewith storage for an indication of a respective supermodule fitness value; storing a blueprint genome database having a pool of candidate blueprints for solving the provided problem, each of the candidate blueprints identifying respective values for a plurality of blueprint topology hyperparameters of the blueprint, the blueprint topology hyperparameters including a number of included supermodules, and interconnects among the included supermodules, each candidate blueprint having associated therewith storage for an indication of a respective blueprint fitness value; instantiating each of at least a training subset of the blueprints in the pool of candidate blueprints, at least one of the blueprints being instantiated more than once, each instantiation of a candidate blueprint including identifying for the instantiation a supermodule from the pool of candidate supermodules for each of the supermodules identified in the blueprint; training neural networks on training data from the source of training data, the neural networks are modules which are identified by supermodules in each of the blueprint instantiations, the training further includes modifying submodules of the neural networks in dependence upon back-propagation algorithms; for each given one of the blueprints in the training subset of blueprints, evaluating each instantiation of the given blueprint on validation data, to develop a blueprint instantiation fitness value associated with each of the blueprint instantiations, updating fitness values of all supermodules identified for inclusion in each instantiation of the given blueprint in dependence upon the fitness value of the blueprint instantiation, and updating a blueprint fitness value for the given blueprint in dependence upon the fitness values for the instantiations of the blueprint; selecting blueprints for discarding from the pool of candidate blueprints in dependence upon their updated fitness values; selecting supermodules from the candidate supermodule pool for discarding in dependence upon their updated fitness values; forming new supermodules in dependence upon a respective set of at least one parent supermodule from the pool of candidate supermodules; forming new blueprints in dependence upon a respective set of at least one parent blueprint from the pool of candidate blueprints; and deploying a selected one of the blueprints remaining in the candidate blueprint pool, instantiated with supermodules selected from the candidate supermodule pool. 19. The non-transitory computer readable storage medium of claim 18, wherein each supermodule in the pool of candidate supermodules further belongs to a subpopulation of the supermodules, wherein the blueprint topology hyperparameters of blueprints in the pool of candidate blueprints also identify a supermodule subpopulation for each included supermodule, wherein the instantiating includes selecting, for each supermodule identified in the blueprint, a supermodule from the subpopulation of supermodules which is identified by the blueprint, wherein, in selecting supermodules from the candidate supermodule pool for discarding in dependence upon their updated fitness values, doing so in further dependence upon the subpopulation to which the supermodules belong, wherein, in forming new supermodules in dependence upon a respective set of at least one parent supermodule from the pool of candidate supermodules, forming the new supermodules only in dependence upon parent supermodules which belong to the same subpopulation, and wherein re-speciating the supermodules in the pool of candidate supermodules into updated subpopulations. 20. The non-transitory computer readable storage medium of claim 18, implementing the method further comprising repeatedly performing, for each of a plurality of generations, the training, the evaluating, the updating, the selecting, the forming, and the deploying.