Systems and methods for training a neural network to optimize network performance, including sampling an applied dropout rate for one or more nodes of the network to evaluate a current generalization performance of one or more training models. An optimized annealing schedule may be generated based on the sampling, wherein the optimized annealing schedule includes an altered dropout rate configured to improve a generalization performance of the network. A number of nodes of the network may be adjusted in accordance with a dropout rate specified in the optimized annealing schedule. The steps may then be iterated until the generalization performance of the network is maximized.
Please help me write a proper abstract based on the patent claims. CLAIM: 1-12. (canceled) 13. A system for training a neural network to optimize network performance, comprising: a sampler configured to iteratively sample, using a processor, an applied dropout rate for one or more nodes of the network to evaluate a current generalization performance of one or more training models; an annealing schedule generator configured to iteratively generate, using the processor, an optimized annealing schedule based on the sampling, wherein the optimized annealing schedule includes an altered dropout rate configured to improve a generalization performance of the network; and an adjuster configured to iteratively adjust a number of nodes of the network in accordance with a dropout rate specified in the optimized annealing schedule until a generalization performance of the network is maximized. 14. The system as recited in claim 13, wherein the adjuster is further configured to zero a fixed percentage of the nodes. 15. The system as recited in claim 13, wherein the adjuster is further configured to gradually decrease a dropout probability of the input or output nodes in the network during the training. 16. The system as recited in claim 13, wherein the adjuster is further configured to increase the dropout rate for successive iterations to prevent overfitting of training data. 17. The system as recited in claim 13, wherein the adjuster is further configured to randomly set the output of one or more nodes to zero with dropout probability pd, and wherein the adjusting further comprises iteratively adjusting the dropout probability pd. 18. The system as recited in claim 13, wherein the system further comprises one or more bias correction mechanisms configured to correct biases associated with each of one or more linear projections taken at one or more network layers so that they are consistent with a subspace implied by a dropout mask applied to one or more inputs of the layer. 19. The system as recited in claim 18, wherein the bias correction mechanism is further configured to input dropout mask specific biases for each of the one or more linear projections. 20. A computer readable storage medium comprising a computer readable program for training a neural network to optimize network performance, wherein the computer readable program when executed on a computer causes the computer to perform the steps of: sampling an applied dropout rate for one or more nodes of the network to evaluate a current generalization performance of one or more training models; generating, using a processor, an optimized annealing schedule based on the sampling, wherein the optimized annealing schedule includes an altered dropout rate configured to improve a generalization performance of the network; adjusting a number of nodes of the network in accordance with a dropout rate specified in the optimized annealing schedule; and iterating until the generalization performance of the network is maximized. 21. The computer readable storage medium as recited in claim 20, wherein the adjusting a number of nodes further comprises zeroing a fixed percentage of the nodes. 22. The computer readable storage medium as recited in claim 20, wherein the adjusting a number of nodes further comprises gradually decreasing a dropout probability of the nodes in the network during the training. 23. The computer readable storage medium as recited in claim 20, wherein the adjusting a number of nodes further comprises increasing the dropout rate for successive iterations to prevent overfitting of training data. 24. The computer readable storage medium as recited in claim 20, wherein the applied dropout rate is sampled from a distribution over dropout rates, which are estimated or evolved as network training proceeds. 25. The computer readable storage medium as recited in claim 20, wherein the adjusting a number of nodes further comprises randomly setting the output of one or more of the nodes to zero with dropout probability pd, and wherein the adjusting a number of nodes further comprises iteratively adjusting the dropout probability pd. 26. The computer readable storage medium as recited in claim 25, wherein the adjusting a number of nodes further comprises applying one of a linear or geometric fixed decaying schedule to adjust the dropout probability pd. 27. The computer readable storage medium as recited in claim 20, wherein the generating the optimized annealing schedule further comprises generating an optimized joint annealing schedule, as indicated by a loss function on the held-out data, for at least one of a learning rate, the dropout probability pd, and any other hyperparameters of the learning procedure for two or more training models. 28. The computer readable storage medium as recited in claim 27, wherein generating the optimized joint annealing schedule further comprises: considering all or a subset of the set of combinations implied by one of holding fixed, increasing, or decreasing each hyperparameter by a specified amount; and selecting a subset, including the N best performing models, N>=1, of models that result, based on one or more iterations of learning, for application in additional training iterations. 29. The computer readable storage medium as recited in claim 20, wherein the training further comprises correcting biases associated with each of one or more linear projections taken at one or more network layers so that they are consistent with a subspace implied by a dropout mask applied to one or more inputs of the layer. 30. The computer readable storage medium as recited in claim 29, wherein the training further comprises inputting dropout mask specific biases for each of the one or more linear projections. 31. The computer readable storage medium as recited in claim 30, wherein the dropout mask specific biases are realized with a matrix of learned biases, and a total bias to apply to each linear projection is determined by multiplying a dropout mask vector by a matrix.