Methods and systems for performing a sequence of machine learning tasks. One system includes a sequence of deep neural networks (DNNs), including: a first DNN corresponding to a first machine learning task, wherein the first DNN comprises a first plurality of indexed layers, and each layer in the first plurality of indexed layers is configured to receive a respective layer input and process the layer input to generate a respective layer output; and one or more subsequent DNNs corresponding to one or more respective machine learning tasks, wherein each subsequent DNN comprises a respective plurality of indexed layers, and each layer in a respective plurality of indexed layers with index greater than one receives input from a preceding layer of the respective subsequent DNN, and one or more preceding layers of respective preceding DNNs, wherein a preceding layer is a layer whose index is one less than the current index.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A neural network system implemented by one or more computers, the neural network system comprising a sequence of deep neural networks (DNNs), wherein each DNN in the sequence of DNNs has been trained to perform a respective machine learning task, and wherein the sequence of DNN comprises: a first DNN that corresponds to a first machine learning task, wherein (i) the first DNN comprises a first plurality of indexed layers, and (ii) each layer in the first plurality of indexed layers is configured to receive a respective layer input and process the layer input to generate a respective layer output; and one or more subsequent DNNs corresponding to one or more respective machine learning tasks, wherein (i) each subsequent DNN comprises a respective plurality of indexed layers, and (ii) each layer in a respective plurality of indexed layers with index greater than one receives input from (i) a preceding layer of the respective subsequent DNN, and (ii) one or more preceding layers of respective preceding DNNs, wherein a preceding layer is a layer whose index is one less than the current index. 2. The system of claim 1, wherein each layer with index equal to one in a respective plurality of indexed layers receives a respective subsequent DNN input. 3. The system of claim 1, wherein (i) the first plurality of indexed layers comprises L hidden layers, and (ii) each respective plurality of indexed layers s comprises Ms hidden layers. 4. The system of claim 3, wherein L is not equal to Ms for each s. 5. The system of claim 3, wherein L is equal to Ms for one or more s. 6. The system of claim 3, wherein each layer in the first plurality of indexed layers comprises a hidden activation hi(1)εni, where ni represents a number of hidden units at layer i≦L. 7. The system of claim 6, wherein each layer in a respective plurality of indexed layers s comprises a hidden activation hj(s)εnj, where nj represents a number of hidden units at layer j≦Ms. 8. The system of claim 7, wherein the sequence of machine learning tasks comprises k+1 machine learning tasks, and wherein an activation of the k-th subsequent DNN is given by h i ( k ) = σ ( W i ( k )  h i - 1 ( k ) + ∑ j < k  U ij ( k )  h i - 1 ( j ) ) wherein hi(k) represents an activation of the k-th subsequent DNN, Wi(k)εn1×ni-1 represents a weight matrix of layer i of the k-th subsequent DNN, Uij(k)εn1×nj represents lateral connections from layer i of the k-th subsequent DNN to layer i−1 of the j-th subsequent DNN and σ represents an element-wise non linearity. 9. The system of claim 7, wherein the sequence of machine learning tasks comprises k machine learning tasks, and wherein an activation of the k-th subsequent DNN is given by hi(k)=σ(Wi(k)hi-1(k)+Uij(k)σ(Vij(k)αi-1(<k)αi-1(<k)hi-1(<k))) wherein hi(k) represents an activation of the k-th subsequent DNN, Wi(k)εn1×ni-1 represents a weight matrix of layer i of the k-th subsequent DNN, Uij(k)εn1×nj represents lateral connections from layer i of the k-th subsequent DNN to layer i−1 of the j-th subsequent DNN, σ represents an element-wise non linearity, Vij(k)εn1×ni-1(<k) represents a projection matrix and σi-1(<k) is a learned scalar. 10. The system of claim 1, wherein the sequence of machine learning tasks comprises independent machine learning tasks. 11. The system of claim 1, wherein the sequence of machine learning tasks comprises one or more of (i) adversarial machine learning tasks, (ii) classification tasks, (iii) robot learning tasks, or (iv) generative modeling tasks. 12. A method for sequentially training a sequence of deep neural networks (DNNs) to perform a sequence of machine learning tasks, each DNN in the sequence corresponding to a respective machine learning task, and the method comprising: for a first machine learning task in the sequence: training a first DNN in the sequence that corresponds to the first machine learning task to perform the first machine learning task, wherein (i) the first DNN comprises a first plurality of indexed layers, and (ii) each layer in the first plurality of indexed layers is configured to receive a respective layer input and process the layer input to generate a respective layer output; for each subsequent machine learning task in the sequence: training a subsequent DNN corresponding to the machine learning task to perform the machine learning task, wherein (i) the subsequent DNN comprises a subsequent plurality of indexed layers, and (ii) each layer with index greater than one in the subsequent indexed plurality of layers receives input from (i) a preceding layer of the subsequent DNN, and (ii) one or more preceding layers of respective preceding DNNs, wherein a preceding layer is a layer whose index is one less than the current index. 13. The method of claim 12, wherein each DNN layer with index equal to one in a respective plurality of indexed layers receives a respective subsequent DNN input. 14. The method of claim 12, wherein the sequence of machine learning tasks comprises independent machine learning tasks. 15. The method of claim 12, wherein the sequence of machine learning tasks comprises one or more of (i) adversarial machine learning tasks, (ii) classification tasks, (iii) robot learning tasks, or (iv) generative modeling tasks. 16. The method of claim 12, wherein (i) the first DNN comprises one or more respective DNN parameters, and (ii) each subsequent DNN comprises one or more respective subsequent DNN parameters, and wherein training each subsequent DNN comprises setting preceding DNN parameters of preceding DNNs to constant values. 17. The method of claim 12, wherein training each subsequent DNN further comprises adjusting values of the respective subsequent DNN parameters using a machine learning training technique. 18. A method for processing an input using a sequence of deep neural networks (DNNs), wherein each DNN in the sequence of DNNs has been trained to perform a respective machine learning task, the sequence of DNN comprising: a first DNN that corresponds to a first machine learning task, wherein (i) the first DNN comprises a first plurality of indexed layers, and (ii) each layer in the first plurality of indexed layers is configured to receive a respective layer input and process the layer input to generate a respective layer output; and one or more subsequent DNNs corresponding to one or more respective machine learning tasks, wherein (i) each subsequent DNN comprises a respective plurality of indexed layers, and (ii) each layer in a respective plurality of indexed layers with index greater than one receives input from (i) a preceding layer of the respective subsequent DNN, and (ii) one or more preceding layers of respective preceding DNNs, wherein a preceding layer is a layer whose index is one less than the current index; the method comprising: receiving an input as part of a machine learning task corresponding to a last subsequent DNN in the sequence of DNNs; and processing the input using the last subsequent DNN in the sequence to generate a last subsequent DNN output for the machine learning task. 19. The method of claim 18, wherein each layer with index equal to one in a respective plurality of indexed layers receives a respective subsequent DNN input. 20. The method of claim 18, wherein the machine learning task corresponding to the last subsequent DNN in the sequence of DNNs comprises (i) an adversarial machine learning task, (ii) a classification task, (iii) a robot learning task or (iv) a generative modeling task.