Methods and apparatus for training a multi-layer artificial neural network for use in speech recognition. The method comprises determining for a first speech pattern of the plurality of speech patterns, using a first processing pipeline, network activations for a plurality of nodes of the artificial neural network in response to providing the first speech pattern as input to the artificial neural network, determining based, at least in part, on the network activations and a selection criterion, whether the artificial neural network should be trained on the first speech pattern, and updating, using a second processing pipeline, network weights between nodes of the artificial neural network based, at least in part, on the network activations when it is determined that the artificial neural network should be trained on the first speech pattern.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method of training a multi-layer artificial neural network for use in speech recognition, wherein each of the layers in the artificial neural network includes a plurality of nodes, each of the plurality of nodes in a layer being connected to a plurality of nodes in one or more adjacent layers of the artificial neural network, wherein the connections between nodes in the artificial neural network are associated with network weights, the method comprising: providing a plurality of speech patterns as input to the artificial neural network; determining for a first speech pattern of the plurality of speech patterns, using a first processing pipeline, network activations for the plurality of nodes of the artificial neural network in response to providing the first speech pattern as input to the artificial neural network; determining based, at least in part, on the network activations and a selection criterion, whether the artificial neural network should be trained on the first speech pattern; and updating, using a second processing pipeline, the network weights based, at least in part, on the network activations when it is determined that the artificial neural network should be trained on the first speech pattern. 2. The method of claim 1, wherein determining whether the artificial neural network should be trained on the first speech pattern comprises: determining an output error for the first speech pattern based, at least in part, on the network activations; and determining that the artificial neural network should be trained on the first speech pattern when the output error is greater than a threshold value. 3. The method of claim 1, wherein the selection criterion comprises a focused attention backpropagation criterion, and wherein determining whether the artificial neural network should be trained on the first speech pattern comprises applying the focused attention back propagation criterion to the network activations. 4. The method of claim 1, further comprising: determining based, at least in part, on the selection criterion, whether the first speech pattern has already been learned by the artificial neural network; and copying data corresponding to the first speech pattern from the first processing pipeline to the second processing pipeline when it is determined that the first speech pattern has not already been learned by the artificial neural network. 5. The method of claim 4, wherein determining whether the first speech pattern has already been learned by the artificial neural network comprises: determining an output error for the first speech pattern; comparing the output error to a threshold value; and determining that the speech pattern has already been learned by the artificial neural network when the output error is less than the threshold value. 6. The method of claim 5, wherein determining the output error comprises determining a mean squared error based, at least in part, on the network activations. 7. The method of claim 4, wherein copying data corresponding to the first speech pattern comprises determining whether a processing buffer of the second processing pipeline is full and copying the data when it is determined that the processing buffer is not full. 8. The method of claim 7, further comprising updating the network weights in response to determining that the processing buffer is full. 9. The method of claim 4, wherein copying the data comprises creating a pointer in a data structure of the second processing pipeline that references a location of the data in a data structure of the first processing pipeline. 10. The method of claim 1, wherein updating the network weights comprises backpropagating errors through multiple layers of the multi-layer artificial neural network. 11. The method of claim 1, further comprising: processing data in the first processing pipeline using multiple processors and processing data in the second processing pipeline using the multiple processors. 12. The method of claim 11, wherein processing data using multiple processors comprises assigning to each of the multiple processors, at least two layers of the multi-layer artificial neural network, wherein the multiple processors are arranged to pass data from one processor to another processor to enable pipelined parallel processing of data in the first processing pipeline and the second processing pipeline. 13. The method of claim 11, wherein at least two of the multiple processors are graphics processing units (GPUs). 14. The method of claim 1, further comprises: determining whether artificial neural network has been sufficiently trained; and iteratively calculating network activations using the first processing pipeline and updating network weights using the second processing pipeline until it is determined that the artificial neural network has been sufficiently trained. 15. An apparatus configured to train a multi-layer artificial neural network for use in speech recognition, wherein each of the layers in the artificial neural network includes a plurality of nodes, each of the plurality of nodes in a layer being connected to a plurality of nodes in one or more adjacent layers of the artificial neural network, wherein the connections between nodes in the artificial neural network are associated with network weights, the apparatus comprising: at least one processor programmed to implement a first processing pipeline and a second processing pipeline for training the artificial neural network based, at least in part, on a plurality of speech patterns provided to the artificial neural network as input; wherein the first processing pipeline is configured to determine for a first speech pattern of the plurality of speech patterns, network activations for the plurality of nodes of the artificial neural network in response to providing the first speech pattern as input to the artificial neural network; wherein the at least one processor is further programmed to determine based, at least in part, on the network activations and a selection criterion, whether the artificial neural network should be trained on the first speech pattern; and wherein the second processing pipeline is configured to update the network weights based, at least in part, on the network activations when it is determined that the artificial neural network should be trained on the first speech pattern. 16. The apparatus of claim 15, wherein the at least one processor is further programmed to: determine based, at least in part, on the selection criterion, whether the first speech pattern has already been learned by the artificial neural network; and copy data corresponding to the first speech pattern from the first processing pipeline to the second processing pipeline when it is determined that the first speech pattern has not already been learned by the artificial neural network. 17. The apparatus of claim 15, wherein the at least one processor comprises at least two graphics processing units (GPUs), wherein each of the at least two GPUs is assigned to perform calculations for at least two layers of the multi-layer artificial neural network, and wherein the at least two GPUs are arranged to pass data between each other to enable pipelined parallel processing of data in the first processing pipeline and the second processing pipeline. 18. A non-transitory computer readable medium encoded with a plurality of instructions that, when executed by at least one computer processor, perform a method of training a multi-layer artificial neural network for use in speech recognition, wherein each of the layers in the artificial neural network includes a plurality of nodes, each of the plurality of nodes in a layer being connected to a plurality of nodes in one or more adjacent layers of the artificial neural network, wherein the connections between nodes in the artificial neural network are associated with network weights, the method comprising: providing a plurality of speech patterns as input to the artificial neural network; determining for a first speech pattern of the plurality of speech patterns, using a first processing pipeline, network activations for the plurality of nodes of the artificial neural network in response to providing the first speech pattern as input to the artificial neural network; determining based, at least in part, on the network activations and a selection criterion, whether the artificial neural network should be trained on the first speech pattern; and updating, using a second processing pipeline, the network weights for the plurality of nodes of the artificial neural network based, at least in part, on the network activations when it is determined that the artificial neural network should be trained on the first speech pattern. 19. The non-transitory computer readable medium of claim 18, wherein the selection criterion comprises a focused attention backpropagation criterion, and wherein determining whether the artificial neural network should be trained on the first speech pattern comprises applying the focused attention back propagation criterion to the network activations. 20. The non-transitory computer readable medium of claim 18, wherein the method further comprises: determining based, at least in part, on the selection criterion, whether the first speech pattern has already been learned by the artificial neural network; and copying data corresponding to the first speech pattern from the first processing pipeline to the second processing pipeline when it is determined that the first speech pattern has not already been learned by the artificial neural network.