Methods, and systems, including computer programs encoded on computer storage media for generating data items. A method includes reading a glimpse from a data item using a decoder hidden state vector of a decoder for a preceding time step, providing, as input to a encoder, the glimpse and decoder hidden state vector for the preceding time step for processing, receiving, as output from the encoder, a generated encoder hidden state vector for the time step, generating a decoder input from the generated encoder hidden state vector, providing the decoder input to the decoder for processing, receiving, as output from the decoder, a generated a decoder hidden state vector for the time step, generating a neural network output update from the decoder hidden state vector for the time step, and combining the neural network output update with a current neural network output to generate an updated neural network output.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A neural network system implemented by one or more computers, the neural network system comprising: an encoder neural network, wherein the encoder neural network is a recurrent neural network that is configured to, for each input data item processed by the encoder neural network and at each time step of a plurality steps: receive a glimpse captured by reading from the input data item, receive a decoder hidden state vector of a decoder neural network for the preceding time step, and process the glimpse, the decoder hidden state vector, and an encoder hidden state vector of the encoder neural network from the preceding time step to generate an encoder hidden state vector for the time step; a decoder neural network, wherein the decoder neural network is a recurrent neural network that is configured to, for each of the plurality of time steps: receive a decoder input for the time step, and process the decoder hidden state vector for the preceding time step and the decoder input to generate a decoder hidden state vector for the time step; and a subsystem, wherein the subsystem is configured to, for each of the time steps: read the glimpse from the input data item using the decoder hidden state vector for the preceding time step; provide the glimpse as input to the encoder neural network; generate the decoder input for the decoder neural network from the encoder hidden state vector at the time step; provide the decoder input as input to the decoder neural network for the time step; generate a neural network output update for the time step from the decoder hidden state vector for the time step; and combine the neural network output update for the time step with a current neural network output to generate an updated neural network output. 2. The neural network system of claim 1, wherein the encoder neural network and the decoder neural network are long short term memory neural networks. 3. The neural network system of claim 1, wherein the subsystem is further configured to train the encoder neural network and the decoder neural network to autoencode input data items. 4. The neural network system of claim 3, wherein training the encoder neural network and the decoder neural network to autoencode input data items comprises training the neural networks to generate a neural network output that is a reconstruction of the input data item. 5. The neural network system of claim 1, wherein the subsystem is further configured to provide the encoder hidden state vectors from each of the time steps for a particular data item as features of the particular data item. 6. The neural network system of claim 5, wherein the features of the particular data item are provided for use in processing the particular data item during a semi-supervising learning procedure. 7. The neural network system of claim 1, wherein the input data items are images. 8. The neural network system of claim 1, wherein in the input data items are videos. 9. The neural network system of claim 1, wherein the input data items are images and wherein the glimpse captured by reading from the input data item is an image patch generated by applying an array of Gaussian filters to the image. 10. The neural network system of claim 9, wherein the parameters for applying the array of Gaussian filters are generated by applying a linear transformation to the decoder hidden state vector for the preceding time step. 11. The neural network system of claim 1, wherein generating the decoder input for the decoder neural network from the encoder hidden state vector at the time step comprises: using the encoder hidden state vector for the time step to parameterize a distribution of a latent vector; and sampling the decoder input from the distribution. 12. The neural network system of claim 1, wherein the subsystem is further configured to cause the decoder neural network to process a sequence of inputs sampled from a prior distribution to generate a new neural network output. 13. The neural network system of claim 12, wherein the data items are images, and wherein the new neural network output is a new image generated by the decoder neural network. 14. The neural network system of claim 1, wherein the data items are speech fragments or features of speech fragments. 15. A computer implemented method for processing an input data item through a recurrent encoder neural network and a recurrent decoder neural network to generate an updated neural network output, the method comprising, for each input data item and at each time step of a plurality steps: reading a glimpse from the input data item using a decoder hidden state vector of the decoder neural network for the preceding time step; providing, as input to the encoder neural network, the (i) glimpse and (ii) decoder hidden state vector for the preceding time step for processing; receiving, as output from the encoder neural network, a generated encoder hidden state vector for the time step; generating a decoder input for the decoder neural network from the generated encoder hidden state vector at the time step; providing, as input to the decoder neural network, the decoder input for processing; receiving, as output from the decoder neural network, a generated a decoder hidden state vector for the time step; generating a neural network output update for the time step from the decoder hidden state vector for the time step; and combining the neural network output update for the time step with a current neural network output to generate an updated neural network output. 16. The method of claim 15, further comprising providing the encoder hidden state vectors from each of the time steps for a particular data item as features of the particular data item. 17. The method of claim 15, wherein generating the decoder input for the decoder neural network from the encoder hidden state vector at the time step comprises: using the encoder hidden state vector for the time step to parameterize a distribution of a latent vector; and sampling the decoder input from the distribution. 18. The method of claim 15, further comprising causing the decoder neural network to process a sequence of inputs sampled from a prior distribution to generate a new neural network output. 19. The method of claim 18, wherein the data items are images, and wherein the new neural network output is a new image generated by the decoder neural network. 20. A computer storage medium encoded with instructions that, when executed by one or more computers, cause one or more computers to implement a neural network system, the neural network system comprising: an encoder neural network, wherein the encoder neural network is a recurrent neural network that is configured to, for each input data item processed by the encoder neural network and at each time step of a plurality steps: receive a glimpse captured by reading from the input data item, receive a decoder hidden state vector of a decoder neural network for the preceding time step, and process the glimpse, the decoder hidden state vector, and an encoder hidden state vector of the encoder neural network from the preceding time step to generate an encoder hidden state vector for the time step; a decoder neural network, wherein the decoder neural network is a recurrent neural network that is configured to, for each of the plurality of time steps: receive a decoder input for the time step, and process the decoder hidden state vector for the preceding time step and the decoder input to generate a decoder hidden state vector for the time step; and a subsystem, wherein the subsystem is configured to, for each of the time steps: read the glimpse from the input data item using the decoder hidden state vector for the preceding time step; provide the glimpse as input to the encoder neural network; generate the decoder input for the decoder neural network from the encoder hidden state vector at the time step; provide the decoder input as input to the decoder neural network for the time step; generate a neural network output update for the time step from the decoder hidden state vector for the time step; and combine the neural network output update for the time step with a current neural network output to generate an updated neural network output. 21. A neural network system implemented by one or more computers, the neural network system comprising: a decoder neural network, wherein the decoder neural network is a recurrent neural network that is configured to, for each of the plurality of time steps: receive a decoder input for the time step, and process the decoder hidden state vector for the preceding time step and the decoder input to generate a decoder hidden state vector for the time step; and a subsystem, wherein the subsystem is configured to, for each of the time steps: generate the decoder input for the decoder neural network; provide the decoder input as input to the decoder neural network for the time step; generate a neural network output update for the time step from the decoder hidden state vector for the time step; and combine the neural network output update for the time step with a current neural network output to generate an updated neural network output. 22. The neural network system of claim 21, wherein generating decoder input for the decoder neural network comprises selecting latent variable samples from a prior distribution. 23. The neural network system of claim 21, wherein the updated neural network outputs are images, and wherein the updated neural network output for a last time step in the plurality of time steps is a new image generated by the neural network system.