We describe a method of reinforcement learning for a subject system having multiple states and actions to move from one state to the next. Training data is generated by operating on the system with a succession of actions and used to train a second neural network. Target values for training the second neural network are derived from a first neural network which is generated by copying weights of the second neural network at intervals.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A method of reinforcement learning, the method comprising: inputting training data relating to a subject system, the subject system having a plurality of states and, for each state, a set of actions to move from one of said states to a next said state; wherein said training data is generated by operating on said system with a succession of said actions and comprises starting state data, action data and next state data defining, respectively for a plurality of said actions, a starting state, an action, and a next said state resulting from the action; and training a second neural network using said training data and target values for said second neural network derived from a first neural network; the method further comprising: generating or updating said first neural network from said second neural network. 2. A method as claimed in claim 1 further comprising selecting said actions using learnt action-value parameters from said second neural network, wherein said actions are selected responsive to an action-value parameter determined for each action of a set of actions available at a state of said system. 3. A method as claimed in claim 2 wherein said training data comprises experience data derived from said selected actions, the method further comprising generating said experience data by storing data defining said actions selected by said second neural network in association with data defining respective said starting states and next states for the actions. 4. A method as claimed in claim 3 further comprising generating said target values by providing said data defining said actions and said next states to said first neural network, and training said second neural network using said target values and said data defining said starting states. 5. A method as claimed in claim 2 further comprising: inputting state data defining a state of said system; providing said second neural network with a representation of said state of said system; retrieving from said second neural network a learnt said action-value parameter for each action of said set of actions available at said state; and selecting an action to perform having a maximum or minimum said learnt action-value parameter from said second neural network. 6. A method as claimed in claim 5 further comprising storing experience data from said system, wherein said experience data is generated by operating on said system with said actions selected using said second neural network, and wherein said training data comprises said stored experience data. 7. A method as claimed in claim 6 further comprising: selecting, from said experience data, starting state data, action data and next state data for one of said plurality of actions; providing said first neural network with a representation of said next state from said next state data; determining, from said first neural network, a maximum or minimum learnt action-value parameter for said next state; determining a target value for training said second neural network from said maximum or minimum learnt action-value parameter for said next state. 8. A method as claimed in claim 7 wherein said training of said second neural network comprises providing said second neural network with a representation of said starting state from said starting state data and adjusting weights of said neural network to bring a learnt action-value parameter for an action defined by said action data closer to said target value. 9. A method as claimed in claim 7 wherein said experience data further comprises reward data defining a reward value or cost value of said system resulting from said action taken, and wherein said determining of said target value comprises adjusting said maximum or minimum learnt action-value parameter for said next state by said reward value or said cost value respectively. 10. A method as claimed in claim 1 wherein a state of said system comprises a sequence of observations of said system over time representing a history of said system. 11. A method as claimed in claim 2 wherein said training of said second neural network alternates with said selecting of said actions and comprises incrementally updating a set of weights of said second neural network used for selecting said actions. 12. A method as claimed in claim 1 wherein said generating or updating of said first neural network from said second neural network is performed at intervals after repeated said selecting of said actions using said second neural network and said training of said second neural network. 13. A method as claimed in claim 12 wherein said generating or updating of said first neural network from said second neural network comprises copying a set of weights of said second neural network to said first neural network. 14. A method as claimed in claim 1 wherein a said state is defined by image data. 15. A method as claimed in claim 1 wherein said first and second neural networks comprise deep neural networks with a convolutional neural network input stage. 16. A non-transitory data carrier carrying processor control code to implement the method of claim 1. 17. A method of Q-learning wherein Q values are determined by a neural network and used to select actions to be performed on a system to move the system between states, wherein a first neural network is used to generate a Q-value for a target for training a second neural network used to select said actions. 18. A method as claimed in claim 17 wherein at intervals said first neural network is refreshed from said second neural network. 19. A method as claimed in claim 18 wherein weights of said first neural network are quasi-stationary, remaining substantially unchanged during intervals between said refreshing. 20. A method as claimed in claim 19 further comprising storing a record of said selected actions and states, and using said record to generate said Q-value for said target.