A learning agent is disclosed that receives data in sequence from one or more sequential data sources; generates a model modelling sequences of data and actions; and selects an action maximizing the expected future value of a reward function, wherein the reward function depends at least partly on at least one of: a measure of the change in complexity of the model, or a measure of the complexity of the change in the model. The measure of the change in complexity of the model may be based on, for example, the change in description length of the first part of a two-part code describing one or more sequences of received data and actions, the change in description length of a statistical distribution modelling, the description length of the change in the first part of the two-part code, or the description length of the change in the statistical distribution modelling.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A reinforcement learning system, comprising: one or more processors; and one or more programs residing on a memory and executable by the one or more processors, the one or more programs configured to: perform actions from a set of available actions; receive data in sequence from one or more sequential data sources; generate a model that models sequences of the received data and the performed actions; and select an action to maximize an expected future value of a reward function, wherein the reward function depends at least partly on at least one of: a measure of a change in complexity of the model or a measure of the complexity of the change in the model. 2. The system of claim 1, wherein the measure of the change in complexity of the model is based on a change in description length of the first part of a two-part code describing one or more sequences of received data and actions. 3. The system of claim 1, wherein the measure of the change in complexity of the model is based on a change in negative log likelihood of the first part of a two-part code describing one or more sequences of received data and actions. 4. The system of claim 1, wherein the measure of the complexity of the change in the model is based on a description length of a change in a first part of a two-part code describing one or more sequences of received data and actions. 5. The system of claim 1, wherein the measure of the complexity of the change in the model is based on a negative log likelihood of a change in a first part of a two-part code describing one or more sequences of received data and actions. 6. The system of claim 1, wherein the measure of the change in complexity of the model is based on a change in description length of a statistical distribution modelling one or more sequences of received data and actions. 7. The system of claim 1, wherein the measure of the change in complexity of the model is based on a change in negative log likelihood of a statistical distribution modelling one or more sequences of received data and actions. 8. The system of claim 1, wherein the measure of the complexity of the change in the model is based on a description length of a change in a statistical distribution modelling one or more sequences of received data and actions. 9. The system of claim 1, wherein the measure of the complexity of the change in the model is based on a negative log likelihood of a change in a statistical distribution modelling one or more sequences of received data and actions. 10. The system of claim 1, wherein the model is represented as a neural network that approximately predicts sequences of received data and actions. 11. The system of claim 1, wherein the model is represented as a replay memory that stores sequences of received data and actions, and an action is at least sometimes selected according to an action-value function learned via Q Learning with experience replay. 12. The system of claim 1, wherein the model is represented as a statistical distribution factorized into potential functions over cliques on a factor graph containing nodes corresponding to the data sources as well as zero or more additional nodes corresponding to auxiliary variables. 13. The system of claim 12, wherein a first potential function is similar to a second potential function when variables in the first potential function are substituted for variables in the second potential function, and the complexity of the model is reduced by the second potential function referencing the first potential function or by the first and second potential functions referencing a common function. 14. The system of claim 12, wherein a potential function over a clique is conditioned on a conditioning value at each node in the clique, where the conditioning value at each node is one of: a data value received from a data source associated with the node; a random value; or a maximum likelihood value estimated from the factor graph with potential functions conditioned on another set of conditioning values. 15. The system of claim 14, wherein the another set of conditioning values are different from a previous point in time. 16. A method for reinforcement learning, comprising the steps of: receiving data in sequence from the one or more sequential data sources; generating a model, wherein the model is configured to model sequences of the received data and actions; and selecting an action maximizing the expected future value of a reward function, wherein the reward function depends at least partly on at least one of: a measure of the change in complexity of the model, or a measure of the complexity of the change in the model. 17. The method of claim 16, wherein the measure of the change in complexity of the model is based on a change in description length of the first part of a two-part code describing one or more sequences of received data and actions. 18. The method of claim 16, wherein the measure of the change in complexity of the model is based on a change in negative log likelihood of the first part of a two-part code describing one or more sequences of received data and actions. 19. The method of claim 16, wherein the measure of the complexity of the change in the model is based on a description length of a change in a first part of a two-part code describing one or more sequences of received data and actions. 20. The method of claim 16, wherein the measure of the complexity of the change in the model is based on a negative log likelihood of a change in a first part of a two-part code describing one or more sequences of received data and actions. 21. The method of claim 16, wherein the measure of the change in complexity of the model is based on a change in description length of a statistical distribution modelling one or more sequences of received data and actions. 22. The method of claim 16, wherein the measure of the change in complexity of the model is based on a change in negative log likelihood of a statistical distribution modelling one or more sequences of received data and actions. 23. The method of claim 16, wherein the measure of the complexity of the change in the model is based on a description length of a change in a statistical distribution modelling one or more sequences of received data and actions. 24. The method of claim 16, wherein the measure of the complexity of the change in the model is based on a negative log likelihood of a change in a statistical distribution modelling one or more sequences of received data and actions. 25. The method of claim 16, wherein the model is represented as a neural network that approximately predicts sequences of received data and actions. 26. The method of claim 16, wherein the model is represented as a replay memory that stores sequences of received data and actions, and an action is at least sometimes selected according to an action-value function learned via Q Learning with experience replay. 27. The method of claim 16, wherein the model is represented as a statistical distribution factorized into potential functions over cliques on a factor graph containing nodes corresponding to the data sources as well as zero or more additional nodes corresponding to auxiliary variables. 28. The method of claim 27, wherein a first potential function is similar to a second potential function when variables in the first potential function are substituted for variables in the second potential function, and the complexity of the model is reduced by the second potential function referencing the first potential function or by the first and second potential functions referencing a common function. 29. The method of claim 27, wherein a potential function over a clique is conditioned on a conditioning value at each node in the clique, where the conditioning value at each node is one of: a data value received from a data source associated with the node; a random value; or a maximum likelihood value estimated from the factor graph with potential functions conditioned on another set of conditioning values. 30. The method of claim 29, wherein the another set of conditioning values are different from a previous point in time.