A system implementing reinforcement learning the system comprising a computer processor and a computer readable medium having computer executable instructions executed by said processor; said computer readable medium including instructions for providing: an action values module that receives environmental state as input, containing at least one adaptive element that learns state and/or action values based on an error signal; an action selection module coupled to the action values module; and an error calculation module coupled to both the action values and action selection module, which computes an error signal used to update state and/or action values in the action values module.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A system implementing reinforcement learning the system comprising a computer processor and a computer readable medium having computer executable instructions executed by said processor; said computer readable medium including instructions for providing: an action values module that receives environmental state as input, containing at least one adaptive element that learns state and/or action values based on an error signal; an action selection module coupled to the action values module; an error calculation module coupled to both the action values and action selection module, which computes an error signal used to update state and/or action values in the action values module; wherein each module or sub-module comprises a plurality of nonlinear components, wherein each nonlinear component is configured to generate a scalar or vector output in response to the input and is coupled to the output module by at least one weighted coupling; the output from each nonlinear component is weighted by the connection weights of the corresponding weighted couplings and the weighted outputs are provided to the output module to form the output modifier; the input to the system is either discrete or continuous in time and space; and, the input to the system is one of a scalar and a multidimensional vector. 2. The system of claim 1, wherein multiple instances of the system are composed into a hierarchical or recurrent structure, wherein the output of one instance performs one or more of adding new state input to the input of the downstream instance; modifying state in the downstream instance; and modifies the reward signal of the downstream instance. 3. The system of claim 1, wherein an error module computes an error that may include an integrative discount 4. The system of claim 1, wherein the module representing state/action values consists of two interconnected sub-modules, each of which receives state information with or without time delay as input, and the output of one sub-module is used to train the other in order to allow state and/or action value updates to be transferred over time 5. The system of claim 1, wherein there are initial couplings within and between different modules of the system, where each weighted coupling has a corresponding connection weight such that the output generated by each nonlinear component is weighted by the corresponding connection weights to generate a weighted output 6. The system of claim 5, wherein a neural compiler is used to determine the initial couplings and connection weights 7. The system of claim 1 wherein at least one of the nonlinear components in an adaptive sub module that generates a multidimensional output is coupled to the action selection and/or error calculation modules by a plurality of weighted couplings, one weighted coupling for each dimension of the multidimensional output modifier. 8. The system of claim 1, wherein a learning sub-module is configured to update connection weights based on the initial output and the outputs generated by the nonlinear components 9. The system of claim 1, wherein a learning sub-module is configured to update the connection weights based on an outer product of the initial output and the outputs from the nonlinear components. 10. The system of claim 1, wherein each nonlinear component has a tuning curve that determines the output generated by the nonlinear component in response to any input and the tuning curve for each nonlinear component may be generated randomly. 11. The system of claim 1, wherein the nonlinear components are simulated neurons. 12. The system of claim 11, wherein the neurons are spiking neurons. 13. The system of claim 1, wherein the components are implemented in hardware specialized for simulating the nonlinear components. 14. A computer implemented method for reinforcement learning comprising receiving by an action values module stored on a computer readable medium environmental state as input, containing at least one adaptive element that learns state and/or action values based on an error signal; providing on the computer readable medium an action selection module coupled to the action values module; computing an error signal to update state and/or action values in the action values module by a calculation module coupled to both the action values and action selection module wherein each module or sub-module comprises a plurality of nonlinear components, wherein each nonlinear component is configured to generate a scalar or vector output in response to the input and is coupled to the output module by at least one weighted coupling; the output from each nonlinear component is weighted by the connection weights of the corresponding weighted couplings and the weighted outputs are provided to the output module to form the output modifier; the input to the system is either discrete or continuous in time and space; and, the input to the system is one of a scalar and a multidimensional vector 15. The method of claim 14, further comprising repeating the method in a hierarchical or recurrent manner such that the output of one instance of the method performs one or more of adding new state input to the input of the downstream instance; modifying state in the downstream instance; and modifies the reward signal of the downstream instance. 16. The method of claim 14, further comprising computing by an error module an error that may include an integrative discount 17. The method of claim 14, wherein the module representing state/action values consists of two interconnected sub-modules, each of which receives state information with or without time delay as input, and the output of one sub-module is used to train the other in order to allow state and/or action value updates to be transferred over time 18. The method of claim 14, wherein there are initial couplings within and between different modules, where each weighted coupling has a corresponding connection weight such that the output generated by each nonlinear component is weighted by the corresponding connection weights to generate a weighted output 19. The method of claim 18, further comprising determining by a neural complier the initial couplings and connection weights 20. The method of claim 14 wherein at least one of the nonlinear components in an adaptive submodule that generates a multidimensional output is coupled to the action selection and/or error calculation modules by a plurality of weighted couplings, one weighted coupling for each dimension of the multidimensional output modifier. 21. The method of claim 14, further comprising updating by a learning sub-module connection weights based on the initial output and the outputs generated by the nonlinear components 22. The method of claim 14, further comprising updating by a learning sub-module the connection weights based on an outer product of the initial output and the outputs from the nonlinear components. 23. The method of claim 14, wherein each nonlinear component has a tuning curve that determines the output generated by the nonlinear component in response to any input and the tuning curve for each nonlinear component may be generated randomly. 24. The method of claim 14, wherein the nonlinear components are simulated neurons. 25. The method of claim 24, wherein the neurons are spiking neurons.