The disclosure provides an approach for learning to schedule control fragments for physics-based virtual character simulations and physical robot control. Given precomputed tracking controllers, a simulation application segments the controllers into control fragments and learns a scheduler that selects control fragments at runtime to accomplish a task. In one embodiment, each scheduler may be modeled with a Q-network that maps a high-level representation of the state of the simulation to a control fragment for execution. In such a case, the deep Q-learning algorithm applied to learn the Q-network schedulers may be adapted to use a reward function that prefers the original controller sequence and an exploration strategy that gives more chance to in-sequence control fragments than to out-of-sequence control fragments. Such a modified Q-learning algorithm learns schedulers that are capable of following the original controller sequence most of the time while selecting out-of-sequence control fragments when necessary.
Please help me write a proper abstract based on the patent claims. CLAIM: 1. A computer-implemented method for controlling motion, comprising: receiving one or more tracking controllers; training a neural network for scheduling the received controllers or portions thereof based on a state of simulation and a task; and performing a simulation of a virtual character or controlling motion of a physical robot using the trained neural network. 2. The method of claim 1, further comprising: segmenting the received controller into control fragments, wherein the trained neural network takes the state of simulation and the task as input and outputs rewards for each the control fragments, and wherein the control fragment with the highest reward is selected for execution at each time step. 3. The method of claim 2, wherein the training uses at least one of a reward function which prefers a sequence of the received tracking controllers or an exploration strategy which gives more chance to in-sequence control fragments. 4. The method of claim 3, wherein: the neural network computes Q-values, and the training uses a deep Q-learning algorithm. 5. The method of claim 4, wherein the Q-network includes an input layer including a state vector modeling the simulation state and the task, an output layer which computes the Q-values of all actions with each element of an output vector corresponding to a respective control fragment, and one or more hidden layers. 6. The method of claim 1, wherein the received controllers are constructed from at least one of motion capture data, a hand-animation, recorded robot motions, or a trajectory optimization technique. 7. The method of claim 1, wherein each of the received controllers is one of an open-loop controller or an open-loop controller enhanced with a linear feedback policy. 8. The method of claim 1, wherein the character is simulated or the robot is controlled to walk on a ball, balance on a bongo board, skateboard, run, recover from a push, or breakdance. 9. A non-transitory computer-readable storage medium storing a program, which, when executed by a processor performs operations for controlling motion, the operations comprising: receiving one or more tracking controllers; training a neural network for scheduling the received controllers or portions thereof based on a state of simulation and a task; and performing a simulation of a virtual character or controlling motion of a physical robot using the trained neural network. 10. The non-transitory computer-readable storage medium of claim 9, the operations further comprising: segmenting the received controller into control fragments, wherein the trained neural network takes the state of simulation and the task as input and outputs rewards for each the control fragments, and wherein the control fragment with the highest reward is selected for execution at each time step. 11. The non-transitory computer-readable storage medium of claim 10, wherein the training uses at least one of a reward function which prefers a sequence of the received tracking controllers or an exploration strategy which gives more chance to in-sequence control fragments. 12. The non-transitory computer-readable storage medium of claim 11, wherein: the neural network computes Q-values, and the training uses a deep Q-learning algorithm. 13. The non-transitory computer-readable storage medium of claim 12, wherein the Q-network includes an input layer including a state vector modeling the simulation state and the task, an output layer which computes an output layer which computes the Q-values of all actions with each element of an output vector corresponding to a respective control fragment, and one or more hidden layers. 14. The non-transitory computer-readable storage medium of claim 9, wherein the received controllers are constructed from at least one of motion capture data, a hand-animation, recorded robot motions, or a trajectory optimization technique. 15. The non-transitory computer-readable storage medium of claim 9, wherein each of the received controllers is one of an open-loop controller or an open-loop controller enhanced with a linear feedback policy. 16. The non-transitory computer-readable storage medium of claim 9, wherein the character is simulated or the robot is controlled to walk on a ball, balance on a bongo board, skateboard, run, recover from a push, or breakdance. 17. A system, comprising: a processor; and a memory, wherein the memory includes a program configured to perform operations for controlling motion, the operations comprising: receiving one or more tracking controllers, training a neural network for scheduling the received controllers or portions thereof based on a state of simulation and a task, and performing a simulation of a virtual character or controlling motion of a physical robot using the trained neural network. 18. The system of claim 17, the operations further comprising: segmenting the received controller into control fragments, wherein the trained neural network takes the state of simulation and the task as input and outputs rewards for each the control fragments, and wherein the control fragment with the highest reward is selected for execution at each time step. 19. The system of claim 18, wherein the training uses at least one of a reward function which prefers a sequence of the received tracking controllers or an exploration strategy which gives more chance to in-sequence control fragments. 20. The system of claim 19, wherein: the neural network computes Q-values, and the training uses a deep Q-learning algorithm.