<p>Another paper to make the list, from the value-based school, is <a href="https://arxiv.org/abs/1609.07152" rel="nofollow noreferrer">Input Convex Neural Networks</a>. The idea is to require Q(s,a) to be convex in actions (not necessarily in states). Then, solving the argmax Q inference is reduced to finding the global optimum using the convexity, much faster than an exhaustive sweep and easier to implement than other value-based approaches. Yet, likely at the expense of a reduced representation power than usual feedforward or convolutional neural networks. </p>
How can I apply reinforcement learning to continuous action spaces?
<p>I'm trying to get an agent to learn the mouse movements necessary to best perform some task in a reinforcement learning setting (i.e. the reward signal is the only feedback for learning).</p>

<p>I'm hoping to use the Q-learning technique, but while I've found <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.17.2539&amp;rep=rep1&amp;type=pdf">a way to extend this method to continuous state spaces</a>, I can't seem to figure out how to accommodate a problem with a continuous action space.</p>

<p>I could just force all mouse movement to be of a certain magnitude and in only a certain number of different directions, but any reasonable way of making the actions discrete would yield a huge action space. Since standard Q-learning requires the agent to evaluate <em>all</em> possible actions, such an approximation doesn't solve the problem in any practical sense.</p>