Learning from a scalar reward in continuous action space environments is difficult and often requires millions if not billions of interactions.   We introduce state aligned vector rewards, which are easily defined in metric state spaces and allow our deep reinforcement learning agent to tackle the curse of dimensionality.   Our agent learns to map from action distributions to state change distributions implicitly defined in a quantile function neural network.    We further introduce a new reinforcement learning technique inspired by quantile regression which does not limit agents to explicitly parameterized action distributions.   Our results in high dimensional state spaces show that training with vector rewards allows our agent to learn multiple times faster than an agent training with scalar rewards.
We train with state aligned vector rewards an agent predicting state changes from action distributions, using a new reinforcement learning technique inspired by quantile regression. Presents algorithm that aims to speed up reinforcement learning in situations where the reward is aligned with the state space.  This paper addresses RL in the continuous action space, using a re-parametrised policy and a novel vector-based training objective. This work proposes to mix distributional RL with a net in charge of modeling the evolution of the world in terms of quantiles, claiming improvements in sample efficiency.