Unentangled quantum reinforcement learning agents in the OpenAI Gym
Classical reinforcement learning (RL) has generated excellent results in
different regions; however, its sample inefficiency remains a critical issue.
In this paper, we provide concrete numerical evidence that the sample
efficiency (the speed of convergence) of quantum RL could be better than that
of classical RL, and for achieving comparable learning performance, quantum RL
could use much (at least one order of magnitude) fewer trainable parameters
than classical RL. Specifically, we employ the popular benchmarking
environments of RL in the OpenAI Gym, and show that our quantum RL agent
converges faster than classical fully-connected neural networks (FCNs) in the
tasks of CartPole and Acrobot under the same optimization process. We also
successfully train the first quantum RL agent that can complete the task of
LunarLander in the OpenAI Gym. Our quantum RL agent only requires a
single-qubit-based variational quantum circuit without entangling gates,
followed by a classical neural network (NN) to post-process the measurement
output. Finally, we could accomplish the aforementioned tasks on the real IBM
quantum machines. To the best of our knowledge, none of the earlier quantum RL
agents could do that.
Jen-Yueh Hsiao Yuxuan Du Wei-Yin Chiang Min-Hsiu Hsieh Hsi-Sheng Goan