**RL in Large Action Spaces:** Being able to reason in an environment with a large number of discrete actions is essential to bringing reinforcement learning to a larger class of problems. Most of the prior work concentrated on factorizing the action space into binary subspaces [@pazis2011generalized; @dulac2012fast; @lagoudakis2003reinforcement]. Other works proposed to embed the discrete actions into a continuous space, use a continuous-action policy gradient to find optimal actions in the continuous space, and finally, choose the nearest discrete action [@dulac2015deep; @van2009using]. He et. al. ([-@DBLP:journals/corr/HeCHGLDO15]) extended DQN to unbounded (natural language) action spaces. His algorithm learns representations for the states and actions with two different DNNs and then models the Q values as an inner product between these representation vectors. While this approach can generalize to large action spaces, in practice, they only considered a small number of available actions (4) in each state.

Learning to eliminate actions was first mentioned by [@even2003action] who studied elimination in multi-armed bandits and tabular MDPs. They proposed to learn confidence intervals around the value function in each state and then use it to eliminate actions that are not optimal with high probability. Lipton et al. ([-@lipton2016combating]) studied a related problem where an 