it to generalize to unseen states. Since the elimination signal provides immediate feedback, it is faster to learn which actions to eliminate (e.g., with a contextual bandit using the elimination signal) than to learn the optimal actions using only the reward (due to long term consequences). Therefore, we can design an algorithm that enjoys better performance by exploring invalid actions less frequently.

More specifically, we propose a system that learns an approximation of the Q-function and *concurrently learns to eliminate actions*. We focus on tasks where natural language characterizes both the states and the actions. In particular, the actions correspond to fixed length sentences defined over a finite dictionary (of words). In this case, the action space is of combinatorial size (in the length of the sentence and the size of the dictionary) and irrelevant actions must be eliminated to learn. We introduce a novel DRL approach with two DNNs, a DQN and an Action Elimination Network (AEN), both designed using a Convolutional Neural Network (CNN) that is suited to NLP tasks [@kim2014convolutional]. Using the last layer activations of the AEN, we design a linear contextual bandit model that eliminates irrelevant actions with high probability, balancing exploration/exploitation, and allowing the 