probability $\epsilon,$ return $\text{Uniform}(A')$ Otherwise, return $\underset{{a\in A'}}{\arg\max} Q(s,a)$ return $r$ $A'\leftarrow \{a: E(s)_a - \sqrt{\beta \phi(s)^TV_a^{-1}\phi(s)}  < \ell \}$ return $(r+\gamma\underset{{a\in A'}}{\text{max}}Q(s,a))$ $V^{-1}_a = \left(\sum_{j:a_j=a} \phi(s_j)\phi(s_j)^T + \lambda I\right)^{-1}$ $b_a = \sum_{j:a_j=a} \phi(s_j)^Te_j $ Set LastLayer($E^-_a$) $\gets V^{-1}_ab_a$ return $E^-, V^{-1}$

\[alg1\]

Our Algorithm presents a hybrid approach for DRL with Action Elimination (AE), by incorporating AE into the well-known DQN algorithm to yield our AE-DQN (Algorithm \[alg1\] and Figure \[fig:diagram\]). AE-DQN trains two networks: a DQN denoted by $Q$ and an AEN denoted by $E$. The algorithm uses $E$, and creates a linear contextual bandit model from it every $L$ iterations with procedure **AENUpdate()**. This procedure uses the activations of the last hidden layer of $E$ as features, $\phi(s) \leftarrow \mbox{LastLayerActivations}(E(s)),$ which are then used to create a contextual linear bandit model ($V_a=\lambda I+\sum_{j:a_j=a} \phi(s_j)\phi(s_j)^T,b_a=\sum_{j:a_j=a} \phi(s_j)^Te_j$). AENUpdate() proceeds by solving this model, and plugging the solution into the target AEN (LastLayer($E^-_a$) $\gets V^{-1}_ab_a$). The contextual linear bandit model ($E^-,V$) is then used to eliminate actions (with high probability) via the **ACT()** and **Targets()** functions. ACT() follows an $\epsilon-$greedy mechanism on the admissible actions set $A'=\{ a: E(s)_a-\sqrt{\beta\phi(s)^TV^{-1}_a\phi(s)}<\ell\}$. If it decides to exploit, then it selects the action with 