of AUC occurs. In inference, we use the model with the highest AUC on the validation set.

**Masking** In order to ensure that the agent avoids selecting the same feature more than once, we apply a multiplicative mask to the agents’ Q values so that Q values of features that were already selected are multiplied by zero and consequently will not be selected again

**$\epsilon$-greedy sampling probabilities** In order to explore new paths, it is a common practice to select the action that maximizes the $Q$ values in equations  and  with probability $1-\epsilon$, rather than with probability 1, and select a action uniformly at random with probability $\epsilon$. Usually practitioners use a time-decay policy for $\epsilon$. Here, instead of using uniform probabilities for action sampling, we sample each action $a$ with probability which is proportional to the absolute correlation of the corresponding question with the target label over the training set. This heuristic helps choosing actions which are more informative about the target with higher probability.

**Architectures** Our best results were achieved using straightforward depth 3 MLP architectures for both the guesser and the Q network. We also experimented with a more sophisticated design, where the state update mechanism is 