according to a probabilistic selection strategy that picks each action other than $\widehat{a}_t$ with probability roughly inversely proportional to how much worse it is predicted to be as compared with $\widehat{a}_t$, as well as roughly inversely proportional to the learning rate $\gamma_m$. The effects of this strategy are twofold. First, *at each round*, by assigning the greedy action the highest probability and each non-greedy action a probability roughly inverse to the predicted reward gap, we ensure that the better an action is predicted to be, the more likely it will be selected. Second, *across different epochs*, by controlling the probabilites of non-greedy actions roughly inverse to the gradually increasing learning rate $\gamma_m$, we ensure that the algorithm “explores more” in the beginning rounds where the learning rate is small, and gradually “exploits more” in later rounds where the learning rate becomes larger — this is why we view our learning rate a sequential balancer between exploration and exploitation.

Algorithmic Components and Comparisons with Literature
------------------------------------------------------

`FALCON` is a very simple algorithm, and can be viewed a novel combination of three algorithmic components: (i) an epoch schedule, (ii) the greedy use of an offline least-squares regression oracle, and (iii) a probabilistic 