infinite ${{\mathcal{F}}}$ (we will extend our results to general infinite ${{\mathcal{F}}}$ in §\[sec:extension\]). For this case, the gold standard is an algorithm that achieves $\widetilde{O}(\sqrt{KT\log|{{\mathcal{F}}}|})$ regret with the total number of oracle calls being polynomial/sublinear in $T$ (see @agarwal2012contextual [@foster2018practical]). As for the optimization oracle, we assume access to the following (unweighted) least-sqaures regression oracle that solves $$\label{eq:lsr}
\arg\min_{f\in{{\mathcal{F}}}}\sum_{s=1}^t (f(x_s,a_s)-y_s)^2$$ for any input sequence $(x_1,a_1,y_1),\cdots,(x_t,a_t,y_t)\in{{\mathcal{X}}}\times{{\mathcal{A}}}\times[0,1]$. Without loss of generality[^1], we assume that the oracle (\[eq:lsr\]) always return a same solution for two input sequences that are completely the same. Note that the above least-squares regression oracle that we assume is even simpler than the weighted one (\[eq:wlsr\]) assumed in [@foster2018practical], as it does not need to consider the weights.

Key Challenges
--------------

Before we proceed to present our results, we would like to illustrate the key technical hurdles of using offline regression oracles to achieve the optimal regret for contextual bandits. We will then provide a few comments explaining how our approach gets around these technical hurdles.

As was pointed out before, three excellent papers [@agarwal2012contextual; @foster2018practical; @foster2020beyond] have made important progress towards solving contextual bandits via regression approaches. Understanding the gap between the existing results and our desired result 