every single time period, without repetitions of assortments in general.

Our model also allows the revenue for each product $j$ to change over time. In particular, we associate the revenue parameter $r_{tj}$ for the product $j$ at time $t$.

This model generalizes the widely adopted (generalized) linear contextual bandit from machine learning literature (see, e.g., [@filippi2010parametric; @chu11; @Abbasi-Yadkori:2011; @Agrawal:13; @li2017provably] and references therein) in a non-trivial way since the MNL cannot be written in a generalized linear model form (when an assortment contains more than one product, see Section \[sec:related\] for more details). It is also worthwhile noting that this model incorporates a personalized MNL model proposed by [@Wang:17:person] as a special case, where each product $j$ is associated with a fixed but unknown coefficient $\theta(j)$ and each arriving customer at time $t$ with an observable feature vector $x_t$ (see Section \[sec:related\] for a more detailed discussion). On the other hand, we choose to motivate our model from product contextual information since in practice, obtaining products’ features is usually easier (and less sensitive) than extracting customers’ preferences.

Given this contextual MNL choice model, the key challenge is how to design a policy that simultaneously learns the unknown coefficient $\theta_0$ and 