not given as *a priori* and cannot be easily estimated due to the insufficiency of historical data. This motivates the research of dynamic assortment optimization, which has attracted a lot of attentions from the revenue management community in recent years. A typical dynamic assortment optimization problem assumes a finite selling horizon of length $T$ with a large $T$. At each time period, the seller offers an assortment of products (with the size upper bounded by $K$) to an arriving customer. The seller observes the customer’s purchase decision, which further provides useful information for learning utility parameters of the underlying choice model. The multinomial logit model (MNL) has been widely used in dynamic assortment optimization literature, see, e.g., [@Caro2007; @Rusmevichientong2010; @Saure2013; @Agrawal16MNLBandit; @Agrawal17Thompson; @Chen:18tight; @Chen:18near].

In the age of e-commerce, side information of products is widely available (e.g., brand, color, size, texture, popularity, historical selling information), which is important in characterizing customers’ preferences for products. Moreover, some features are not static and could change over time (e.g., popularity score or ratings). The feature/contextual information of products will facilitate accurate assortment decisions that are tailored to customers’ preferences. In particular, we assume at each time $t=1,\ldots, T$, each product $j$ is associated 