process on the other; some variation may be helpful to explore the system and search for a desirable policy [@SusanMurphy_2016_CORR_BatchOffPolicyAvgRwd]. We consider the parameterized stochastic policy, $\pi_{\theta}\left(a\mid s\right)$, where $\theta\in\mtbbR^{m}$ is the unknown parameter. Such policy is interpretable in the sense that we could know the key features that contribute most to the policy by analyzing the estimated $\widehat{\theta}$, which is important to behavior scientists for the state (feature) design [@huitian_2014_NIPS_ActCriticBandit4JITAI; @SusanMurphy_2016_CORR_BatchOffPolicyAvgRwd].

In RL, value is a core concept that quantifies the quality of a policy $\pi$ [@Sutton_2012_Book_ReinforcementLearning]. There are two definitions of values: the state value and the state-action ($Q$-) value [@Abe_2004_SIGKDD_CrossChannelRL]. In mHealth, the $Q$-value is considered because the model (i.e. state transition and immediate reward) is assumed to be unknown, and $Q$-value allows for action selection without knowing the model while the state value requires the model for the action selection [@Michail_2003_JMLR_LSPI_LSTDQ]. Formally, the $Q$-value $Q^{\pi}\left(s,a\right)\in\mtbbR^{\left|\mtcalS\right|\times\left|\mtcalA\right|}$ measures the total amount of rewards an agent can obtain when starting from state $s$, first choosing action $a$ and then following the policy $\pi$. Specially, the discounted reward is one of the most commonly used value measures $$Q^{\pi}\left(s,a\right)=\mtexpect_{a_{i}\sim\pi,s_{i}\sim\mtcalP}\left\{ \sum_{i=0}^{\infty}\gamma^{i}r_{i}\mid s_{0}=s,a_{0}=a\right\} .\label{eq:Q_value}$$

The goal of RL is to learn an optimal 