X}})$ represents the baseline effect, and $C_0({{\bm X}})$ denotes the contrast effect as $$C_0({{\bm X}})={\mbox{E}}(Y|{{\bm X}},A=1)-{\mbox{E}}(Y|{{\bm X}},A=0).$$ Therefore, $g^{\mathrm{opt}}_\mu({{\bm X}})={\mathrm{1}}\{C_0({{\bm X}})>0$}. In Q-learning, a parametric model is often employed as a working model, $${\mbox{E}}(Y|{{\bm X}},A)=h({{\bm X}}; {{\mbox{\boldmath $\gamma$}}})+AC({{\bm X}};{{\mbox{\boldmath $\beta$}}}),$$ where $h({{\bm X}};{{\mbox{\boldmath $\gamma$}}})$ and $C({{\bm X}};{{\mbox{\boldmath $\beta$}}})$ are posited parametric models for $h_0({{\bm X}})$ and $C_0({{\bm X}})$ respectively. Commonly a linear model is assumed for simplicity and interpretability, i.e., $h({{\bm X}};{{\mbox{\boldmath $\gamma$}}})={{\mbox{\boldmath $\gamma$}}}{^{\mbox{\tiny {\sf T}}}}\tilde{{{\bm X}}}$ and $C({{\bm X}};{{\mbox{\boldmath $\beta$}}})={{\mbox{\boldmath $\beta$}}}{^{\mbox{\tiny {\sf T}}}}\tilde{{{\bm X}}}$, where $\tilde{{{\bm X}}}=({{\mathbf 1}},{{\bm X}}{^{\mbox{\tiny {\sf T}}}}){^{\mbox{\tiny {\sf T}}}}$. Given the observation $\{(Y_i,{{\bm X}}_i,A_i);\;i=1,\ldots,n\}$, the Q-learning procedure estimates the parameters $({{\mbox{\boldmath $\beta$}}},{{\mbox{\boldmath $\gamma$}}})$ by minimizing the squared error loss $$L_{1n}({{\mbox{\boldmath $\beta$}}},{{\mbox{\boldmath $\gamma$}}})=\frac{1}{n}\sum_{i=1}^{n}\left\{Y_i-h({{\bm X}}_i;{{\mbox{\boldmath $\gamma$}}})-A_iC({{\bm X}}_i;{{\mbox{\boldmath $\beta$}}})\right\}^2.$$ Denote the optimized point as $(\hat{{{\mbox{\boldmath $\beta$}}}}^Q,\hat{{{\mbox{\boldmath $\gamma$}}}}^Q)$. The estimated optimal ITR based on Q-learning is then $\hat{g}^{Q}({{\bm x}})\triangleq{\mathrm{1}}\{C({{\bm x}};\hat{{{\mbox{\boldmath $\beta$}}}}^Q)>0\}$, which is a consistent estimator of $g^{\mathrm{opt}}_\mu({{\bm x}})$ if both $h({{\bm X}};{{\mbox{\boldmath $\gamma$}}})$ and $C({{\bm X}};{{\mbox{\boldmath $\beta$}}})$ are correctly specified.

A-learning is a semiparametric improvement of Q-learning by modeling only the contrast function $C_0({{\bm X}})$ rather than the full Q-function. This is reasonable based on the observation that the optimal ITR $g^{\mathrm{opt}}_\mu$ only depends on $C_0({{\bm X}})$. By 