by $I(S=s,W=w,A=a)/P(S=s,W=w,A=a)(Y-\bar{Q}_{P}(s,W,A))$. So we obtain the
following formula for the canonical gradient of $P\rightarrow E{\cal
W}\bar{Q}_{P}(s,W,A){\bf\phi}$:

$\begin{array}[]{l}\int_{w,a}dP(w,a){\cal
W}(w,a)\frac{I(S=s,W=w,A=a)}{P(S=s,W=w,A=a)}(Y-\bar{Q}_{P}(s,W,A)){\bf\phi}(W,A)\\\
={\cal W}(W,A)I(S=s)/P(S=s\mid
W,A){\bf\phi}(W,A)(Y-\bar{Q}_{P}(s,W,A)).\end{array}$

So the canonical gradient of $P\rightarrow E{\cal
W}\\{\bar{Q}_{P}(1,W,A){\bf\phi}-\bar{Q}_{P}(0,W,A){\bf\phi}\\}$ is given by

$D_{\beta,P,2}(O)={\cal W}\frac{I(S=1)-I(S=0)}{\Pi_{P}(S\mid
W,A)}I_{P}^{-1}{\bf\phi}(Y-\bar{Q}_{P}(S,W,A)).$

This proves the lemma. ∎

Binary outcome: We note that this working model applies to $Y\in\\{0,1\\}$,
but in that case the linear model $m_{\beta}=\beta^{\top}\phi$ is not
respecting the bound $-1\leq\tau_{P}\leq 1$. For binary outcomes it might be
more appropriate to consider a semiparametric regression model on the logistic
scale. In that case, the working model assumes $\mbox{Logit}P(Y=1\mid
S,W,A)=\theta(W,A)+S\beta^{\top}\phi$, and
$(\theta(P),\beta(P))=\arg\min_{\beta,\theta}PL(m_{\theta,\beta})$ is the log-
likelihood projection of $\bar{Q}_{P}$ onto this semiparametric working model.
Here $\mbox{Logit}m_{\theta,\beta}=\theta(W,A)+S\beta^{\top}\phi$ denotes the
semiparametric logistic regression model with unknown baseline function
$\theta$. We can work out the canonical gradient $D_{\beta,P}$ for this
definition of $\beta(P)$, and the new form can still be plugged in into our
expressions for the canonical gradient of $\Psi_{{\cal M}_{w,n}}^{*}$ and
$\Psi_{{\cal M}_{w,n}}$ below. With the canonical gradient of $\beta(P)$, and
the results from previous subsections, it is straightforward to derive the
canonical gradient of $\Psi_{{\cal
M}_{w}}^{*}(P)=\tilde{\Psi}(P)-\Psi^{\\#}_{{\cal M}_{w}}(P)$, which is
presented in the following lemma.

######  Lemma 11.

Consider a linear working model $m_{\beta}=\beta^{\top}\phi$ for
$\tau_{P}(A,W)=\bar{Q}_{P}(1,W,A)-\bar{Q}_{P}(0,W,A)$. Let $D_{\beta,P}$ be
the canonical gradient of $\beta(P)$ defined by

$\beta(P)\equiv\arg\min_{\beta}E_{P}{\cal
W}\left\\{\tau_{P}(W,A)-m_{\beta}(W,A)\right\\}^{2}.$

This canonical gradient is presented in the above Lemma 10. We have that the
canonical gradient of $\Psi_{{\cal
M}_{w}}^{*}=\tilde{\Psi}(P)-\Psi^{\\#}_{{\cal M}_{w}}(P)$ at $P$ is given by

$D_{{\cal M}_{w},P}^{*}=D_{\tilde{\Psi},P}-D^{\\#}_{{\cal M}_{w},P},$

where

$\displaystyle D_{\tilde{\Psi},P}=$ $\displaystyle\frac{2A-1}{g_{P}(A\mid
W)}(Y-E_{P}(Y\mid A,W))+E_{P}(Y\mid A=1,W)-E_{P}(Y\mid A=0,W)-\tilde{\Psi}(P)$
$\displaystyle=$ $\displaystyle\frac{2A-1}{g_{P}(A\mid W)}(Y-E_{P}(Y\mid
S,W,A))+$ $\displaystyle\frac{2A-1}{g_{P}(A\mid W)}(E_{P}(Y\mid
S=1,W,A)-E_{P}(Y\mid S=0,W,A))(S-\Pi_{P}(1\mid W,A))+$ $\displaystyle
E_{P}(Y\mid A=1,W)-E_{P}(Y\mid A=0,W)-\tilde{\Psi}(P),$

and

$D^{\\#}_{{\cal M}_{w},P}=D_{{\cal M}_{w},1,P}+D_{{\cal M}_{w},\Pi,P}+D_{{\cal
M}_{w},\beta,P}$

with

$\displaystyle D_{{\cal M}_{w},\beta,P}^{\\#}$
$\displaystyle\equiv\left\\{E_{P}\Pi_{P}(0\mid
W,0){\phi}^{\top}(W,0)\right\\}D_{\beta,P}(O)-\left\\{E_{P}\Pi_{P}(0\mid
W,1){\phi}^{\top}(W,1)\right\\}D_{\beta,P}(O)$ $\displaystyle D^{\\#}_{{\cal
M}_{w},1,P}$ $\displaystyle\equiv\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)-\Pi_{P}(0\mid
W,1)\tau_{w,\beta(P)}(W,1)-\Psi_{{\cal M}_{w}}^{\\#}(P)$ $\displaystyle
D^{\\#}_{{\cal M}_{w},\Pi_{P},P}$
$\displaystyle\equiv\left\\{\frac{A}{g_{P}(1\mid
W)}\tau_{w,\beta}(W,1)-\frac{1-A}{g_{P}(0\mid
W)}\tau_{w,\beta}(W,0)\right\\}(S-\Pi_{P}(1\mid W,A)).$

###### Proof.

The contribution from $\beta(P)$ in the canonical gradient $D_{{\cal
M}_{w},P}^{\\#}$ of $\Psi^{\\#}_{{\cal M}_{w}}$ is given by

$\begin{array}[]{l}D_{{\cal
M}_{w},\beta,P}^{\\#}\equiv\left\\{E_{P}\Pi_{P}(0\mid
W,1)\frac{d}{d\beta(P)}\tau_{w,\beta(P)}(W,1)\right\\}^{\top}D_{\beta,P}(O)\\\
+\left\\{E_{P}\Pi_{P}(0\mid
W,0)\frac{d}{d\beta(P)}\tau_{w,\beta(P)}(W,0)\right\\}^{\top}D_{\beta,P}(O).\end{array}$

The other components of $D_{{\cal M}_{w},P}^{\\#}$ are identical to the ones
presented in the case that we have a parametric working model and can thus
copies from the corresponding above lemma. This provides us with the canonical
gradient of $\Psi^{\\#}_{{\cal M}_{w}}$ at $P$ for the semiparametric
regression working model. This then also gives canonical gradient of
$\Psi_{{\cal M}_{w}}^{*}(P)=\tilde{\Psi}(P)-\Psi^{\\#}_{{\cal M}_{w}}(P)$. ∎

TMLE: Note,

$\displaystyle D_{{\cal M}_{w},P}^{*}=$ $\displaystyle\frac{2A-1}{g(A\mid
W)}(Y-\bar{Q}_{P}(S,W,A))+$ $\displaystyle\frac{2S-1}{\Pi_{P}(S\mid
W,A)}\left\\{E_{P}\Pi_{P}(0\mid W,1){\bf\phi}(W,1)\right\\}^{\top}{\cal
W}I_{P}^{-1}{\bf\phi}(Y-\bar{Q}_{P}(S,W,A))-$
$\displaystyle\frac{2S-1}{\Pi_{P}(S\mid W,A)}\left\\{E_{P}\Pi_{P}(0\mid
W,0){\bf\phi}(W,0)\right\\}^{\top}{\cal
W}I_{P}^{-1}{\bf\phi}(Y-\bar{Q}_{P}(S,W,A))+$
$\displaystyle\frac{2A-1}{g_{P}(A\mid W)}\tau_{P}(W,A)(S-\Pi_{P}(1\mid W,A))+$
$\displaystyle\left\\{-\frac{A}{g_{P}(1\mid
W)}\tau_{w,\beta}(W,1)+\frac{(1-A)}{g_{P}(0\mid
W)}\tau_{w,\beta}(W,0)\right\\}(S-\Pi_{P}(1\mid W,A))+$
$\displaystyle\left\\{E_{P}\Pi_{P}(0\mid
W,1){\bf\phi}(W,1)\right\\}^{\top}\left\\{{\cal
W}(\tau_{P}-m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal
W}(\tau_{P}-m_{\beta})I_{P}^{-1}{\bf\phi}\right\\}-$
$\displaystyle\left\\{E_{P}\Pi_{P}(0\mid
W,0){\bf\phi}(W,0)\right\\}^{\top}\left\\{{\cal
W}(\tau_{P}-m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal
W}(\tau_{P}-m_{\beta})I_{P}^{-1}{\bf\phi}\right\\}+$ $\displaystyle
E_{P}(Y\mid A=1,W)-E_{P}(Y\mid A=0,W)-\tilde{\Psi}(P)+$
$\displaystyle\Pi_{P}(0\mid W,1)\tau_{w,\beta(P)}(W,1)-\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)+\Psi_{{\cal M}_{w}}^{\\#}(P)$ $\displaystyle\equiv$
$\displaystyle
C_{Y}(g,\Pi_{P})(Y-\bar{Q}_{P}(S,W,A))+C_{S}(g,\bar{Q}_{P},\beta)(S-\Pi_{P}(1\mid
W,A))-$ $\displaystyle
c_{P}^{\top}\left\\{(\tau_{P}-m_{\beta})I_{P}^{-1}{\bf\phi}-P(\tau_{P}-m_{\beta})I_{P}^{-1}{\bf\phi}\right\\}+D_{{\cal
M}_{w},P,W}^{*}(W).$

Regarding computing a TMLE of $\tilde{\Psi}(P_{0})$ we can first target an
initial estimator $\bar{Q}_{n}$ of $E(Y\mid S,W,A)$ with clever covariate
$(2A-1)/g_{n}(A\mid W)$ to obtain a $\bar{Q}_{n}^{*}$, then construct a clever
covariate $(2A-1)/g_{n}(A\mid
W)(\bar{Q}_{n}^{*}(1,W,A)-\bar{Q}_{n}^{*}(0,W,A))$ to target an initial
estimator $\Pi_{n}(S\mid W,A)$ of $\Pi_{0}(S\mid W,A)$ into a $\Pi_{n}^{*}$,
and we use the empirical measure $P_{W,n}$ as estimator of $P_{W,0}$. This
results in a closed-form TMLE of $\tilde{\Psi}(P_{0})$ solving its canonical
gradient. Regarding computing a TMLE of $\Psi_{{\cal M}_{w,sp}}^{\\#}(P_{0})$,
we can target $\bar{Q}_{n}$ with a clever covariate $C_{Y}(\Pi_{n})$ as can be
read off from the second and third line in the above EIC representation. With
this $\bar{Q}_{n}^{*}$ we can project it on the working model minimizing
empirical mean of squared residuals giving a corresponding $\beta_{n}$. This
guarantees solving the 6-th and 7-th row of the EIC representation above. We
can then target $\Pi_{n}$ with a clever covariate $C(g_{n}(A\mid
W),\beta_{n})$, one can read off from line 5 in the above representation, to
obtain $\Pi_{n}^{*}$. Finally, we use the empirical distribution of $W$ as
estimator of $P_{W,0}$. We have then solved all components of
$P_{n}D^{\\#}_{{\cal M}_{w},P_{n}^{*}}\approx 0$ for $\Psi_{{\cal
M}_{w}}^{\\#}$ in an incompatible way since with $C_{Y}(\Pi_{n})$ uses
$\Pi_{n}$ while $\Pi_{n}^{*}$ is used in the other components. Notice that
this did not require any iteration. However, we could now redo the targeting
of $\bar{Q}_{n}^{*}$ with the updated clever covariate $C_{Y}(\Pi_{n}^{*})$.
After a few iterations one obtains the desired TMLE
$(\bar{Q}_{n}^{*},\beta_{n}^{*},\Pi_{n}^{*})$ solving $P_{n}D^{\\#}_{{\cal
M}_{w},P_{n}^{*}}\approx 0$. Finally, instead of doing a separate TMLE of
$\tilde{\Psi}(P_{0})$ and $\Psi^{\\#}_{{\cal M}_{w,n}}(P_{0})$ we could also
do a single plug-in of $\Psi_{{\cal M}_{w,n}}^{*}(P_{0})$. In that case, we
use $C_{Y}(g_{n},\Pi_{n})$ to target $\bar{Q}_{n}$ into a $\bar{Q}_{n}^{*}$;
map this $\bar{Q}_{n}^{*}$ into the $\beta_{n}$; target $\Pi_{n}$ with
$C_{S}(g_{n},\bar{Q}_{n}^{*},\beta_{n})$ into a $\Pi_{n}^{*}$, thereby
obtaining a first round $(\bar{Q}_{n}^{*},\beta_{n},\Pi_{n}^{*})$. We can
iterate this sequential targeting a few times till $P_{n}D^{*}_{{\cal
M}_{w},P_{n}^{*}}\approx 0$.

### Canonical gradient of $\Psi_{{\cal M}_{w,n}}$ for parametric working model
for $\bar{Q}_{P}$

In the definition of $\Psi_{{\cal M}_{w}}^{*}=\tilde{\Psi}-\Psi^{\\#}_{{\cal
M}_{w}}$, we used the projection only in the $\Psi^{\\#}_{{\cal M}_{w}}$
parameter, but not in $\tilde{\Psi}$. We now want to determine the canonical
gradient of

$\Psi_{{\cal M}_{w}}(P)\equiv\tilde{\Psi}_{{\cal M}_{w}}(P)-\Psi^{\\#}_{{\cal
M}_{w}}(P),$

where now

$\tilde{\Psi}_{{\cal M}_{w}}(P)=E_{P}\left\\{\sum_{s}\Pi_{P}(s\mid
W,A=1)m_{\beta(P)}(s,W,1)-\sum_{s}\Pi_{P}(s\mid
W,A=0)m_{\beta(P)}(s,W,0)\right\\}.$

######  Lemma 12.

Consider a parametric working model $\\{m_{\beta}:\beta\\}$ for
$\bar{Q}_{P}=E_{P}(Y\mid S,W,A)$, which we assume to be linear
$\beta^{\top}{\bf\phi}$ for the squared-error loss
$L(m_{\beta})=(Y-m_{\beta})^{2}$ when $Y$ is continuous and logistic linear
$\log m_{\beta}/(1-m_{\beta})=\beta^{\top}{\bf\phi}$ for the binary outcome
log-likelihood loss $L(m_{\beta})=-\log m_{\beta}^{Y}(1-m_{\beta})^{1-Y}$ when
$Y$ is binary or continuous in $(0,1)$. Let $D_{\beta,P}$ be the canonical
gradient of $\beta$ at $P$ as specified in Lemma 9. We have that the canonical
gradient of $\Psi_{{\cal M}_{w}}$ at $P$ is given by

$D_{{\cal M}_{w},P}=D_{{\cal M}_{w},\tilde{\Psi},P}-D^{\\#}_{{\cal M}_{w},P},$

where $D^{\\#}_{{\cal M}_{w},P}$ is defined in previous lemma, and

$D_{{\cal M}_{w},\tilde{\Psi},P}=D_{{\cal
M}_{w},\tilde{\Psi},p_{W},P}+D_{{\cal M}_{w},\tilde{\Psi},\beta,P}+D_{{\cal
M}_{w},\tilde{\Psi},\Pi,P},$

with

$\displaystyle D_{{\cal M}_{w},\tilde{\Psi},p_{W},P}=$
$\displaystyle\sum_{s}\Pi_{P}(s\mid
W,A=1)m_{\beta(P)}(s,W,1)-\sum_{s}\Pi_{P}(s\mid
W,A=0)m_{\beta(P)}(s,W,0)-\tilde{\Psi}_{{\cal M}_{w}}(P)$ $\displaystyle
D_{{\cal M}_{w},\tilde{\Psi},\beta,P}=$ $\displaystyle
E_{P}\left\\{\sum_{s}\Pi_{P}(s\mid
W,A=1)\frac{d}{d\beta}m_{\beta}(s,W,1)^{\top}\right\\}D_{\beta,P}(O)-$
$\displaystyle E_{P}\left\\{\sum_{s}\Pi_{P}(s\mid
W,A=0)\frac{d}{d\beta}m_{\beta}(s,W,0)^{\top}\right\\}D_{\beta,P}(O)$
$\displaystyle D_{{\cal M}_{w},\tilde{\Psi},\Pi,P}=$
$\displaystyle\frac{A}{g_{P}(1\mid
W)}(m_{\beta}(S=1,W,1)-m_{\beta}(S=0,W,1))(S-\Pi_{P}(1\mid W,1))-$
$\displaystyle\frac{1-A}{g_{P}(0\mid
W)}(m_{\beta}(S=1,W,0)-m_{\beta}(S=0,W,0))(S-\Pi_{P}(1\mid W,0)).$

###### Proof.

We already derived the canonical gradient of $\Psi^{\\#}_{{\cal M}_{w}}$ at
$P$, so it remains to determine the canonical gradient of $\tilde{\Psi}_{{\cal
M}_{w}}$. The $p_{W}$-score component of $D_{{\cal M}_{w},\tilde{\Psi},P}$ is
trivially verified. The $p_{Y\mid S,W,A}$-component of $D_{{\cal
M}_{w},\tilde{\Psi},P}$ is also easily verified. The influence curve of
$\Pi_{P}(s\mid w,a)$ is given by
$D_{\Pi,(s,w,a)}=I(W=w,A=a)/P(w,a)(I(S=s)-\Pi_{P}(s\mid w,a))$. Thus we obtain

$\displaystyle D_{{\cal M}_{w},\tilde{\Psi},\Pi,P}$
$\displaystyle=E_{P}\left\\{\sum_{s}D_{\Pi,(s,w,1)}m_{\beta(P)}(W,1,s)-\sum_{s}D_{\Pi,(s,w,0),P}m_{\beta(P)}(W,0,s)\right\\}$
$\displaystyle=\int_{w}dP(w)\sum_{s}I(W=w,A=1)/P(w,1)(I(S=s)-\Pi(s\mid
w,1))m_{\beta}(w,1,s)$
$\displaystyle-\int_{w}dP(w)\sum_{s}I(W=w,A=0)/P(w,0)(I(S=s)-\Pi(s\mid
w,0))m_{\beta}(w,0,s)$ $\displaystyle=\frac{I(A=1)}{P(A=1\mid
W)}\sum_{s}m_{\beta}(W,1,s)(I(S=s)-\Pi(s\mid W,1))$
$\displaystyle-\frac{I(A=0)}{P(A=0\mid
W)}\sum_{s}m_{\beta}(W,0,s)(I(S=s)-\Pi(s\mid W,0)).$

Note $\sum_{s}m_{\beta}(W,1,s)(I(S=s)-\Pi(s\mid
W,1))=(m_{\beta}(W,1,s=1)-m_{\beta}(W,1,s=0))(I(S=1)-\Pi(1\mid W,1))$.
Similarly, $\sum_{s}m_{\beta}(W,0,s)(I(S=s)-\Pi(s\mid
W,0))=(m_{\beta}(W,0,1)-m_{\beta}(W,0,s=0))(I(S=1)-\Pi(1\mid W,0))$. This
completes the proof of the lemma. ∎

TMLE: Plugging in our expression for $D_{\beta,P}$, results in the following
expression:

$\displaystyle D_{{\cal M}_{w,p},P}$
$\displaystyle=E_{P}\left\\{\sum_{s}\Pi_{P}(s\mid
W,A=1)\frac{d}{d\beta}m_{\beta}(s,W,1)^{\top}\right\\}I_{P}^{-1}{\bf\phi}(Y-m_{\beta})$
$\displaystyle-E_{P}\left\\{\sum_{s}\Pi_{P}(s\mid
W,A=0)\frac{d}{d\beta}m_{\beta}(s,W,0)^{\top}\right\\}I_{P}^{-1}{\bf\phi}(Y-m_{\beta})$
$\displaystyle+\left\\{E_{P}\Pi_{P}(0\mid
W,1)\frac{d}{d\beta(P)}\tau_{w,\beta(P)}(W,1)\right\\}^{\top}I_{P}^{-1}{\bf\phi}(Y-m_{\beta})$
$\displaystyle-\left\\{E_{P}\Pi_{P}(0\mid
W,0)\frac{d}{d\beta(P)}\tau_{w,\beta(P)}(W,0)\right\\}^{\top}I_{P}^{-1}{\bf\phi}(Y-m_{\beta})$
$\displaystyle-2\frac{1-A}{g_{P}(0\mid
W)}(m_{\beta}(S=1,W,0)-m_{\beta}(S=0,W,0))(S-\Pi_{P}(1\mid W,0))$
$\displaystyle+\Pi_{P}(0\mid W,1)\tau_{w,\beta(P)}(W,1)-\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)+\Psi_{{\cal M}_{w}}^{\\#}(P)$
$\displaystyle+\sum_{s}\Pi_{P}(s\mid
W,1)m_{\beta(P)}(W,1,s)-\sum_{s}\Pi_{P}(s\mid
W,0)m_{\beta(P)}(W,0,s)-\tilde{\Psi}_{{\cal M}_{w}}(P)$ $\displaystyle\equiv
C_{Y}(\Pi_{P})(Y-\beta^{\top}{\bf\phi})+C_{S}(g_{P},\beta)(S-\Pi_{P}(1\mid
W,A))+D_{{\cal M}_{w,p},P,W}(W).$

Given initial estimators $\beta_{n},\Pi_{n},g_{n}$, we can use
$C_{Y}(\Pi_{n})$, which is a particular linear combination of ${\bf\phi}$, to
target $\beta_{n}$ into a $\beta_{n}^{*}$ that solves
$P_{n}C_{Y}(\Pi_{n})(Y-\beta_{n}^{*\top}{\bf\phi})=0$. Note if $\beta_{n}$ is
already a least squares estimator, or logistic regression MLE, then this score
equation is already solved by $\beta_{n}$, but in case we use lasso-
penalization, then this targeting step is important. If one uses first lasso
and then refit the resulting working model with MLE (i.e., relaxed-lasso),
then the same remark applies. Given $(g_{n},\beta_{n}^{*})$, we can use
$C_{S}(g_{n},\beta_{n}^{*})$ to target $\Pi_{n}$ into a $\Pi_{n}^{*}$ solving
$P_{n}C_{S}(g_{n},\beta_{n}^{*})(S-\Pi_{n}^{*}(1\mid W,A))=0$. By using the
empirical distribution of $W$, we already have $P_{n}D_{{\cal
M}_{w,p},P_{n}^{*},W}=0$. Note that if $\beta_{n}$ already was an MLE, then
this describes a closed-form TMLE $P_{n}^{*}$, and no iteration is needed. In
general, we can iterate this sequential targeting of $\beta_{n}$ and $\Pi_{n}$
a few times to obtain the desired solution $P_{n}D_{{\cal
M}_{w,p},P_{n}^{*}}\approx 0$.

### Canonical gradient of $\Psi_{{\cal M}_{w}}$ for semiparametric regression
working model for $\bar{Q}_{P}$

We have the following lemma.

######  Lemma 13.

Let $m_{\beta}(w,a,s)=sm_{\beta}(w,a)$. Given a working model
$m_{\beta}=\beta{\bf\phi}$ for
$\tau_{P}(A,W)=\bar{Q}_{P}(1,W,A)-\bar{Q}_{P}(0,W,A)$, let $D_{\beta,P}$ be
the canonical gradient of $\beta(P)$ defined by

$\beta(P)\equiv\arg\min_{\beta}E_{P}{\cal
W}\left\\{\tau_{P}(W,A)-m_{\beta}(W,A)\right\\}^{2}.$

This is presented in above lemma. We have that the canonical gradient of
$\Psi_{{\cal M}_{w}}$ at $P$ is given by

$D_{{\cal M}_{w},P}=D_{{\cal M}_{w},\tilde{\Psi},P}-D^{\\#}_{{\cal M}_{w},P},$

where $D^{\\#}_{{\cal M}_{w},P}$ is defined in previous lemma, and

$D_{{\cal M}_{w},\tilde{\Psi},P}=D_{{\cal
M}_{w},\tilde{\Psi},p_{W},P}+D_{{\cal M}_{w},\tilde{\Psi},\beta,P}+D_{{\cal
M}_{w},\tilde{\Psi},\theta,P}+D_{{\cal M}_{w},\tilde{\Psi},\Pi,P},$

with

$\displaystyle D_{{\cal M}_{w},\tilde{\Psi},p_{W},P}$
$\displaystyle=\left\\{\Pi_{P}(1\mid W,1)m_{\beta(P)}(W,1)-\Pi_{P}(1\mid
W,0)m_{\beta(P)}(W,0)\right\\}-\tilde{\Psi}_{{\cal M}_{w}}(P)$ $\displaystyle
D_{{\cal M}_{w},\tilde{\Psi},\beta,P}$ $\displaystyle=E_{P}\Pi_{P}(1\mid
W,1){\bf\phi}(W,1)^{\top}D_{\beta,P}(O)-E_{P}\Pi_{P}(1\mid
W,0){\bf\phi}(W,0)^{\top}D_{\beta,P}(O)$ $\displaystyle D_{{\cal
M}_{w},\tilde{\Psi},\theta,P}$ $\displaystyle=\frac{I(S=0)}{\Pi_{P}(S\mid
W,A)}\frac{2A-1}{g_{P}(A\mid W)}(Y-\bar{Q}_{P}(S,W,A))$ $\displaystyle
D_{{\cal M}_{w},\tilde{\Psi},\Pi,P}$ $\displaystyle=\frac{2A-1}{g_{P}(A\mid
W)}m_{\beta}(W,A)(S-\Pi_{P}(1\mid W,A)).$

###### Proof.

Given our derivation of the canonical gradient of $\Psi^{\\#}_{{\cal M}_{w}}$
at $P$, to obtain the canonical gradient of $\Psi_{{\cal
M}_{w}}=\tilde{\Psi}_{{\cal M}_{w}}-\Psi_{{\cal M}_{w}}^{\\#}$ at $P$ it
suffices to determine the canonical gradient of

$\tilde{\Psi}_{{\cal M}_{w}}(P)=E_{P}\sum_{s}\Pi_{P}(s\mid
W,1)\bar{Q}_{\theta(P),\beta(P)}(s,W,1)-E_{P}\sum_{s}\Pi_{P}(s\mid
W,0)\bar{Q}_{\theta(P),\beta(P)}(s,W,0),$

where $\theta(P)=E_{P}(Y\mid S=0,W,A)$, and
$\bar{Q}_{\theta,\beta}(S,W,A)=\theta(W,A)+Sm_{\beta}(W,A)$. The components of
this canonical gradient due to dependence of $\tilde{\Psi}_{{\cal M}_{w}}$ on
$p_{W}$ and $\Pi$ are identical to the ones presented earlier. For the
contribution of $\beta(P$) in $D_{{\cal M}_{w},\tilde{\Psi}}$ we use the
analogue expression we obtained earlier for this canonical gradient with the
parametric working model $m_{\beta}(s,w,a)$, but replacing $m_{\beta}(s,w,a)$
by our $m_{\beta}(s,w,a)=sm_{\beta}(w,a)$ for $\tau_{P}$.

It remains to determine the contribution from
$P\rightarrow\bar{Q}_{\theta(P),\beta}$. The canonical gradient of
$\theta(P)(a,w)$ is given by $I(S=0,W=w,A=a)/p(s=0,w,a)(Y-\theta(P)(W,A))$. So
we obtain

$\displaystyle D_{{\cal M}_{w},\tilde{\Psi},\theta,P}$
$\displaystyle=\int_{w}dP(w)\sum_{s}\Pi_{P}(s\mid
w,1)\frac{I(S=0,W=w,A=1)}{p(s=0,w,1)}(Y-\theta(W,A))$
$\displaystyle-\int_{w}dP(w)\sum_{s}\Pi_{P}(s\mid
w,0)\frac{I(S=0,W=w,A=0)}{p(s=0,w,0)}(Y-\theta(W,A))$
$\displaystyle=\frac{I(S=0,A=1)}{p(S=0,A=1\mid
W)}(Y-\theta(W,A))\sum_{s}\Pi_{P}(s\mid W,1)$
$\displaystyle-\frac{I(S=0,A=0)}{p(S=0,A=0\mid
W)}(Y-\theta(W,A))\sum_{s}\Pi_{P}(s\mid W,0)$
$\displaystyle=\left\\{\frac{I(S=0,A=1)}{p(S=0,A=1\mid
W)}-\frac{I(S=0,A=0)}{p(S=0,A=0\mid W)}\right\\}(Y-\theta(P)(W,A)).$

This completes the proof. ∎

TMLE: We note that the $D_{\beta,P}$ contributions in $\tilde{\Psi}_{{\cal
M}_{w}}$ and $\Psi^{\\#}_{{\cal M}_{w}}$ can be combined:

$\displaystyle E_{P}\Pi_{P}(1\mid
W,1){\bf\phi}(W,1)^{\top}D_{\beta,P}(O)-E_{P}\Pi_{P}(1\mid
W,0){\bf\phi}(W,0)^{\top}D_{\beta,P}(O)$ $\displaystyle+E_{P}\Pi_{P}(0\mid
W,1){\phi}^{\top}(W,1)D_{\beta,P}(O)-E_{P}\Pi_{P}(0\mid
W,0){\phi}^{\top}(W,0)D_{\beta,P}(O)$
$\displaystyle=E_{P}({\bf\phi}(W,1)-{\bf\phi}(W,0))^{\top}D_{\beta,P}$
$\displaystyle\equiv c_{P}^{\top}{\cal W}\frac{2S-1}{\Pi(S\mid
A,W)}I_{P}^{-1}{\bf\phi}(Y-\bar{Q}_{P})+c_{P}^{\top}\left\\{{\cal W}(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal W}\\{(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}\\}\right\\}.$

Thus, summing all the terms in $D_{{\cal M}_{w},P}=D_{{\cal
M}_{w},\tilde{\Psi},P}-D_{{\cal M}_{w},P}^{\\#}$ and using the above, and that
there is perfect cancelation of the contributions from $\Pi_{P}$ yields the
following:

$\displaystyle D_{{\cal M}_{w},P}$ $\displaystyle=\Pi_{P}(1\mid
W,1)m_{\beta(P)}(W,1)-\Pi_{P}(1\mid W,0)m_{\beta(P)}(W,0)-\tilde{\Psi}_{{\cal
M}_{w}}(P)$ $\displaystyle+E_{P}\Pi_{P}(1\mid
W,1){\bf\phi}(W,1)^{\top}D_{\beta,P}(O)-E_{P}\Pi_{P}(1\mid
W,0){\bf\phi}(W,0)^{\top}D_{\beta,P}(O)$
$\displaystyle+\frac{I(S=0)}{\Pi(S\mid W,A)}\frac{2A-1}{g_{P}(A\mid
W)}(Y-\bar{Q}_{P}(S,W,A))+\frac{2A-1}{g_{P}(A\mid
W)}m_{\beta}(W,A)(S-\Pi_{P}(1\mid W,A))$
$\displaystyle-\left\\{-E_{P}\Pi_{P}(0\mid
W,1){\phi}^{\top}(W,1)\right\\}D_{\beta,P}(O)-\left\\{E_{P}\Pi_{P}(0\mid
W,0){\phi}^{\top}(W,0)\right\\}D_{\beta,P}(O)$ $\displaystyle+\Pi_{P}(0\mid
W,1)\tau_{w,\beta(P)}(W,1)-\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)+\Psi_{{\cal M}_{w}}^{\\#}(P)$
$\displaystyle-\frac{2A-1}{g_{P}(A\mid W)}m_{\beta}(W,A)(S-\Pi_{P}(1\mid
W,A))$ $\displaystyle=c_{P}^{\top}{\cal W}\frac{2S-1}{\Pi_{P}(S\mid
A,W)}I_{P}^{-1}{\bf\phi}(Y-\bar{Q}_{P})+c_{P}^{\top}\left\\{{\cal W}(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal W}\\{(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}\\}\right\\}$
$\displaystyle+\left\\{\Pi_{P}(1\mid W,1)m_{\beta(P)}(W,1)-\Pi_{P}(1\mid
W,0)m_{\beta(P)}(W,0)\right\\}-\tilde{\Psi}_{{\cal M}_{w}}(P)$
$\displaystyle+\frac{I(S=0)}{\Pi_{P}(S\mid W,A)}\frac{2A-1}{g(A\mid
W)}(Y-\bar{Q}_{P}(S,W,A))+\frac{2A-1}{g(A\mid
W)}m_{\beta}(W,A)(I(S=1)-\Pi(1\mid W,A))$ $\displaystyle+\Pi_{P}(0\mid
W,1)\tau_{w,\beta(P)}(W,1)-\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)+\Psi_{{\cal M}_{w}}^{\\#}(P)$
$\displaystyle-\frac{2A-1}{g_{P}(A\mid W)}m_{\beta}(W,A)(S-\Pi_{P}(1\mid
W,A))$ $\displaystyle=c_{P}^{\top}{\cal W}\frac{2S-1}{\Pi(S\mid
A,W)}I_{P}^{-1}{\bf\phi}(Y-\bar{Q}_{P})+\frac{I(S=0)}{\Pi_{P}(S\mid
W,A)}\frac{2A-1}{g_{P}(A\mid W)}(Y-\bar{Q}_{P}(S,W,A))$
$\displaystyle+c_{P}^{\top}\left\\{{\cal W}(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal W}\\{(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}\\}\right\\}$
$\displaystyle+\left\\{\Pi_{P}(1\mid W,1)m_{\beta(P)}(W,1)-\Pi_{P}(1\mid
W,0)m_{\beta(P)}(W,0)\right\\}-\tilde{\Psi}_{{\cal M}_{w}}(P)$
$\displaystyle+\Pi_{P}(0\mid W,1)\tau_{w,\beta(P)}(W,1)-\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)+\Psi_{{\cal M}_{w}}^{\\#}(P)$ $\displaystyle\equiv
C_{Y}(\Pi_{P},g_{P})(Y-\bar{Q}_{P})+c_{P}^{\top}\left\\{{\cal W}(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal W}\\{(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}\\}\right\\}+D_{{\cal M}_{w},W,P}.$

######  Lemma 14.

Consider the setting of previous lemma. We have

$\displaystyle D_{{\cal M}_{w},P}$
$\displaystyle=C_{Y}(\Pi,g)(Y-\bar{Q}_{P})+c_{P}^{\top}\left\\{{\cal W}(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}-P{\cal W}(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}\right\\}+D_{{\cal M}_{w},W,P},$

where

$\displaystyle C_{Y}(\Pi,g)$ $\displaystyle\equiv c_{P}^{\top}{\cal
W}\frac{2S-1}{\Pi_{P}(S\mid
A,W)}I_{P}^{-1}{\bf\phi}+\frac{I(S=0)}{\Pi_{P}(S\mid W,A)}\frac{2A-1}{g(A\mid
W)}$ $\displaystyle c_{P}^{\top}$ $\displaystyle\equiv
E_{P}({\bf\phi}(W,1)-{\bf\phi}(W,0))^{\top}$ $\displaystyle D_{{\cal
M}_{w},W,P}$ $\displaystyle\equiv\Pi_{P}(1\mid
W,1)m_{\beta(P)}(W,1)-\Pi_{P}(1\mid W,0)m_{\beta(P)}(W,0)-\tilde{\Psi}_{{\cal
M}_{w}}(P)$ $\displaystyle+\Pi_{P}(0\mid
W,1)\tau_{w,\beta(P)}(W,1)-\Pi_{P}(0\mid
W,0)\tau_{w,\beta(P)}(W,0)+\Psi_{{\cal M}_{w}}^{\\#}(P).$

Given initial estimators $g_{n},\Pi_{n},\bar{Q}_{n}$, we can target
$\bar{Q}_{n}$ with clever covariate $C_{Y}(\Pi_{n},g_{n})$ to obtain a
targeted $\bar{Q}_{n}^{*}$ that solves
$P_{n}C_{Y}(\Pi_{n},g_{n})(Y-\bar{Q}_{n}^{*})=0$. We map $\bar{Q}_{n}^{*}$
into $\beta_{n}$ by minimizing $P_{n}(\tau_{n}^{*}-m_{\beta})^{2}$, where
$\tau_{n}^{*}$ is the estimator of $\tau_{0}$ implied by $\bar{Q}_{n}^{*}$.
This $\beta_{n}$ solves the empirical mean of $c_{P}^{\top}\left\\{(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}-P\\{(\tau-
m_{\beta})I_{P}^{-1}{\bf\phi}\\}\right\\}$. The empirical mean of $D_{{\cal
M}_{w},W,P_{n}^{*}}$ equals zero due to estimating the distribution of $W$
with the empirical measure. Thus, this TMLE requires no iteration and is a
closed-form simple one-step TMLE.

## References

  * [1] US Congress  “21st Century Cures Act. HR 34, 114th Congress”, 2016
  * [2] US FDA  “Framework for FDA’s real-world evidence program”  In _Silver Spring, MD: US Department of Health and Human Services Food and Drug Administration_ , 2018
  * [3] J. Lin, G. Yu and M. Gamalo  “Matching within a hybrid RCT/RWD: framework on associated causal estimands”  In _Journal of Biopharmaceutical Statistics_ 33.4 Taylor & Francis, 2023, pp. 439–451
  * [4] K.. Rudolph and M.. van der Laan  “Robust estimation of encouragement design intervention effects transported across sites”  In _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 79.5 Oxford University Press, 2017, pp. 1509–1525
  * [5] I.. Dahabreh et al.  “Generalizing causal inferences from individuals in randomized trials to all trial-eligible individuals”  In _Biometrics_ 75.2 Oxford University Press, 2019, pp. 685–694
  * [6] L. van der Laan, M. Carone, A. Luedtke and M.. van der Laan  “Adaptive debiased machine learning using data-driven model selection techniques”  In _arXiv preprint arXiv:2307.12544_ , 2023
  * [7] M.. van der Laan, D. Benkeser and W. Cai  “Efficient estimation of pathwise differentiable target parameters with the undersmoothed highly adaptive lasso”  In _The International Journal of Biostatistics_ 19.1 De Gruyter, 2023, pp. 261–289
  * [8] K. Viele et al.  “Use of historical control data for assessing treatment effects in clinical trials”  In _Pharmaceutical statistics_ 13.1 Wiley Online Library, 2014, pp. 41–54
  * [9] S. Yang, C. Gao, D. Zeng and X. Wang  “Elastic integrative analysis of randomised trial and real-world data for treatment heterogeneity estimation”  In _Journal of the Royal Statistical Society Series B: Statistical Methodology_ 85.3 Oxford University Press US, 2023, pp. 575–596
  * [10] W. Li, F. Liu and D. Snavely  “Revisit of test-then-pool methods and some practical considerations”  In _Pharmaceutical statistics_ 19.5 Wiley Online Library, 2020, pp. 498–517
  * [11] S.. Pocock  “The combination of randomized and historical controls in clinical trials”  In _Journal of chronic diseases_ 29.3 Elsevier, 1976, pp. 175–188
  * [12] J.. Ibrahim and M. Chen  “Power prior distributions for regression models”  In _Statistical Science_ JSTOR, 2000, pp. 46–60
  * [13] B.. Hobbs, D.. Sargent and B.. Carlin  “Commensurate priors for incorporating historical information in clinical trials using general and generalized linear models”  In _Bayesian Analysis (Online)_ 7.3 NIH Public Access, 2012, pp. 639
  * [14] H. Schmidli et al.  “Robust meta-analytic-predictive priors in clinical trials with historical control information”  In _Biometrics_ 70.4 Oxford University Press, 2014, pp. 1023–1032
  * [15] X. Lin and R.. Evans  “Many Data: Combine Experimental and Observational Data through a Power Likelihood”  In _arXiv preprint arXiv:2304.02339_ , 2023
  * [16] L.. Dang et al.  “A Cross-Validated Targeted Maximum Likelihood Estimator for Data-Adaptive Experiment Selection Applied to the Augmentation of RCT Control Arms with External Data”  In _arXiv preprint arXiv:2210.05802_ , 2022
  * [17] S. Chen, B. Zhang and T. Ye  “Minimax rates and adaptivity in combining experimental and observational data”  In _arXiv preprint arXiv:2109.10522_ , 2021
  * [18] D. Cheng and T. Cai  “Adaptive combination of randomized and observational data”  In _arXiv preprint arXiv:2111.15012_ , 2021
  * [19] N. Kallus, A.. Puli and U. Shalit  “Removing hidden confounding by experimental grounding”  In _Advances in neural information processing systems_ 31, 2018
  * [20] L. Wu and S. Yang  “Integrative $R$-learner of heterogeneous treatment effects combining experimental and observational studies”  In _Conference on Causal Learning and Reasoning_ , 2022, pp. 904–926  PMLR
  * [21] C. Shyr, B. Ren, P. Patil and G. Parmigiani  “Multi-study R-learner for Heterogeneous Treatment Effect Estimation”  In _arXiv preprint arXiv:2306.01086_ , 2023
  * [22] A. Schuler et al.  “Increasing the efficiency of randomized trial estimates via linear adjustment for a prognostic score”  In _The International Journal of Biostatistics_ 18.2 De Gruyter, 2022, pp. 329–356
  * [23] D. Benkeser and M.. van der Laan  “The highly adaptive lasso estimator”  In _2016 IEEE international conference on data science and advanced analytics (DSAA)_ , 2016, pp. 689–696  IEEE
  * [24] N.. Hejazi, J.. Coyle and M.. van der Laan  “hal9001: Scalable highly adaptive lasso regression inR”  In _Journal of Open Source Software_ 5.53, 2020, pp. 2526
  * [25] M.. van der Laan, E.. Polley and A.. Hubbard  “Super learner”  In _Statistical applications in genetics and molecular biology_ 6.1 De Gruyter, 2007
  * [26] M.. van der Laan et al.  “Targeted Learning in R: Causal Data Science with the tlverse Software Ecosystem”, 2020
  * [27] A.. Bibaut and M.. van der Laan  “Fast rates for empirical risk minimization over c$\backslash$adl$\backslash$ag functions with bounded sectional variation norm”  In _arXiv preprint arXiv:1907.09244_ , 2019
  * [28] J. Friedman, T. Hastie and R. Tibshirani  “Regularization paths for generalized linear models via coordinate descent”  In _Journal of statistical software_ 33.1 NIH Public Access, 2010, pp. 1
  * [29] A.. Hubbard, S. Kherad-Pajouh and M.. van der Laan  “Statistical inference for data adaptive target parameters”  In _The international journal of biostatistics_ 12.1 De Gruyter, 2016, pp. 3–19
  * [30] Z. Wang, W. Zhang and M.. van der Laan  “Super Ensemble Learning Using the Highly-Adaptive-Lasso”  In _arXiv preprint arXiv:2312.16953_ , 2023