TSIATIS and DAVIDIAN

*Marie Davidian, Department of Statistics, North Carolina State University, Raleigh, NC 27695-8203, USA.

# Group sequential methods for interim monitoring of randomized clinical
trials with time-lagged outcome

Anastasios A. Tsiatis  Marie Davidian  Department of Statistics, North
Carolina State University, North Carolina, USA<EMAIL_ADDRESS>
(26 April 2022; 6 June 2022; 6 June 2022)

###### Abstract

[Summary]The primary analysis in two-arm clinical trials usually involves
inference on a scalar treatment effect parameter; e.g., depending on the
outcome, the difference of treatment-specific means, risk difference, risk
ratio, or odds ratio. Most clinical trials are monitored for the possibility
of early stopping. Because ordinarily the outcome on any given subject can be
ascertained only after some time lag, at the time of an interim analysis,
among the subjects already enrolled, the outcome is known for only a subset
and is effectively censored for those who have not been enrolled sufficiently
long for it to be observed. Typically, the interim analysis is based only on
the data from subjects for whom the outcome has been ascertained. A goal of an
interim analysis is to stop the trial as soon as the evidence is strong enough
to do so, suggesting that the analysis ideally should make the most efficient
use of all available data, thus including information on censoring as well as
other baseline and time-dependent covariates in a principled way. A general
group sequential framework is proposed for clinical trials with a time-lagged
outcome. Treatment effect estimators that take account of censoring and
incorporate covariate information at an interim analysis are derived using
semiparametric theory and are demonstrated to lead to stronger evidence for
early stopping than standard approaches. The associated test statistics are
shown to have the independent increments structure, so that standard software
can be used to obtain stopping boundaries.

###### keywords:

augmented inverse probability weighting, early stopping, influence function,
proportion of information

††articletype: Research Article

##  1 Introduction

In many randomized clinical trials, the primary analysis involves a comparison
of two treatments, typically an active or experimental agent versus a control,
which is formalized as inference on a scalar treatment effect parameter. When
the primary outcome is a continuous measure, this parameter is usually the
difference of treatment-specific means. For a binary outcome, the treatment
effect parameter may be the risk difference, risk ratio, or odds ratio; and
the odds ratio under the assumption of a proportional odds model is often the
treatment effect parameter of interest in trials involving an ordinal
categorical outcome. The primary analysis is ordinarily based on a test
statistic constructed using an estimator for the parameter of interest, e.g.,
the difference of sample means or maximum likelihood (ML) estimator for the
odds ratio in a proportional odds model. The overall sample size is
established so that the power to detect a clinically meaningful departure from
the hypothesis of no treatment effect at a given level of significance using
the test statistic at the final analysis achieves some desired value, e.g.,
90%.

Most later-stage clinical trials are monitored for the possibility of early
stopping for efficacy or futility by a data and safety monitoring board
(DSMB), with interim analyses planned at either fixed, predetermined analysis
times or when specified proportions of the total “statistical information” to
be gained from the completed trial have accrued1. Ordinarily, at the time of
an interim analysis, the test statistic to be used for the final analysis is
computed based on the available data and compared to a suitable stopping
boundary constructed to preserve the overall operating characteristics of the
trial2, 3.

Because of staggered entry into the trial, the data available at the time of
an interim analysis are from subjects who have already enrolled. Moreover, the
primary outcome $Y$ is ordinarily not known immediately but is ascertained
after some lag time $T$, say. In some trials, the lag time $T$ is the same for
all participants, as in the case where $Y$ is a continuous outcome that will
be measured at a prespecified follow-up time $\mathcal{T}_{F}$, e.g.,
$\mathcal{T}_{F}=$ one year, so that $T=\mathcal{T}_{F}$ for all subjects, and
the treatment parameter is the difference in treatment means of $Y$ at one
year. Here, at the time of an interim analysis, $Y$ will be available only for
subjects enrolled for at least one year, so that the analysis can be based
only on the data for these subjects.

In other settings, the time lag $T$ may be different for different
participants, as is the case in many clinical trials of COVID-19 therapeutics
conducted by the Accelerating COVID-19 Therapeutic Interventions and Vaccines
(ACTIV) public-private partnership. In an ongoing clinical trial coordinated
through the ACTIV-3b: Therapeutics for Severely Ill Inpatients with COVID-19
(TESICO) master protocol4, patients hospitalized with acute respiratory
distress syndrome are randomized to receive an active agent or placebo and
followed for up to $\mathcal{T}_{F}=90$ days. The primary outcome $Y$ is an
ordinal categorical variable with six levels. The first five categories
reflect a subject’s status at 90 days following enrollment: 1, at home and off
oxygen for at least 77 days (the most favorable category); 2, at home and off
oxygen for at least 49 but no more than 76 days; 3, at home and off oxygen for
at least 1 but no more than 48 days; 4, not hospitalized and either at home on
oxygen or receiving care elsewhere; and 5, still hospitalized or in hospice
care. Category 6 (the worst) corresponds to death within the 90 day follow-up
period. While Categories 1-5 cannot be ascertained until a subject has been
followed for the full 90 days, that a subject’s outcome is Category 6 is known
at the time of death. Thus, the time lag before ascertainment is
$T=\mathcal{T}_{F}=90$ days for subjects with $Y=1,\ldots,5$ and is equal to
the (random) time of death $T\leq\mathcal{T}_{F}=90$ for those with $Y=6$. In
TESICO, the treatment effect parameter is the odds ratio for active agent
relative to placebo under an assumed proportional odds model. Similarly, in a
clinical trial coordinated through the ACTIV-2: A Study for Outpatients With
COVID-19 master protocol (Study A5401)5, subjects within seven days of self-
reported COVID-19 onset are randomized to receive an active agent or placebo
and followed for up to $\mathcal{T}_{F}=28$ days for the binary outcome $Y$,
where $Y=1$ if the subject dies or is or hospitalized within 28 days and $Y=0$
otherwise. For subjects who die or are hospitalized at time $T$ prior to 28
days, $Y=1$ is ascertained after a time lag $T\leq\mathcal{T}_{F}=28$, whereas
that $Y=0$ can be ascertained only after the full 28 days, and
$T=\mathcal{T}_{F}=28$. Here, the treatment parameter is the relative risk
(risk ratio) of hospitalization/death for active agent versus placebo.

At the time of an interim analysis in TESICO and A5401, the available data
include the outcomes for all enrolled subjects who have been followed for at
least 90 and 28 days, so for whom $T=\mathcal{T}_{F}=90$ or 28, respectively,
along with the outcomes for enrolled subjects who do not have 90 or 28 days of
follow up but have already been observed to die ($Y=6$) in TESICO or to be
hospitalized or die ($Y=1$) in A5401 ($T\leq\mathcal{T}_{F}=90$ or 28). Thus,
information on Category 6 in TESICO will accumulate more rapidly than that on
the other categories; similarly, information on hospitalization/death in A5401
will accrue more quickly than information on subjects who remain alive and
unhospitalized at day 28. Intuitively, basing an interim analysis on all
observed outcomes will naively overrepresent $Y=6$ and $Y=1$ and lead to
potentially biased inference on the treatment effect parameters.

To characterize this issue more precisely, if $C$ is the time from a subject’s
entry into the study to the time of an interim analysis, then $Y$ is known at
the time of an interim analysis if $C>T$. Otherwise, the ascertainment time
$T$ is censored at $C$ and $Y$ is not observed. Basing the analysis on all
subjects with $C>T$ without taking appropriate account of the fact that $Y$ is
not available for those with $C\leq T$ leads to the bias noted above. These
considerations suggest that a valid interim analysis can be obtained by using
only the data from enrolled subjects followed for the full, maximum follow-up
period $\mathcal{T}_{F}$, i.e., for whom $C\geq\mathcal{T}_{F}$. In studies
like those above involving a continuous outcome or ordinal categorical
outcome, as in TESICO, the standard interim analysis is based on the estimator
to be used at the final analysis using only the data on subjects with
$C\geq\mathcal{T}_{F}$, as there is no apparent general approach to
“adjusting” for the censoring. In the case of a binary outcome as in A5401,
the standard interim analysis does use the information on censoring; e.g., if
the treatment effect is the risk ratio, the estimator is the ratio of the
treatment-specific Kaplan-Meier estimators for the probability of death or
hospitalization at $\mathcal{T}_{F}=28$ days.

A goal of an interim analysis is to stop the trial as early as possible if
there is sufficiently strong evidence to do so. It is thus natural to consider
whether or not it is possible to make more efficient use of the available data
at the time of an interim analysis to enhance precision and thus the strength
of the evidence for stopping. One step toward increasing efficiency of interim
analyses would be a general approach to accounting for censoring for any
outcome and treatment effect parameter to allow incorporation of partial
information; e.g., in TESICO, a subject who is at day 45 since study entry and
still in the hospital at the time of an analysis, so for whom
$C=45<\mathcal{T}_{F}=90$, can end up only in Categories 3–6, so have
$Y=3,4,5$, or 6, at 90 days. In addition, there may be baseline covariates as
well as intermediate measures of the outcome or other post-treatment variables
that could be exploited to increase precision at an interim analysis.

In this article, we propose a general group sequential framework for clinical
trials with a possibly censored, time-lagged outcome, which leads to practical
strategies for interim monitoring. Treatment effect estimators are proposed
via application of semiparametric theory6, 7, which dictates how censoring can
be taken into account and baseline and time-dependent covariate information
can be exploited in a principled way to increase precision and thus yield
stronger evidence for early stopping. Estimation of the risk ratio via
treatment-specific Kaplan-Meier estimators as described above emerges as a
simple special case, which can be improved upon through incorporation of
covariates. We show that the test statistics based on these estimators have an
independent increments structure8, which allows standard software for
constructing stopping boundaries2, 3, 9 to be used. Two interim monitoring
strategies are discussed: an information-based monitoring approach under which
the trial will continue, with possibly a larger sample size than originally
planned, until the full, target statistical information accrues; and a fixed-
sample size approach appropriate in settings where the planned sample size
cannot be increased due to resource and other constraints. We focus on the
common case of two treatments; extension of the developments to more than two
treatments is possible10 and could be adapted to group sequential methods for
multi-arm trials11, 12.

In Section 2 we introduce the basic statistical framework and assumptions, and
we sketch the estimation approach and state the independent increments
property in Section 3. In Section 4, we describe practical implementation of
the resulting approach to interim monitoring. We demonstrate the performance
of the methods in a series of simulation studies in Section 5, and we present
a case study exemplifying the use of the methods for a simulated trial based
on TESICO. Technical details and sketches of proofs of results are given in
the Appendix.

##  2 Statistical framework

###  2.1 General model

As in Section 1, denote the outcome by $Y$. Let $A$ denote the treatment
indicator, where $A=0$ (1) corresponds to control (active/experimental
treatment), and $\pi=\mbox{pr}(A=1)$ is the probability of being assigned to
active treatment; and let $X$ be a vector of baseline covariates. Treatment
effects are often characterized in terms of a model for (features of) the
distribution of $(Y,A)$ or $(Y,A,X)$, which involves parameters
$(\alpha^{T},\beta)$, where $\beta$ is the scalar treatment effect parameter
of interest and $\alpha$ is a vector of nuisance parameters, and the model is
parameterized such that $\beta=0$ corresponds to the null hypothesis of no
treatment effect.

In the case of the first example in Section 1 of continuous $Y$,
$\beta=E(Y|A=1)-E(Y|A=0)$; equivalently,

$E(Y|A=a)=\alpha+\beta a,\hskip 10.84006pta=0,1.$ (1)

For an ordinal categorical outcome with $c$ categories, as in TESICO with
$c=6$, the outcome can be represented as either a scalar random variable $Y$
taking values $1,\ldots,c$ or a random vector
$Y=\\{I(Cat=1),\ldots,I(Cat=c-1)\\}$, where $Cat$ takes values $1,\ldots,c$.
Using the first definition, the treatment effect can be defined through an
assumed proportional odds model

$\mbox{logit}\\{\mbox{pr}(Y\leq j|A=a)\\}=\alpha_{j}+\beta a,\hskip
10.84006ptj=1,\ldots,c-1,\hskip
10.84006pt\alpha=(\alpha_{1},\ldots,\alpha_{c-1})^{T},\hskip 10.84006pta=0,1,$
(2)

where $\mbox{logit}(p)=\log\\{p/(1-p)\\}$, so that $\beta$ is the log odds
ratio of interest. If the conditional (on $X$) treatment effect is of
interest, replace (2) by $\mbox{logit}\\{\mbox{pr}(Y\leq
j|X=x,A=a)\\}=\alpha_{j}+\beta a+\xi^{T}x$, where now
$\alpha=(\alpha_{1},\ldots,\alpha_{c-1},\xi^{T})^{T}$. If $Y$ is binary as in
A5401 and the relative risk (risk ratio)
$\mbox{pr}(Y=1|A=1)/\mbox{pr}(Y=1|A=0)$ is the focus, taking

$E(Y|A=a)=\exp(\alpha+\beta a),\hskip 10.84006pta=0,1,$ (3)

corresponds to log relative risk $\beta$.

In general, estimators for the parameter of interest $\beta$ in (1)-(3) and
other models based on the data available at the final analysis, at which time
$(Y,A,X)$ are known for all $n$ participants, are obtained by solving, jointly
in $\alpha$ and $\beta$, appropriate estimating equations. Namely, with
independent and identically distributed (iid) data $(Y_{i},A_{i},X_{i})$,
$i=1,\ldots,n$, available, and $p$ equal to the dimension of
$(\alpha^{T},\beta)^{T}$, $\widehat{\alpha}$ and $\widehat{\beta}$ solve in
$\alpha$ and $\beta$ equations of the form

$\sum^{n}_{i=1}\mathcal{M}(Y_{i},A_{i},X_{i};\alpha,\beta)=0,$ (4)

where $\mathcal{M}(Y,A,X;\alpha,\beta)$ is a $p$-dimensional vector of
functions such that $E\\{\mathcal{M}(Y,A,X;\alpha_{0},\beta_{0})\\}=0$, and
$\alpha_{0}$ and $\beta_{0}$ are the true values of $\alpha$ and $\beta$ under
the assumption that the model is correctly specified. For example, under
models (1) and (3)

$\mbox{(a) }\mathcal{M}(Y,A,X;\alpha,\beta)=\left(\begin{array}[]{c}1\\\
A\end{array}\right)(Y-\alpha-\beta A)\hskip 10.84006pt\mbox{ and }\hskip
10.84006pt\mbox{(b)
}\mathcal{M}(Y,A,X;\alpha,\beta)=\left(\begin{array}[]{c}1\\\
A\end{array}\right)\\{Y-\exp(\alpha+\beta A)\\},$ (5)

respectively. Writing
$\overline{Y}_{a}=\sum^{n}_{i=1}Y_{i}I(A_{i}=a)/\sum^{n}_{i=1}I(A_{i}=a)$,
$a=0,1$, the treatment-specific sample means, the estimator obtained from
(5)(a) is $\widehat{\beta}=\overline{Y}_{1}-\overline{Y}_{0}$ and that from
(5)(b) is $\widehat{\beta}=\log(\overline{Y}_{1}/\overline{Y}_{0})$, which is
the estimator for the relative risk used in A5401. Under model (2), with
$\mbox{expit}(u)=e^{u}/(1+e^{u})$,

$\mathcal{M}(Y,A,X;\alpha,\beta)=\mathcal{D}(A)\left(\begin{array}[]{c}I(Y\leq
1)-\mbox{expit}(\alpha_{1}+\beta A)\\\ \vdots\\\ I(Y\leq
c-1)-\mbox{expit}(\alpha_{c-1}+\beta A)\end{array}\right),$ (6)

where $\mathcal{D}(A)$ is a $(c\times c-1)$ matrix of functions of $A$; the ML
estimator13 takes
$\mathcal{D}(A)=D^{T}(A;\alpha,\beta)V^{-1}(A;\alpha,\beta)$, where
$D(A;\alpha,\beta)$ is the $(c-1\times c)$ gradient matrix of the vector
$\\{I(Y\leq 1)-\mbox{expit}(\alpha_{1}+\beta A),\ldots,I(Y\leq
c-1)-\mbox{expit}(\alpha_{c-1}+\beta A)\\}^{T}$ in (6) with respect to
$\alpha_{1},\ldots,\alpha_{c-1},\beta$, and $V(A;\alpha,\beta)$ is its
$(c-1\times c-1)$ conditional covariance matrix given $A$.

In general, given a particular model and estimating equations defined by the
corresponding function $\mathcal{M}(Y,A,X;\alpha,\beta)$, let
$\mathcal{G}(\alpha,\beta)$ be the last row of the $(p\times p)$ matrix

$-\left[E\left\\{\frac{\partial\mathcal{M}(Y,A,X;\alpha,\beta)}{\partial\alpha^{T}\partial\beta}\right\\}\right]^{-1},$
(7)

where the matrix inside the expectation is the $(p\times p)$ matrix of partial
derivatives of the $p$ components of $\mathcal{M}(Y,A,X;\alpha,\beta)$ with
respect to $(\alpha^{T},\beta)^{T}$. Then, with
$\mathcal{G}(\alpha_{0},\beta_{0})$ denoting this expression evaluated at
$\alpha_{0},\beta_{0}$,

$m(Y,A,X;\alpha_{0},\beta_{0})=\mathcal{G}(\alpha_{0},\beta_{0})\mathcal{M}(Y,A,X;\alpha_{0},\beta_{0})$
(8)

is referred to as the influence function of the corresponding estimator for
$\beta$ and has mean zero. From the theory of M-estimation14 and
semiparametric theory7, it can be shown the estimator $\widehat{\beta}$
obtained by solving in $\beta$ the estimating equation

$\sum^{n}_{i=1}m(Y,A,X;\widehat{\alpha},\beta)=0,$ (9)

where $\widehat{\alpha}$ is any root-$n$ consistent estimator for $\alpha$,
has influence function (8). Tsiatis et al.10 show this explicitly in the case
of (6). Such estimators are consistent for the true value $\beta_{0}$ and
asymptotically normal, where the variance of the limiting normal distribution
of $n^{1/2}(\widehat{\beta}-\beta_{0})$ is equal to
$\mbox{var}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}=E\\{m(Y,A,X;\alpha_{0},\beta_{0})^{2}\\}$,
so that approximate (large sample) standard errors and test statistics are
readily derived.

From semiparametric theory7, there is a one-to-one correspondence between
influence functions and estimators. Thus, if the form of influence functions
in a specific model involving a parameter $\beta$ can be derived, estimating
equations leading to estimators for $\beta$ can be developed. As we
demonstrate in Section 3, influence functions corresponding to estimators for
$\beta$ based on the data available at an interim analysis can be derived from
the influence function (8), and the resulting estimators exploit baseline and
time-dependent covariate information to gain precision.

###  2.2 Data and assumptions

To characterize the data that would be available at an interim analysis, we
first describe more fully the data that would be available at the final
analysis if the trial were carried out to completion. Subjects enter the trial
in a staggered fashion; thus, if the trial starts at calendar time 0, denote
by $E$ the calendar time at which a subject enters the trial. As in Section 1,
let $T$ denote the time lag in ascertaining the outcome $Y$; thus, $T$ is the
time since entry at which $Y$ is determined, measured on the scale of subject
time. We assume that $Y$ can be determined with certainty by the maximum
follow-up period $\mathcal{T}_{F}$ for any subject, so that
$\mbox{pr}(T\leq\mathcal{T}_{F})=1$. In addition to baseline covariates $X$,
time-dependent covariate information may be collected on each participant up
to the time $Y$ is ascertained. Denote by $L(u)$ the vector of such
information at time $u$ following entry into the study, and let
$\bar{L}(u)=\\{L(s):0\leq s\leq u\\}$ be the history of the time-dependent
covariate information through time $u$. Thus, $\bar{L}(T)$ represents the
covariate history for a subject for whom $Y$ is ascertained after time lag
$T$.

With these definitions, for a trial with planned total sample size $n$, the
data available at the final analysis are iid

$\\{E_{i},X_{i},A_{i},T_{i},Y_{i},\bar{L}_{i}(T_{i})\\},\hskip
10.84006pti=1,\ldots,n;$ (10)

we refer to (10) as the full data. As in Section 2.1, estimation of $\beta$ at
the final analysis is based only on the data on $Y$, $A$, and possibly $X$ (in
the case of conditional inference), and $E$, $T$, and $\bar{L}(T)$ are not
used, and we call (8) a full data influence function.

Now consider the data that would be available at an interim analysis at
calendar time $t$ following the start of the trial at calendar time 0. It
proves convenient for the developments in Section 3 to represent these data in
terms of the full data (10) that would be available at the final analysis were
the trial to be carried out to completion. At $t$, data will be observed only
for subjects for whom $E\leq t$. For such subjects, define $C(t)=t-E$ to be
the censoring time, i.e., the time from a participant’s entry into the study
to the time of the interim analysis. If the time lag $T$ a subject would have
in ascertaining the outcome is such that $T\leq C(t)$, then $Y$ would be
available at $t$; otherwise, $Y$ would not yet be observed. Accordingly,
define $U(t)=\min\\{T,C(t)\\}$ and $\Delta(t)=I\\{T\leq C(t)\\}$, so that $Y$
is available at the time of the interim analysis only if $\Delta(t)=1$. With
these definitions, the data available at an interim analysis at calendar time
$t$ can be represented as iid

$\mathcal{O}^{(t)}_{i}=I(E_{i}\leq
t)\Big{[}\,E_{i},X_{i},A_{i},U_{i}(t),\Delta_{i}(t),\Delta_{i}(t)Y_{i},\bar{L}_{i}\\{U_{i}(t)\\}\,\Big{]},\hskip
10.84006pti=1,\ldots,n,$ (11)

where then $n(t)=\sum^{n}_{i=1}I(E_{i}\leq t)$ is the number of subjects of
the $n$ planned enrolled in the trial by calendar time $t$.

As noted in Section 1, an interim analysis that uses all of the available
data, including those from subjects for whom $T<\mathcal{T}_{F}$, can naively
overrepresent some values of the outcome over others. In terms of (11), the
data on which this naive analysis would be based involve only subjects $i$ who
are enrolled and whose outcome is available, i.e., for whom $I\\{E_{i}\leq
t),\Delta_{i}(t)=1\\}=1$. In contrast, a valid analysis that uses only the
data from subjects enrolled for at least the full, maximum follow-up period
$\mathcal{T}_{F}$ involves subjects $i$ for whom $I\\{E_{i}\leq
t,C_{i}(t)\geq\mathcal{T}_{F}\\}=1$. In the next section, we appeal to
semiparametric theory as noted at the end of Section 2.1 to deduce methods
yielding valid inference on $\beta$ based on the available data (11) that can
improve substantially on this analysis and thus lead to more efficient interim
analyses.

##  3 Inference based on interim data

###  3.1 Treatment effect estimation

We first present general estimating equations using the available data (11) at
an interim analysis at time $t$ that yield treatment effect estimators
offering gains in precision relative to the estimator based only on subjects
for whom $I\\{E_{i}\leq t,C_{i}(t)\geq\mathcal{T}_{F}\\}=1$. Letting
“$\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}$” denote statistical independence, assume
that $X\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}A$, which is guaranteed by randomization. Also
assume that

$E\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\\{X,A,T,Y,\bar{L}(T)\\};$ (12)

(12) implies that subjects enter according to a completely random process,
which is reasonable in many trials. Because $C(t)=t-E$, (12) also implies that
$C(t)\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\\{X,A,T,Y,\bar{L}(T)\\}$. We discuss
weakening these assumptions in Section 7. We also require that
$\mbox{pr}\\{C(t)>\mathcal{T}_{F}\\}>0$, so that there is positive probability
of seeing subjects for whom the final outcome has been ascertained at an
interim analysis at $t$ and so that the first interim analysis must occur at
least $\mathcal{T}_{F}$ time units after the start of the trial.

We first summarize the theoretical underpinnings of the practical, more
efficient interim monitoring approach we propose in Section 4. Under the above
assumptions, if $m(Y,A,X;\alpha_{0},\beta_{0})$ is the influence function of a
given estimator for a treatment effect parameter $\beta$ in a model for
$(Y,A,X)$ as in Section 2.1, so based on the full data (10), then
semiparametric theory yields that influence functions for estimators for
$\beta$ based on the available data $\mathcal{O}^{(t)}$ in (11) at an interim
analysis at time $t$ are of the form

$\displaystyle\frac{I(E\leq t)}{\mbox{pr}(E\leq t)}$
$\displaystyle\left(\frac{\Delta(t)m(Y,A,X;\alpha_{0},\beta_{0})}{\mathcal{K}_{t}\\{U(t)\\}}+\int^{t}_{0}\frac{dM^{(t)}_{c}(u)\mu(m,u;\alpha_{0},\beta_{0})}{\mathcal{K}_{t}(u)}\right.$
(13)
$\displaystyle-(A-\pi)f(X)+\left.\int^{t}_{0}dM^{(t)}_{c}(u)\left[h\\{u,X,A,\bar{L}(u)\\}-\mu(h,u)\right]\right),$

where $f(X)$ is an arbitrary function of $X$; $h\\{u,X,A,\bar{L}(u)\\}$ is an
arbitrary function of $u$, $X$, $A$, and $\bar{L}(u)$; and

$\mathcal{K}_{t}(u)=\mbox{pr}\\{C(t)\geq u|E\leq t\\},\hskip
10.84006pt\mu(m,u;\alpha_{0},\beta_{0})=E\\{m(Y,A,X;\alpha_{0},\beta_{0})|T\geq
u\\},\hskip 10.84006pt\mu(h,u)=E\big{[}h\\{u,X,A,\bar{L}(u)\\}|T\geq
u\big{]},$
$dM^{(t)}_{c}(u)=dN^{(t)}_{c}(u)-d\Lambda^{(t)}_{c}(u)\mathcal{Y}^{(t)}(u),\hskip
10.84006ptN^{(t)}_{c}(u)=I\\{U(t)\leq u,\Delta(t)=0\\},\hskip
10.84006pt\mathcal{Y}^{(t)}(u)=I\\{U(t)\geq u\\},\hskip
10.84006pt\Lambda^{(t)}_{c}(u)=-\log\\{\mathcal{K}_{t}(u)\\}.$

Here, $\mathcal{K}_{t}(u)$ is the survival distribution for the censoring
variable $C(t)$ at the time of the interim analysis, $N^{(t)}_{c}(u)$ and
$\mathcal{Y}^{(t)}(u)$ are the censoring counting process and at-risk process,
and $\Lambda^{(t)}_{c}(u)$ is the cumulative hazard function for censoring.

Let $\widehat{\mathcal{K}}_{t}(u)$ be the Kaplan-Meier estimator for
$\mathcal{K}_{t}(u)$ using the data $\\{U_{i}(t),1-\Delta_{i}(t)\\}$ for $i$
such that $E_{i}\leq t$, and define
$\widehat{\Lambda}^{(t)}_{c}(u)=-\log\\{\widehat{\mathcal{K}}_{t}(u)\\}$,
$d\widehat{M}^{(t)}_{ci}(u)=dN^{(t)}_{ci}(u)-d\widehat{\Lambda}^{(t)}_{c}(u)\mathcal{Y}^{(t)}_{i}(u)$,
and $\widehat{\pi}_{t}=\sum^{n}_{i=1}I(E_{i}\leq t)A_{i}/n(t)$, the proportion
of enrolled subjects at $t$ assigned to active treatment. Then it can be shown
that estimating equations corresponding to the influence functions in (13)
based on the available data (11) yielding estimators for $\beta$ are of the
form

$\sum^{n}_{i=1}I(E_{i}\leq
t)\left[\frac{\Delta_{i}(t)m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\beta\\}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t)\\}}-(A_{i}-\widehat{\pi}_{t})f(X_{i})+\int^{t}_{0}d\widehat{M}^{(t)}_{ci}(u)h\\{u,X_{i},A_{i},\bar{L}_{i}(u)\\}\right]=0,$
(14)

where $\widehat{\alpha}(t)$ is a consistent estimator for $\alpha$ based on
the available data at $t$. For a specific model, corresponding full data
influence function $m(Y,A,X;\alpha_{0},\beta_{0})$, and choice of the
functions $f(X)$ and $h\\{u,X,A,\bar{L}(u)\\}$, to be discussed momentarily,
an estimator for $\beta$ based on the data available at interim analysis time
$t$ is the solution to (14).

Taking $f(X)\equiv 0$ and $h\\{u,X,A,\bar{L}(u)\\}\equiv 0$ in (14) yields the
estimating equation

$\sum^{n}_{i=1}I(E_{i}\leq
t)\left[\frac{\Delta_{i}(t)m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\beta\\}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t)\\}}\right]=0,$
(15)

whose solution is a so-called inverse probability weighted complete case
(IPWCC) estimator, which effectively bases estimation of $\beta$ on only
subjects for whom $Y$ is available at $t$, but with inverse weighting by the
censoring distribution “adjusting” appropriately for the lag time in
ascertaining the outcome. Judicious nonzero choices of $f(X)$ and
$h\\{u,X,A,\bar{L}(u)\\}$ facilitate exploiting baseline and time-dependent
covariate information to gain efficiency over the IPWCC estimator solving (15)
through the two rightmost “augmentation” terms in the bracketed expression in
(14), leading to what is referred to as an augmented inverse probability
weighted complete case (AIPWCC) estimator for $\beta$; the optimal such
choices are discussed below.

A counterintuitive result from semiparametric theory is that, for any
arbitrary $f(X)$ and $h\\{u,X,A,\bar{L}(u)\\}$, it is possible to improve the
precision of the above estimators by replacing the Kaplan-Meier estimator
$\widehat{\mathcal{K}}_{t}(u)$ by treatment-specific Kaplan-Meier estimators
$\widehat{\mathcal{K}}_{t}(u,a)$, say, obtained using the data
$\\{U_{i}(t),1-\Delta_{i}(t)\\}$ for $i$ such that $E_{i}\leq t$ and
$A_{i}=a$, $a=0,1$, even though because of (12) the distribution of $C(t)$ is
not treatment dependent. This substitution leads to influence functions for
estimators for $\beta$ based on the available data of the form

$\displaystyle\frac{I(E\leq t)}{\mbox{pr}(E\leq t)}$
$\displaystyle\left(\frac{\Delta(t)m(Y,A,X;\alpha_{0},\beta_{0})}{\mathcal{K}_{t}\\{U(t),A\\}}+\int^{t}_{0}\frac{dM^{(t)}_{c}(u,A)\mu(m,u,A;\alpha_{0},\beta_{0})}{\mathcal{K}_{t}(u,A)}\right.$
(16)
$\displaystyle-(A-\pi)f(X)+\left.\int^{t}_{0}dM^{(t)}_{c}(u,A)\left[h\\{u,X,A,\bar{L}(u)\\}-\mu(h,u,A)\right]\right),$

where now

$\mathcal{K}_{t}(u,A)=\mbox{pr}\\{C(t)\geq u|E\leq t,A\\},\hskip
7.22743pt\mu(m,u,A;\alpha_{0},\beta_{0})=E\\{m(Y,A,X\alpha_{0},\beta_{0})|T\geq
u,A\\},\hskip 7.22743pt\mu(h,u,A)=E\big{[}h\\{u,X,A,\bar{L}(u)\\}|T\geq
u,A\big{]},$
$dM^{(t)}_{c}(u,A)=dN^{(t)}_{c}(u)-d\Lambda^{(t)}_{c}(u,A)\mathcal{Y}^{(t)}(u),\hskip
10.84006pt\Lambda^{(t)}_{c}(u,A)=-\log\\{\mathcal{K}_{t}(u,A)\\}.$

Estimating equations corresponding to (16) are then

$\sum^{n}_{i=1}I(E_{i}\leq
t)\left[\frac{\Delta_{i}(t)m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\beta\\}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}}-(A_{i}-\widehat{\pi}_{t})f(X_{i})+\int^{t}_{0}d\widehat{M}^{(t)}_{ci}(u,A_{i})h\\{u,X_{i},A_{i},\bar{L}_{i}(u)\\}\right]=0,$
(17)

where now
$\widehat{\Lambda}^{(t)}_{c}(u,a)=-\log\\{\widehat{\mathcal{K}}_{t}(u,a)\\}$;
and
$d\widehat{M}^{(t)}_{ci}(u,a)=dN^{(t)}_{ci}(u)-d\widehat{\Lambda}^{(t)}_{c}(u,a)\mathcal{Y}^{(t)}_{i}(u)$.
The estimating equations (17) with $f(X)\equiv 0$ and
$h\\{u,X,A,\bar{L}(u)\\}\equiv 0$,

$\sum^{n}_{i=1}I(E_{i}\leq
t)\left[\frac{\Delta_{i}(t)m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\beta\\}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}}\right]=0,$
(18)

yield an IPWCC estimator, and, again, nonzero choices of $f(X)$ and
$h\\{u,X,A,\bar{L}(u)\\}$ lead to an AIPWCC estimator.

When $Y$ is a binary outcome, as in study A5401, it can be shown that the
IPWCC estimator $\widehat{\beta}(t)$ solving (18) is algebraically identical
to the logarithm of the ratio of treatment-specific Kaplan-Meier estimators
for the probability of death or hospitalization at $\mathcal{T}_{F}$ days.
Thus, as noted in Section 1, the standard estimator for the risk ratio at an
interim analysis is a special case of the general formulation here. Moreover,
because this estimator is equivalent to an IPWCC estimator, it should be
possible to obtain more efficient inference on the risk ratio at an interim
analysis via an AIPWCC estimator.

Semiparametric theory provides the optimal choices of $f(X)$ and
$h\\{u,X,A,\bar{L}(u)\\}$ yielding the most precise AIPWCC estimator solving
either of (14) or (17), given by

$\displaystyle f^{opt}(X)$
$\displaystyle=E\\{m(Y,A,X;\alpha_{0},\beta_{0})|X,A=1)-E\\{m(Y,A,X;\alpha_{0},\beta_{0})|X,A=0),$
(19) $\displaystyle
h^{opt}\\{u,X,A,\bar{L}(u)\\}=\frac{E\\{m(Y,A,X;\alpha_{0},\beta_{0})|T\geq
u,X,A,\bar{L}(u)\\}}{\mathcal{K}_{t}(u)}.$

The conditional expectations in (19) are not likely to be known in practice.
We propose an approach to approximating $f^{opt}(X)$ and
$h^{opt}\\{u,X,A,\bar{L}(u)\\}$ in Section 4. We recommend estimating $\beta$
at an interim analysis at time $t$ by $\widehat{\beta}(t)$ solving an
estimating equation of the form (17) with the approximations for $f^{opt}(X)$
and $h^{opt}\\{u,X,A,\bar{L}(u)\\}$ substituted.

From semiparametric theory, estimators solving estimating equations of the
form (14) or (17) are consistent for $\beta_{0}$ (for $n(t)$ and $n$ large)
and asymptotically normal, where, as at the end of Section 2.1, the variance
of the large sample distribution of $\widehat{\beta}(t)$ can be obtained from
the variance of the corresponding influence function. Thus, the resulting
approximate standard errors $SE\\{\widehat{\beta}(t)\\}$ can be used to form a
Wald-type test statistic, $T(t)=\widehat{\beta}(t)/SE\\{\widehat{\beta}(t)\\}$
appropriate for addressing the null hypothesis of no treatment effect,
$\mbox{H}_{0}\\!\\!:\beta_{0}=0$.

We conclude this section by noting an important implication of these results.
In the case where the full data (10) are available, as would be the case at
the conclusion of the trial if not stopped early, the preceding developments
lead to covariate-adjusted estimators for $\beta$ based on the full data that
have the potential to yield increased efficiency over the usual full data
analyses outlined in Section 2.1. In particular, considering (17), if
$t_{end}$ is the calendar time at which the trial concludes with the full data
accrued and outcomes for all subjects ascertained, $I(E_{i}\leq t_{end})=1$,
$\Delta_{i}(t_{end})=1$, $\mathcal{K}_{t_{end}}\\{U_{i}(t_{end}),A_{i}\\}=1$,
and $d\widehat{M}^{(t_{end})}_{ci}(u,A_{i})=0$, $i=1,\ldots,n$, and (17)
becomes

$\sum^{n}_{i=1}\big{[}m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t_{end}),\beta\\}-(A_{i}-\widehat{\pi})f(X_{i})\big{]}=0,$
(20)

where $\widehat{\pi}=n^{-1}\sum^{n}_{i=1}A_{i}$, with corresponding influence
function

$m(Y,A,X;\alpha_{0},\beta_{0})-(A-\pi)f(X).$ (21)

As above, the optimal choice of $f(X)$ leading to the most precise estimator
solving (20) is that given in (19). The estimating equation (20) is of the
form of those in Zhang et al.15. Thus, the proposed approach leads naturally
to estimators for a final analysis that exploit baseline covariate information
to improve efficiency through the “augmentation term” $(A-\pi)f(X)$.

###  3.2 Interim analysis

In practice, interim analyses will be carried out at times
$t_{1}<\cdots<t_{K}$, with the possibility of stopping the trial early, e.g.,
for efficacy if evidence of a large treatment effect emerges at an interim
analysis. That is, focusing on efficacy, the trial may be stopped at the first
interim analysis time at which the relevant test statistic exceeds some
appropriate stopping boundary; that is, if

$|T(t_{j})|\geq b_{j},\hskip 10.84006ptj=1,\ldots,K,$

for a two-sided alternative or $T(t_{j})\geq$ or $\leq b_{j}$, $j=1,\ldots,K$,
for a one-sided alternative, where $b_{j}$, $j=1,\ldots,K$, are the stopping
boundaries. As is well-studied in the group sequential testing literature, the
stopping boundaries $b_{1},\ldots,b_{K}$ are chosen to take into account
multiple comparisons and ensure that the resulting procedure preserves the
desired overall type 1 error1, 2, 3, 9. Standard methods2, 3, 9 for deriving
stopping boundaries are based on the premise that the sequentially-computed
test statistics $T(t_{1}),\ldots,T(t_{K})$ have the so-called independent
increments structure8, 16.

In the Appendix, we sketch an argument demonstrating that, with the optimal
choices of $f(X)$ and $h\\{u,X,A,\bar{L}(u)\\}$ given in (19), the proposed
test statistics, properly normalized, have the independent increments
structure. Owing to this property, the practical strategies for interim
monitoring presented in Section 4 can be implemented using standard software
for computation of stopping boundaries; in the simulations in Section 5, we
use the R package ldbounds17.

##  4 Practical implementation and interim monitoring strategies

###  4.1 Treatment effect estimation

Generalizing the approach in Tsiatis et al.10 in the special case of a
proportional odds model (2), we propose estimation of $\beta$ at an interim
analysis at time $t$ using an AIPWCC estimator solving (17), which can be
obtained via a two-step algorithm.

Assume that the treatment effect $\beta$ of interest is defined within a model
for which, given full data at the end of the study, the estimator for $\beta$
would be obtained jointly with that for $\alpha$ by solving an estimating
equation of the form in (4) for a particular estimating function
$\mathcal{M}(Y,A,X;\alpha,\beta)$. Because the optimal choices $f^{opt}(X)$
and $h^{opt}\\{u,X,A,\bar{L}(u)\\}$ in (19) are not known, we approximate them
by linear combinations of basis functions. Letting
$f_{0}(X),f_{1}(X),\ldots,f_{M}(X)$ be functions of $X$ specified by the
analyst, with $f_{0}(X)\equiv 1$, approximate $f^{opt}(X)$ by

$\sum^{M}_{m=0}\psi_{m}f_{m}(X).$ (22)

Similarly, specify basis functions of $\\{u,X,\bar{L}(u)\\}$,
$h_{1}\\{u,X,\bar{L}(u)\\},\ldots,h_{L}\\{u,X,\bar{L}(u)\\}$, and approximate
$h^{opt}\\{u,X,a,\bar{L}(u)\\}$ by

$\sum^{L}_{\ell=1}\phi_{a,\ell}h_{\ell}\\{u,X,\bar{L}(u)\\},\hskip
10.84006pta=0,1.$ (23)

With suitably chosen basis functions, experience in other contexts6, 10, 15
suggests that this approach can lead to AIPWCC estimators that achieve
substantial efficiency gains over IPWCC estimators.

The AIPWCC estimator for $\beta$ obtained by substituting (22) and (23) in
(17) has influence function (16) with these same substitutions. Because from
semiparametric theory the variance of the estimator depends on the variance of
the influence function, as at the end of Section 2.1, we find the coefficients
$\psi_{m}$, $m=1,\ldots,M$, and $\phi_{a,\ell}$, $\ell=1,\ldots,L$, $a=0,1$,
that minimize this variance, which, from the form of (16), is a least squares
problem, as detailed below.

With these considerations, the two-step algorithm is as follows. At an interim
analysis at time $t$:

Step 1. Estimate $\alpha$ and $\beta$ by solving jointly in $\alpha$ and
$\beta$

$\sum^{n}_{i=1}I(E_{i}\leq
t)\left[\frac{\Delta_{i}(t)\mathcal{M}(Y_{i},A_{i},X_{i};\alpha,\beta)}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}}\right]=0$

to obtain $\widehat{\alpha}(t)$ and $\widehat{\beta}^{init}(t)$;
$\widehat{\beta}^{init}(t)$ is an IPWCC estimator solving (18). Then obtain an
estimator
$\widehat{\mathcal{G}}\\{\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}$ for
$\mathcal{G}(\alpha,\beta)$. If the expectation in (7) is analytically
tractable,
$\widehat{\mathcal{G}}\\{\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}$ is
the last row of (7) with $\widehat{\alpha}(t)$ and $\widehat{\beta}^{init}(t)$
substituted for $\alpha$ and $\beta$; if not, take the estimator
$\widehat{\mathcal{G}}\\{\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}$ to
be the last row of

$-\left[n(t)^{-1}\sum^{n}_{i=1}I(E_{i}\leq
t)\frac{\Delta_{i}(t)}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}}\left.\frac{\partial\mathcal{M}(Y_{i},A_{i},X_{i};\alpha,\beta)}{\partial\alpha^{T}\partial\beta}\right|_{\alpha=\widehat{\alpha}(t),\beta=\widehat{\beta}^{init}(t)}\right]^{-1}.$

For each subject $i$ for whom $E_{i}\leq t$, based on (8), construct

$m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}=\widehat{\mathcal{G}}\\{\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}\mathcal{M}\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}.$

Step 2. Estimate the coefficients $\psi_{m}$, $m=1,\ldots,M$, and
$\phi_{a,\ell}$, $\ell=1,\ldots,L$, $a=0,1$, in the approximations (22) and
(23) by “least squares,” as suggested above. Namely, for each subject $i$ for
whom $E_{i}\leq t$, define the “dependent variable”

$\widehat{\mathfrak{Y}}_{i}(t)=\frac{\Delta_{i}(t)m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}}+\int_{0}^{t}\frac{d\widehat{M}^{(t)}_{ci}(u,A_{i})\widehat{\mu}\\{m,u,A_{i};\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}}{\widehat{\mathcal{K}}_{t}(u,A_{i})},$

where, in the integrand in the second term of the above expression, for
$A_{i}=a$, $a=0,1$,

$\displaystyle\frac{\widehat{\mu}\\{m,u,a;\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}}{\widehat{\mathcal{K}}_{t}(u,a)}=$
$\displaystyle\left\\{\sum_{k=1}^{n}I(E_{k}\leq
t)\mathcal{Y}^{(t)}_{k}(u)I(A_{k}=a)\right\\}^{-1}$
$\displaystyle\times\sum^{n}_{k=1}\left\\{\frac{I(E_{k}\leq
t)\Delta_{k}(t)m\\{Y_{k},A_{k},X_{k};\widehat{\alpha}(t),\widehat{\beta}^{init}(t)\\}}{\widehat{\mathcal{K}}_{t}\\{U_{k}(t),a\\}}\mathcal{Y}^{(t)}_{k}(u)I(A_{k}=a)\right\\}.$

Likewise, for each of these subjects and suitably chosen basis functions as
discussed above, define the $M+1+2L$ “covariates”

$(A_{i}-\widehat{\pi}_{t})f_{m}(X_{i}),\hskip 10.84006ptm=0,1,\ldots,M;$
$I(A_{i}=0)\int^{t}_{0}d\widehat{M}^{(t)}_{ci}(u,0)\big{[}h_{\ell}\\{u,X_{i},\bar{L}_{i}(u)\\}-\widehat{\mu}(h_{\ell},u,0)\big{]},\hskip
10.84006pt\ell=1,\ldots,L,$
$I(A_{i}=1)\int^{t}_{0}d\widehat{M}^{(t)}_{ci}(u,1)\big{[}h_{\ell}\\{u,X_{i},\bar{L}_{i}(u)\\}-\widehat{\mu}(h_{\ell},u,1)\big{]},\hskip
10.84006pt\ell=1,\ldots,L,$

where, for $a=0,1$

$\widehat{\mu}(h_{\ell},u,a)=\left\\{\sum_{k=1}^{n}I(E_{k}\leq
t)\mathcal{Y}^{(t)}_{k}(u)I(A_{k}=a)\right\\}^{-1}\sum^{n}_{k=1}h_{\ell}\\{u,X_{k},\bar{L}_{k}(u)\\}\mathcal{Y}^{(t)}_{k}(u)I(A_{k}=a).$

Then obtain estimators $\widehat{\psi}_{m}$, $m=0,1,\ldots,M$, and
$\widehat{\phi}_{a,\ell}$, $\ell=1,\ldots,L$, $a=0,1$, by linear regression of
$\widehat{\mathfrak{Y}}_{i}(t)$ on the above covariates. Based on this
regression, obtain “predicted values” for each subject $i$ for whom $E_{i}\leq
t$ as

$\displaystyle
Pred_{i}=(A_{i}-\widehat{\pi}_{t})\sum^{M}_{m=0}\widehat{\psi}_{m}f_{m}(X_{i})+\int_{0}^{t}d\widehat{M}^{(t)}_{ci}(u,A_{i})\sum^{L}_{\ell=1}\widehat{\phi}_{A_{i},\ell}\big{[}h_{\ell}\\{u,X_{i},\bar{L}_{i}(u)\\}-\widehat{\mu}(h_{\ell},u,A_{i})\big{]}.$

The estimator for $\beta$ is then obtained as the one-step update

$\widehat{\beta}(t)=\widehat{\beta}^{init}(t)-n(t)^{-1}\sum^{n}_{i=1}I(E_{i}\leq
t)Pred_{i},$ (24)

and an approximate standard error for $\widehat{\beta}(t)$ is given by

$SE\\{\widehat{\beta}(t)\\}=n(t)^{-1}\left[\sum^{n}_{i=1}I(E_{i}\leq
t)\\{\widehat{\mathfrak{Y}}_{i}(t)-Pred_{i}\\}^{2}\right]^{1/2}.$ (25)

By an argument similar to that in the Supplementary Material of Tsiatis et
al.10, the estimator (24) is asymptotically equivalent to an AIPWCC estimator
solving (17).

In some settings, scant time-dependent covariate information may be available.
Here, a special case of the general AIPWCC formulation that still attempts to
gain efficiency from only baseline covariates $X$ is to solve an estimating
equation of the form in (17) but with $f(X)$ as in (19) and
$h\\{u,X,A,\bar{L}(u)\\}=0$. Implementation is as above, but with the
“dependent variable” in Step 2 regressed only on the $M+1$ “covariates”
$(A_{i}-\widehat{\pi}_{t})f_{m}(X_{i})$, $m=0,1,\ldots,M$, to obtain
estimators $\widehat{\psi}_{m}$, $m=0,1,\ldots,M$, and by redefining
$Pred_{i}=(A_{i}-\widehat{\pi}_{t})\sum_{m=0}^{M}\widehat{\psi}_{m}f_{m}(X_{i})$
in the one-step update (24) and its associated standard error (25). For
definiteness, we refer to the resulting estimator as “AIPW1” and that
incorporating time-dependent covariates above as “AIPW2.”

###  4.2 Interim analysis

There is a vast literature on early stopping of clinical trials using group
sequential and other methods; such methods are readily applied if the
independent increments property holds. We now discuss information-based and
fixed-sample size monitoring strategies for using these approaches with the
proposed treatment effect estimators, for which, as argued in the Appendix and
demonstrated empirically in Section 5, the independent increments property
holds exactly or approximately.

In the general information-based monitoring approach1, monitoring and group
sequential tests are based on the proportion of the total information to be
gained from the completed trial available at interim analysis times $t$, where
in the present context information is approximated at time $t$ using the
large-sample approximate standard error of the relevant estimator
$\widehat{\beta}(t)$, $SE\\{\widehat{\beta}(t)\\}$. If a group sequential test
is desired with type 1 error $\alpha$ for testing
$\mbox{H}_{0}\\!\\!:\beta_{0}=0$ and power $(1-\gamma)$ against a clinically
meaningful alternative value $\beta_{0}=\beta_{A}$, say, then the maximum
information $MI$ required to achieve this objective at the final analysis with
a two-sided test is

$MI=\left(\frac{z_{\alpha/2}+z_{\gamma}}{\beta_{A}}\right)^{2}IF,$

where $z_{\delta}$ is the $(1-\delta)$ quantile of the standard normal
distribution, and $IF$ is an inflation factor to account for the loss of power
that results due to repeated testing relative to doing a single final
analysis. For example, the inflation factor associated with using O’Brien-
Fleming stopping boundaries3 is modest, equal to about 1.03; see Tsiatis1.
Information at an interim analysis at time $t$ is approximated as

$\mbox{Inf}(t)=\big{[}SE\\{\widehat{\beta}(t)\\}\big{]}^{-2}.$

Thus, the proportion of information at interim analysis time $t$ is
approximated as

$p(t)=\frac{\mbox{Inf}(t)}{MI}.$ (26)

Given the proportion of information (26) together with, e.g., the Lan-DeMets
spending function9, standard software can be used to obtain stopping
boundaries such that the resulting group sequential testing procedure has the
desired operating characteristics.

Typically, in determining the overall sample size for a clinical trial to
achieve the desired power to detect a meaningful difference at a given level
of significance, the analyst must make assumptions on the values of nuisance
parameters. If these assumptions are not correct, the trial could be
underpowered. An oft-cited advantage of information-based monitoring is that
interim analyses would continue until the information $\mbox{Inf}(t)$ achieves
the maximum information $MI$, guaranteeing the desired operating
characteristics regardless of the values of nuisance parameters. However, if
the assumptions leading to the target sample size are not correct, the
information available at the time this planned sample size is reached and all
participants have the outcome ascertained may be less than $MI$. If evidence
emerges during the trial that $MI$ is unlikely to be achieved, the sample size
might be reestimated and increased so that the full information threshold $MI$
is met.

In many trials, however, resource constraints or other factors may make
exceeding the originally planned sample size impossible, thus rendering
principled information-based monitoring infeasible. If a fixed, maximum sample
size $n_{max}$, say, is planned and inalterable, then the proportion of
information available at an interim analysis at any time $t$ on which stopping
boundaries can be determined must instead be based on $n_{max}$. In terms of
the data (11) available at an interim analysis at $t$, as before,
$n(t)=\sum^{n}_{i=1}I(E_{i}\leq t)$ is the number of subjects who have
enrolled by time $t$; of these subjects, $n_{A}(t)=\sum^{n}_{i=1}I\\{E_{i}\leq
t,C_{i}(t)\geq\mathcal{T}_{F}\\}$ is the number who have been enrolled for the
maximum follow-up period and thus have the outcome ascertained with certainty,
and it is likely that $n_{A}(t)<n(t)$. In general, a typical interim analysis
at $t$ for fixed $n_{max}$ would be based only on the data from these
$n_{A}(t)$ subjects, and, accordingly, the proportion of information available
at $t$ would be $p(t)=n_{A}(t)/n_{max}$, with $p(t)=1$ at the final analysis.
However, here, the proposed IPWCC and AIPWCC estimators allow censoring due to
the time lag in ascertaining the outcome to be taken into account and
incorporation of covariates to increase efficiency, so make use of additional
information in (11) beyond that available on just the $n_{A}(t)$ subjects for
whom the outcome has been ascertained by time $t$. Thus, if monitoring is
based on test statistics constructed from these estimators, the proportion of
information available at $t$ should be between $n_{A}(t)/n_{max}$ and
$n(t)/n_{max}$.

With these considerations, for fixed-sample size monitoring, we propose
characterizing the proportion of information available at an interim analysis
at time $t$ in terms of what we refer to as the effective sample size
$n_{ESS}(t)$, say, at $t$. Intuitively, we define $n_{ESS}(t)$ to be the
number of participants, had they been enrolled for the maximum follow-up
period $\mathcal{T}_{F}$ and had their outcome ascertained with certainty,
that would be required to lead to an estimator for $\beta$ based only on data
from such subjects with the same precision as that achieved by an IPWCC or
AIPWCC estimator for $\beta$ based on all of the available data at $t$. The
proportion of information at $t$ would then be $n_{ESS}(t)/n_{max}$.

To define effective sample size formally, with $n^{*}$ subjects for whom the
outcome has been fully ascertained, indexed by $j=1,\ldots,n^{*}$, consider
the estimator $\widehat{\beta}^{*}$ obtained by solving in $\beta$ the full
data estimating equation (9) based on these $n^{*}$ subjects,

$\sum_{j=1}^{n^{*}}m(Y_{j},A_{j},X_{j};\widehat{\alpha},\beta)=0$

for some consistent estimator $\widehat{\alpha}$. Then from the semiparametric
theory, $\widehat{\beta}^{*}$ has standard error approximately equal to the
square root of $\mbox{var}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}/n^{*}.$ The
effective sample size at an interim analysis at time $t$ for the IPWCC
estimator $\widehat{\beta}(t)$ calculated using the available data (11) at $t$
is the value $n^{*}$ such that
$\mbox{var}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}/n^{*}=SE\\{\widehat{\beta}(t)\\}^{2}$.
Accordingly, define the effective sample size when monitoring is based on the
IPWCC estimator as

$n_{ESS}(t)=\frac{\mbox{var}\\{m(Y,A,X;\alpha_{0},\beta_{0})}{SE\\{\widehat{\beta}(t)\\}^{2}}.$
(27)

Because $\mbox{var}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}$ is not known, in
practice we must estimate it based on the available data, which can be
accomplished via the estimator

$\widehat{\mbox{var}}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}=n(t)^{-1}\sum^{n}_{i=1}I(E_{i}\leq
t)\frac{\Delta_{i}(t)m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\widehat{\beta}(t)\\}^{2}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}}.$
(28)

Thus, in practice, we obtain the approximate effective sample size as

$n_{ESS}(t)=\frac{\widehat{\mbox{var}}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}}{SE\\{\widehat{\beta}(t)\\}^{2}}.$
(29)

The effective sample size for an AIPWCC estimator $\widehat{\beta}(t)$ (either
AIPW1 or AIPW2) calculated using the available data at $t$ via the two-step
algorithm is defined similarly, but with the full data influence function
$m(Y,A,X;\alpha_{0},\beta_{0})$ in (27) replaced by the influence function
$m(Y,A,X;\alpha_{0},\beta_{0})-(A-\pi)f^{opt}(X)$ as in (21). An estimator for
$\mbox{var}\\{m(Y,A,X;\alpha_{0},\beta_{0})-(A-\pi)f^{opt}(X)\\}$ based on the
available data is given by

$\widehat{\mbox{var}}\\{m(Y,A,X;\alpha_{0},\beta_{0})\\}=n(t)^{-1}\sum^{n}_{i=1}I(E_{i}\leq
t)\frac{\Delta_{i}(t)[m\\{Y_{i},A_{i},X_{i};\widehat{\alpha}(t),\widehat{\beta}(t)\\}-Pred^{*}_{i}]^{2}}{\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}},\hskip
10.84006ptPred^{*}_{i}=(A_{i}-\widehat{\pi}_{t})\sum_{m=0}^{M}\widehat{\psi}_{m}f_{m}(X_{i}),$
(30)

where now the “predicted values” $Pred^{*}_{i}$ are obtained by a weighted
least squares regression with “dependent variable”
$m\\{Y,A,X;\widehat{\alpha}(t),\widehat{\beta}(t)\\}$, “covariates”
$(A_{i}-\widehat{\pi}_{t})f_{m}(X_{i})$, $m=0,1,\ldots,M$, and “weights”
$\Delta_{i}(t)/\widehat{\mathcal{K}}_{t}\\{U_{i}(t),A_{i}\\}$. Thus, for
$\widehat{\beta}(t)$ the AIPW1 or AIPW2 estimator, $n_{ESS}(t)$ is defined as
in (29) with the numerator given by (30).

With the appropriate definition of $n_{ESS}(t)$, we approximate the
corresponding proportion of information available at $t$ with interim analyses
based on an IPWCC or AIPWCCC estimator as

$p(t)=\frac{n_{ESS}(t)}{n_{max}}.$ (31)

From (28) and (30), $p(t)=1$ at the final analysis. As for information-based
monitoring, given the proportion of information (31), one can use standard
software with, e.g., the Lan-DeMets spending function9 to obtain stopping
boundaries.

In the simulation studies in the next section, we study the methods under
fixed-sample monitoring, as in our experience this approach is most common in
practice. Moreover, while performance of information-based monitoring with
statistics that possess the independent increments property has been well-
studied8, 18, because our approach to characterizing proportion of information
in fixed-sample monitoring based on the proposed effective sample size measure
is new, evaluation of its performance is required.

##  5 Simulation studies

We present results from several simulation studies, each involving 10000 Monte
Carlo replications. For each simulation scenario, we considered a uniform
enrollment process during the calendar time interval $[0,E_{max}]$ with a
maximum, fixed sample size $n_{max}$ and maximum follow up time
$\mathcal{T}_{F}$, and fixed-sample size monitoring with interim analyses
planned at calendar times $t_{1}<\cdots<t_{K}$ and a final analysis at time
$t_{end}=E_{max}+\mathcal{T}_{F}=t_{K+1}>t_{K}$, for a total of $K+1$ possible
analyses. For simplicity, in each scenario, we took $n_{max}$ to be the sample
size required to achieve roughly 80% or 90% power for a single analysis at
$t_{end}$ and did not include an inflation factor1. At each of the $K+1$
analysis times, we estimated the relevant treatment effect parameter $\beta$
four ways:

  1. i

using the estimator $\widehat{\beta}_{\mathcal{T}_{F}}(t_{j})$ obtained by
carrying out the full data analysis based only on subjects enrolled for at
least the maximum follow-up period $\mathcal{T}_{F}$, so using subjects $i$
for whom $I\\{E_{i}\leq t,C_{i}(t)\geq\mathcal{T}_{F}\\}=1$;

  2. ii

using the IPWCC estimator
$\widehat{\beta}_{IPW}(t_{j})=\widehat{\beta}^{init}(t_{j})$ based on the
available data (11), obtained at Step 1 of the two-step algorithm;

  3. iii

using the AIPWCC estimator $\widehat{\beta}_{AIPW1}(t_{j})$ based on the
available data (11), obtained at Step 2 of the two-step algorithm using only
baseline covariates $X$ to gain efficiency, as at the end of Section 4.1;

  4. iv

using the AIPWCC estimator $\widehat{\beta}_{AIPW2}(t_{j})$ based on the
available data (11) , obtained at Step 2 of the algorithm using both baseline
and time-dependent covariates $X$ and $\bar{L}(u)$.

At the final analysis at time $t_{end}$ at which the outcome has been
ascertained on all $n_{max}$ subjects,
$\widehat{\beta}_{\mathcal{T}_{F}}(t_{end})$ and
$\widehat{\beta}_{IPW}(t_{end})$ yield (versions of) the intended full data
analysis; and $\widehat{\beta}_{AIPW1}(t_{end})$ and
$\widehat{\beta}_{AIPW2}(t_{end})$ are identical and yield the covariate-
adjusted analysis exploiting baseline covariate information discussed at the
end of Section 3.1. In all scenarios, for the null hypothesis
$\mbox{H}_{0}\\!\\!:\beta_{0}=0$ and one-sided alternative hypotheses and
level of significance $\alpha=0.025$, we used the R package ldbounds17 with a
Lan-DeMets spending function to compute both O’Brien-Fleming3 and Pocock2
stopping boundaries at each analysis time $t_{j}$, $j=1,\ldots,K+1$. For
$\widehat{\beta}_{\mathcal{T}_{F}}(t_{j})$, this calculation was based on the
proportion of information $p(t_{j})=n_{A}(t_{j})/n_{max}$; for each of the
IPWCC and AIPWCC estimators $\widehat{\beta}_{IPW}(t_{j})$,
$\widehat{\beta}_{AIPW1}(t_{j})$, and $\widehat{\beta}_{AIPW2}(t_{j})$, the
stopping boundaries were obtained using the approximate proportion of
information (31) based on the relevant approximate effective sample size
$n_{ESS}(t_{j})$ given in (29).

For each scenario, we present the following results from two simulation
studies, one under $\mbox{H}_{0}$, so with data generated with $\beta_{0}=0$,
and one under an alternative of interest $\beta_{0}=\beta_{A}$:

  1. i

for each estimator, Monte Carlo estimates of
$\mbox{cov}\\{\widehat{\beta}(s),\widehat{\beta}(t)\\}$, $s<t$, and
$\mbox{var}\\{\widehat{\beta}(t)\\}$,
$s,t\in\\{t_{1},\ldots,t_{K},t_{end}\\}$; if the independent increments
property holds,
$\mbox{cov}\\{\widehat{\beta}(s),\widehat{\beta}(t)\\}=\mbox{var}\\{\widehat{\beta}(t)\\}$,
$s<t$;

  2. ii

for each estimator at each $t\in\\{t_{1},\ldots,t_{K},t_{end}\\}$, Monte Carlo
mean and standard deviation of $\widehat{\beta}(t)$, Monte Carlo mean of
$SE\\{\widehat{\beta}(t)\\}$, and Monte Carlo mean square error (MSE) for
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ divided by that for
$\widehat{\beta}(t)$;

  3. iii

for each estimator and stopping boundary, the Monte Carlo proportion of data
sets for which $\mbox{H}_{0}$ was rejected, Monte Carlo estimate of expected
sample size, and Monte Carlo estimate of expected stopping time.

The first two simulation scenarios, demonstrating the methods for an ordinal
categorical outcome and a binary outcome, respectively, are based on the
TESICO study with $\mathcal{T}_{F}=90$ days, using the generative models
adopted by Tsiatis et al.10, with $n_{max}=602$, $E_{max}=240$ days, and $K=4$
interim analyses planned at calendar times
$(t_{1},\ldots,t_{4})=(150,195,240,285)$ days, with the final analysis at
$t_{end}=330$ days. For each simulated subject, $A$ was generated as Bernoulli
with $\mbox{pr}(A=1)=\pi=0.5$, where $a=0\,(1)$ corresponds to placebo (active
agent). To produce data for which the proportional odds model (2) holds, we
generated $\Upsilon\sim U(0,1)$ and set
$\Gamma=(1-A)\Upsilon+A\Upsilon(1/\mbox{OR})/\\{1-\Upsilon+\Upsilon(1/e^{\beta})\\}$,
where as in (2) $\beta$ is the log odds ratio, so that the distribution of
$\Gamma$ given $A=1$ satisfies $\mbox{logit}\\{\mbox{pr}(\Gamma\leq
u\,|\,A=1)\\}=\mbox{logit}\\{\mbox{pr}(\Gamma\leq u\,|\,A=0)\\}+\beta$. For
$Y$ an ordinal outcome, we took
$\mbox{pr}(Y=j|A=0)=0.12,0.23,0.17,010,0.05,0.33$ for $j=1,\ldots,6$ as in
Table 1 of Tsiatis et al.10 and thus generated $Y$ according to in which
interval $\Gamma$ fell as determined by the cutpoints
$[0.00,0.12,0.35,0.52,0.62,0.67,1.00]$. Then if $\Gamma<0.52$, so $Y=1,2,$ or
3, we took the time in hospital to be $H=\mathcal{T}_{F}\Gamma/0.52$ and the
number of days at home and off oxygen as $\mathcal{T}_{F}-H$, and
$T=\mathcal{T}_{F}$. If $0.52\leq\Gamma<0.62$ or $0.62\leq\Gamma<0.67$,
corresponding to $Y=4$ or 5, again $T=\mathcal{T}_{F}$; if $\Gamma\geq 0.67$,
corresponding to death, time of death $T=(1-A)T_{0}+AT_{1}$, where $T_{0}\sim
U(0,30)$ if $A=0$ and $T_{1}\sim U(20,50)$ if $A=1$. A baseline covariate was
generated as $X\sim\mathcal{N}\\{1.5(\Upsilon-0.5),1\\}$, so that $X$ is
independent of $A$, correlated with $Y$, and does not affect the proportional
odds model. Two time dependent covariates were generated as
$L_{1}(u)=I(\mathcal{W}<u)$, where
$\mathcal{W}=HI(\Gamma<0.52)+\mathcal{T}_{F}I(\Gamma\geq 0.52)$, so that
$L_{1}(u)=1$ if the subject was still in the hospital at time $u$; and
$L_{2}(u)=(\mathcal{T}_{F}-\mathcal{W})L_{1}(u)$, the number of days the
subject was expected to be out of the hospital at day $\mathcal{T}_{F}$, and
$L(u)=\\{L_{1}(u),L_{2}(u)\\}$. For a scenario with binary outcome, we
generated the data according to the foregoing scheme, except that we defined
$Y=1$, corresponding to death, if $\Gamma\geq 0.67$, and $Y=0$ otherwise.

For the first scenario with ordinal categorical outcome, we generated data as
above under the null hypothesis, so with $\beta=\beta_{0}=0$, and
$\beta=\beta_{A}=\log(1.5)$, corresponding to the alternative for which TESICO
was powered (80% at the final analysis with $n=602$)10, and
$\mbox{H}_{A}\\!\\!:\beta_{0}>0$. Here, $\widehat{\beta}_{\mathcal{T}_{F}}(t)$
is the ML estimator for $\beta$ in (2) obtained using the R function polr in
the MASS package19. Following Tsiatis et al.10, to simplify implementation, we
constructed $\widehat{\beta}_{IPW}(t)$, $\widehat{\beta}_{AIPW1}(t)$, and
$\widehat{\beta}_{AIPW2}(t)$ using the estimating function (6) with
$\mathcal{D}(A)=D^{T}(A;\alpha,\beta)V^{-1}_{ind}(A;\alpha,\beta)$, where
$V_{ind}(A;\alpha,\beta)$ is chosen according to the “working independence”
assumption, so that with full data at the final analysis,
$\widehat{\beta}_{\mathcal{T}_{F}}(t_{end})$ and
$\widehat{\beta}_{IPW}(t_{end})$ are not identical. As shown by Tsiatis et
al.10 and borne out in the simulations below, the efficiency loss for
$\widehat{\beta}_{IPW}(t_{end})$ relative to
$\widehat{\beta}_{\mathcal{T}_{F}}(t_{end})$ is negligible.

Under (a) the null hypothesis and (b) the alternative, the Monte Carlo sample
covariance matrices of the 10000 estimates
$\\{\widehat{\beta}_{AIPW2}(t_{1}),\ldots,\widehat{\beta}_{AIPW2}(t_{4}),\widehat{\beta}_{AIPW2}(t_{end})\\}$
are

$\mbox{(a) }\left(\begin{array}[]{ccccc}0.041&0.027&0.022&0.019&0.019\\\
0.027&0.028&0.021&0.019&0.018\\\ 0.022&0.021&0.022&0.019&0.019\\\
0.019&0.019&0.019&0.019&0.018\\\
0.019&0.018&0.019&0.018&0.018\end{array}\right),\hskip 10.84006pt\mbox{(b)
}\left(\begin{array}[]{ccccc}0.042&0.027&0.022&0.019&0.019\\\
0.027&0.029&0.022&0.019&0.019\\\ 0.022&0.022&0.022&0.019&0.019\\\
0.019&0.019&0.019&0.019&0.019\\\
0.019&0.019&0.019&0.019&0.019\end{array}\right),$ (32)

clearly demonstrating that the independent increments property holds
approximately for this estimator. Analogous results for the other three
estimators are given in the Appendix, showing that the independent increments
property holds approximately for all.

Under the null hypothesis and the alternative, Table 1 presents Monte Carlo
mean and standard deviation, the Monte Carlo average of standard errors
$SE\\{\widehat{\beta}(t)\\}$, and MSE ratio defined above as the Monte Carlo
MSE for the indicated estimator divided by that for
$\widehat{\beta}_{\mathcal{T}_{F}}$. Thus, the MSE ratio reflects efficiency
of the indicated estimator relative to the usual ML estimator using data only
on subjects enrolled for the maximum follow-up period $\mathcal{T}_{F}$. From
the table, all estimators are consistent, with standard errors that track the
Monte Carlo standard deviations, under both hypotheses. The efficiency gains
over $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ achieved at interim analyses by
using any of $\widehat{\beta}_{IPW}(t)$, $\widehat{\beta}_{AIPW1}(t)$, and
$\widehat{\beta}_{AIPW2}(t)$ are substantial. The IPWCC estimator achieves
gains solely through accounting for censoring; the AIPWCC estimators improve
on these gains by additionally incorporating covariates. Notably,
$\widehat{\beta}_{AIPW2}(t)$ yields a two-fold gain at the initial interim
analysis. For all three estimators, the efficiency gains are most pronounced
at the early interim analyses where censoring is the most substantial and
diminish as censoring decreases as the trial progresses. At the final
analysis, $\widehat{\beta}_{\mathcal{T}_{F}}(t_{end})$ and
$\widehat{\beta}_{IPW}(t_{end})$ show very similar performance, with
$\widehat{\beta}_{IPW}(t_{end})$ exhibiting minimal relative loss of
efficiency, as noted above. As expected, $\widehat{\beta}_{AIPW1}(t_{end})$
and $\widehat{\beta}_{AIPW2}(t_{end})$ are identical and, due to the
incorporation of adjustment for baseline covariates, result a 16%-17% gain in
efficiency over the usual final analysis.

Table 1: For Scenario 1 with ordered categorical outcome, performance of estimators for $\beta$ under (a) the null hypothesis $\beta=0$ and (b) the alternative $\beta=\log(1.5)=0.405$ at each interim analysis time $(t_{1},\ldots,t_{4})=(150,195,240,285)$ days and at the final analysis at $t_{end}=330$ days MC Mean is the mean of 10000 Monte Carlo estimates; MC SD is the Monte Carlo standard deviation, Ave MC SE is the mean of Monte Carlo standard errors, and MSE ratio is the ratio of Monte Carlo mean square error for the AIPW2 estimator divided by that for the indicated estimator. | MC Mean | MC SD | Ave MC SE | MSE ratio |  | MC Mean | MC SD | Ave MC SE | MSE ratio
---|---|---|---|---|---|---|---|---|---
| (a) Null Hypothesis
| $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
$t_{1}$ | -0.002 | 0.294 | 0.294 | 1.000 |  | -0.004 | 0.232 | 0.232 | 1.603
$t_{2}$ | -0.002 | 0.221 | 0.221 | 1.000 |  | -0.002 | 0.189 | 0.189 | 1.330
$t_{3}$ | -0.003 | 0.184 | 0.185 | 1.000 |  | -0.002 | 0.166 | 0.164 | 1.239
$t_{4}$ | -0.001 | 0.162 | 0.162 | 1.000 |  | -0.001 | 0.152 | 0.151 | 1.139
$t_{end}$ | 0.000 | 0.146 | 0.146 | 1.000 |  | 0.000 | 0.147 | 0.146 | 0.991
| $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{1}$ | -0.004 | 0.221 | 0.221 | 1.775 |  | -0.005 | 0.203 | 0.198 | 2.095
$t_{2}$ | -0.002 | 0.178 | 0.178 | 1.534 |  | -0.002 | 0.168 | 0.165 | 1.717
$t_{3}$ | -0.002 | 0.156 | 0.154 | 1.399 |  | -0.002 | 0.149 | 0.145 | 1.542
$t_{4}$ | -0.001 | 0.141 | 0 .140 | 1.327 |  | -0.001 | 0.138 | 0.136 | 1.380
$t_{end}$ | 0.000 | 0.135 | 0.135 | 1.169 |  | 0.000 | 0.135 | 0.135 | 1.169
| (b) Alternative Hypothesis
| $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
$t_{1}$ | 0.408 | 0.294 | 0.294 | 1.000 |  | 0.406 | 0.235 | 0.235 | 1.566
$t_{2}$ | 0.406 | 0.220 | 0.221 | 1.000 |  | 0.406 | 0.191 | 0.191 | 1.336
$t_{3}$ | 0.404 | 0.185 | 0.185 | 1.000 |  | 0.405 | 0.167 | 0.165 | 1.221
$t_{4}$ | 0.406 | 0.163 | 0.162 | 1.000 |  | 0.406 | 0.153 | 0.152 | 1.131
$t_{end}$ | 0.406 | 0.147 | 0.146 | 1.000 |  | 0.406 | 0.148 | 0.147 | 0.985
| $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{1}$ | 0.405 | 0.224 | 0.224 | 1.733 |  | 0.406 | 0.204 | 0.200 | 2.078
$t_{2}$ | 0.406 | 0.180 | 0.180 | 1.508 |  | 0.408 | 0.169 | 0.167 | 1.702
$t_{3}$ | 0.405 | 0.158 | 0.155 | 1.378 |  | 0.408 | 0.150 | 0.147 | 1.523
$t_{4}$ | 0.406 | 0.142 | 0.141 | 1.314 |  | 0.407 | 0.139 | 0.137 | 1.373
$t_{end}$ | 0.406 | 0.137 | 0.136 | 1.159 |  | 0.406 | 0.137 | 0.136 | 1.159
Table 2: For Scenario 1 with ordered categorical outcome, interim analysis performance using each estimator with O’Brien-Fleming and Pocock stopping boundaries under (a) the null hypothesis $\beta=0$ and (b) the alternative $\beta=\log(1.5)=0.405$, with maximum sample size $n_{max}=602$ and $t_{end}=330$ days. P(reject) is the proportion of Monte Carlo data sets for which the null hypothesis was rejected; MC E(SS) is the Monte Carlo average of number of subjects enrolled at the time the stopping boundary was crossed (standard deviation); and MC E(Stop) is the Monte Carlo average stopping time (days) (standard deviation). The standard error for entries for P(reject) in (a) is $\approx 0.0016$.  | P(reject) | MC E(SS) | MC E(Stop) |  | P(reject) | MC E(SS) | MC E(Stop)
---|---|---|---|---|---|---|---
| (a) Null Hypothesis
| O’Brien-Fleming |  | Pocock
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ | 0.024 | 601.9 (2.7) | 329.3 (7.5) |  | 0.023 | 599.7 (21.4) | 327.4 (19.5)
$\widehat{\beta}_{IPW}(t)$ | 0.024 | 601.6 (7.8) | 328.4 (12.1) |  | 0.024 | 598.8 (25.9) | 326.7 (22.4)
$\widehat{\beta}_{AIPW1}(t)$ | 0.024 | 601.7 (6.7) | 328.5 (11.3) |  | 0.024 | 598.9 (25.5) | 326.8 (22.0)
$\widehat{\beta}_{AIPW2}(t)$ | 0.024 | 601.2 (11.4) | 327.9 (14.9) |  | 0.027 | 598.0 (28.9) | 326.1 (24.7)
| (b) Alternative Hypothesis
| O’Brien-Fleming |  | Pocock
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ | 0.784 | 592.7 (31.5) | 284.2 (44.7) |  | 0.710 | 548.1 (85.3) | 260.5 (68.7)
$\widehat{\beta}_{IPW}(t)$ | 0.771 | 564.6 (62.5) | 257.1 (55.9) |  | 0.701 | 516.3 (100.3) | 239.0 (74.7)
$\widehat{\beta}_{AIPW1}(t)$ | 0.836 | 562.7 (62.7) | 251.5 (53.4) |  | 0.774 | 508.3 (100.6) | 230.1 (72.2)
$\widehat{\beta}_{AIPW2}(t)$ | 0.841 | 531.9 (81.7) | 231.7 (58.0) |  | 0.783 | 483.4 (103.5) | 215.1 (72.3)

Table 2 presents interim monitoring results using each estimator with both
O’Brien-Fleming and Pocock stopping boundaries under the null hypothesis and
under the alternative $\beta=\log(1.5)$. Under the null, the nominal level
$\alpha=0.025$ is achieved for all estimators. Under the alternative, power
for $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ is slightly shy of the desired 80%,
as expected with no inflation factor; by comparison, the AIPWCC estimators
yield improved power due to inclusion of covariate information. Under the
alternative and both types of boundaries, basing interim analyses on
$\widehat{\beta}_{IPW}(t)$, $\widehat{\beta}_{AIPW1}(t)$, and
$\widehat{\beta}_{AIPW2}(t)$ results in impressive reductions in expected
sample size and expected stopping time relative to
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$, with the gains especially impressive
for $\widehat{\beta}_{AIPW2}(t)$.

For the second scenario with binary outcome, we generated data as above under
the null hypothesis and with the log odds ratio equal to 1.5, which implies a
log relative risk (risk ratio) for death ($Y=1$) of
$\beta=\beta_{0}=\beta_{A}=\log(0.247/0.33)=-0.290$ as in (3) and alternative
hypothesis $\mbox{H}_{A}\\!\\!:\beta_{0}<0$. We took $n_{max}=900$, which
corresponds roughly to 90% power to detect this alternative. The Monte Carlo
sample covariance matrices of the 10000 estimates
$\\{\widehat{\beta}(t_{1}),\ldots,\widehat{\beta}(t_{4}),\widehat{\beta}(t_{end})\\}$
for each of the four estimators under both the null and alternative settings
are shown in the Appendix and exhibit patterns analogous to those in (32),
demonstrating that all estimators have approximately the independent
increments property. Also shown in the Appendix for each estimator at each
analysis time are the Monte Carlo mean and standard deviation, the Monte Carlo
average of standard errors $SE\\{\widehat{\beta}(t)\\}$, and MSE ratio defined
above as the Monte Carlo MSE for the estimator divided by that for
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ under the null and alternative
hypotheses. All estimators are consistent, and standard errors are very close
to the Monte Carlo standard deviations. Under both null and alternative
hypotheses, the estimator $\widehat{\beta}_{IPW}(t)$, which takes censoring at
interim analysis times into account and as noted previously is identical to
the ratio of treatment-specific Kaplan-Meier estimators often used in
practice, achieves substantial efficiency gains over
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$, with a two-fold increase at the first
interim analysis and 24% at the last at $t_{4}=285$ days. These estimators are
equivalent, as expected, at the final analysis. The AIPWCC estimators
$\widehat{\beta}_{AIPW1}(t)$ and $\widehat{\beta}_{AIPW2}(t)$ achieve even
greater gains. Here, $\widehat{\beta}_{AIPW2}(t)$ does not offer improved
performance over $\widehat{\beta}_{AIPW1}(t)$; this behavior is not
surprising, as the time-dependent covariates $L(u)=\\{L_{1}(u),L_{2}(u)\\}$
reflecting length of hospital stay do not provide information on death. As
expected, these estimators are identical at $t_{end}$ and offer 10%-12% gains
in efficiency over the standard analysis through adjustment for the baseline
covariate.

Table 3 shows interim monitoring results using each estimator with O’Brien-
Fleming and Pocock stopping boundaries under the null hypothesis and under the
alternative $\beta_{A}=-0.290$. Again, overall testing procedures achieve the
nominal level. Power gains over $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ under
the alternative are achieved using the AIPWCC estimators. As for the first
scenario, basing interim analyses on $\widehat{\beta}_{IPW}(t)$,
$\widehat{\beta}_{AIPW1}(t)$, and $\widehat{\beta}_{AIPW2}(t)$ yields
substantial reductions in expected sample size and stopping time over
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ under the alternative, especially for
the AIPWCC estimators.

Table 3: For Scenario 2 with binary outcome, interim analysis performance using each estimator with O’Brien-Fleming and Pocock stopping boundaries under (a) the null hypothesis $\beta=0$ and (b) the alternative $\beta=\log(0.247/0.33)=-0.290$, with maximum sample size $n_{max}=900$ and $t_{end}=330$ days. Entries are as in Table 2. The standard error for entries for P(reject) in (a) is $\approx 0.0016$.  | P(reject) | MC E(SS) | MC E(Stop) |  | P(reject) | MC E(SS) | MC E(Stop)
---|---|---|---|---|---|---|---
| (a) Null Hypothesis
| O’Brien-Fleming |  | Pocock
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ | 0.024 | 900.0 (2.3) | 329.4 (6.5) |  | 0.022 | 896.3 (33.6) | 327.4 (19.8)
$\widehat{\beta}_{IPW}(t)$ | 0.024 | 898.8 (15.8) | 328.0 (14.5) |  | 0.023 | 894.3 (42.3) | 326.5 (23.7)
$\widehat{\beta}_{AIPW1}(t)$ | 0.023 | 899.0 (14.0) | 328.1 (13.8) |  | 0.025 | 894.1 (42.8) | 326.3 (24.3)
$\widehat{\beta}_{AIPW2}(t)$ | 0.024 | 899.0 (14.9) | 328.0 (14.2) |  | 0.026 | 894.1 (42.7) | 326.2 (24.4)
| (b) Alternative Hypothesis
| O’Brien-Fleming |  | Pocock
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ | 0.770 | 887.5 (44.3) | 285.9 (44.0) |  | 0.690 | 827.2 (122.4) | 264.4 (67.2)
$\widehat{\beta}_{IPW}(t)$ | 0.767 | 808.3 (121.4) | 241.3 (61.3) |  | 0.700 | 744.9 (157.6) | 228.3 (76.7)
$\widehat{\beta}_{AIPW1}(t)$ | 0.806 | 801.8 (122.5) | 236.1 (59.6) |  | 0.746 | 733.2 (157.3) | 221.4 (74.7)
$\widehat{\beta}_{AIPW2}(t)$ | 0.809 | 799.9 (123.4) | 235.5 (59.7) |  | 0.748 | 731.6 (157.3) | 220.6 (74.7)

The final simulation scenario involves a continuous outcome, with
$n_{max}=300$, $E_{max}=156$ weeks, and $\mathcal{T}_{F}=52$ weeks, so that
enrollment takes place over 3 years, with $K=4$ interim analyses planned at
calendar times 104, 130, 156, 182 weeks and the final analysis at
$t_{end}=208$ weeks. We generated treatment assignment $A$ as Bernoulli with
$\mbox{pr}(A=1)=\pi=0.5$, where $a=0\,(1)$ corresponds to placebo (active
agent); and a categorical baseline covariate $X_{1}$ was generated with
$\mbox{pr}(X_{1}=j)=0.4,0.3,0.2,0.1$ for $j=1,\ldots,4$. With
$(s_{1},\ldots,s_{5})=(0,4,12,24,52)$ weeks, $\sigma=4.5$, $D$ a $(2\times 2)$
matrix with $\mbox{vech}(D)=(80,-0.5,0.08)$, and $\xi=(\xi_{1},\xi_{2})^{T}$,
we generated longitudinal measurements for each subject $i$ according to the
linear mixed effects model
$Z_{ij}=65I(X_{1}=1)+60I(X_{1}=2)+55I(X_{1}=3)+49I(X_{1}=4)+\\{\xi_{1}(1-A)+\xi_{2}A\\}s_{j}+b_{0i}+b_{1i}s_{j}+e_{ij}$,
where $b_{i}=(b_{0i},b_{1i})^{T}\sim\mathcal{N}(0,D)$ independent of
$e_{ij}\sim\mathcal{N}(0,\sigma^{2})$. The outcome for subject $i$ is then
$Y_{i}=Z_{i5}$, the longitudinal measure at $\mathcal{T}_{F}$ weeks. As would
be likely in practice, we included in $X$ only the single baseline covariate
$Z_{i1}$, the value of the longitudinal measure at time 0 and did not also
include $X_{1}$, and we took the single time-dependent covariate $L(u)$ at
time $u$ to be the most recently observed value of the longitudinal
measurements $Z_{ij}$. Under the null hypothesis, $\xi=(-0.3,-0.3)^{T}$; under
the alternative, $\xi=(-0.3,-0.18)^{T}$ corresponding to
$\beta=\beta_{0}=\beta_{A}=6.24$, for which $n_{max}=300$ yields roughly 90%
power at the final analysis. Results shown in the Appendix demonstrate that
the independent increments property holds approximately for all estimators.
Here, the estimators $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ and
$\widehat{\beta}_{IPW}(t)$ are identical because $T=\mathcal{T}_{F}$ for all
subjects, so that both are based only on subjects followed for at least
$\mathcal{T}_{F}$ weeks. Standard errors
$SE\\{\widehat{\beta}_{\mathcal{T}_{F}}(t)\\}$ are obtained from the routine
formula for a difference in sample means assuming common treatment-specific
variance, while $SE\\{\widehat{\beta}_{IPW}(t)\\}$ follows from the IPWCC
influence function; these standard errors are asymptotically equivalent but
differ slightly for finite samples. Incorporation of the baseline covariate
yields 10%-20% gains in efficiency; further incorporation of the last outcome
carried forward as a time-dependent covariate leads to efficiency gains for
$\widehat{\beta}_{AIPW2}(t)$ of 34% to 47%.

Interim monitoring results are shown in Table 4 and are analogous to those in
Tables 2 and 3. Under the null hypothesis, the Monte Carlo rejection
probability for $\widehat{\beta}_{AIPW2}(t)$ with Pocock boundaries exceeds
slightly the nominal 0.025 level. Again, under the alternative, the AIPWCC
estimators result in earlier expected sample sizes and stopping times.

Table 4: For Scenario 3 with continuous outcome, interim analysis performance using each estimator with O’Brien-Fleming and Pocock stopping boundaries under (a) the null hypothesis $\beta=0$ and (b) the alternative $\beta=6.24$, with maximum sample size $n_{max}=300$ and $t_{end}=208$ days. Entries are as in Table 2. The standard error for entries for P(reject) in (a) is $\approx 0.0016$.  | P(reject) | MC E(SS) | MC E(Stop) |  | P(reject) | MC E(SS) | MC E(Stop)
---|---|---|---|---|---|---|---
| (a) Null Hypothesis
| O’Brien-Fleming |  | Pocock
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ | 0.026 | 299.9 (3.1) | 207.4 (5.5) |  | 0.025 | 298.6 (11.3) | 206.2 (12.7)
$\widehat{\beta}_{IPW}(t)$ | 0.026 | 299.8 (3.4) | 207.4 (5.8) |  | 0.025 | 298.6 (11.5) | 206.1 (12.9)
$\widehat{\beta}_{AIPW1}(t)$ | 0.025 | 299.9 (2.6) | 207.5 (5.0) |  | 0.026 | 298.6 (11.2) | 206.2 (12.7)
$\widehat{\beta}_{AIPW2}(t)$ | 0.026 | 299.8 (4.0) | 207.0 (7.4) |  | 0.029 | 298.1 (13.2) | 205.7 (14.5)
| (b) Alternative Hypothesis
| O’Brien-Fleming |  | Pocock
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ | 0.875 | 286.3 (26.0) | 167.8 (30.6) |  | 0.823 | 259.9 (45.2) | 151.9 (41.9)
$\widehat{\beta}_{IPW}(t)$ | 0.876 | 286.0 (26.4) | 167.4 (30.7) |  | 0.826 | 259.1 (45.4) | 151.1 (41.9)
$\widehat{\beta}_{AIPW1}(t)$ | 0.930 | 286.5 (25.5) | 165.5 (28.9) |  | 0.896 | 255.9 (45.2) | 146.0 (39.6)
$\widehat{\beta}_{AIPW2}(t)$ | 0.930 | 265.9 (37.3) | 146.7 (31.2) |  | 0.892 | 239.8 (45.3) | 133.5 (37.8)

We remark that all scenarios reflect the general result that basing interim
analyses on the proposed AIPWCC estimators leads to not only more efficient
inferences but also, because of the increased precision, to a greater
proportion of the total statistical information being available at each
interim analysis time than would be available using the usual methods. This
feature implies that O’Brien-Fleming boundaries will be less conservative for
the proposed estimators, leading to potential gains in expected sample size
and stopping times.

##  6 Application

To demonstrate how use of the methods would proceed in practice as a trial
progresses, we consider the setting of TESICO with ordinal categorical
outcome, where the treatment effect of interest is the log odds ratio $\beta$
in an assumed proportional odds model as in (2). Because this trial is
ongoing, we cannot base this demonstration on data from the trial;
accordingly, we present use of the methods for a simulated data set generated
according to the first simulation scenario in Section 5 with
$\beta=\log(1.5)$, which is based on this study. As in Section 5, the planned
maximum sample size is $n_{max}=602$, with full enrollment reached by
$E_{max}=240$ day. Interim analyses are planned at 150, 195, 240, and 285
days, with the final analysis to be conducted at $t_{end}=330$ days, at which
time all $n_{max}$ participants will have completed the trial with their
outcomes ascertained. For definiteness, we use O’Brien-Fleming stopping
boundaries and focus on the null and alternative hypotheses
$\mbox{H}_{0}\\!\\!:\beta_{0}=0$ versus $\mbox{H}_{A}\\!\\!:\beta_{0}>0$, with
overall level of significance $\alpha=0.025$.

Table 5 shows how the trial would proceed if the analyses were conducted at
each interim analysis time $t$ using each of the estimators
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$, $\widehat{\beta}_{IPW}(t)$,
$\widehat{\beta}_{AIPW1}(t)$, and $\widehat{\beta}_{AIPW2}(t)$. For each
estimator, the proportion of information at each of the interim analysis times
was calculated as described in Section 4.2 and was used to obtain the stopping
boundary. At the first interim analysis at 150 days, the proportion of
information for the ML estimator $\widehat{\beta}_{\mathcal{T}_{F}}(t)$, which
uses only those subjects among the $n(t)$ enrolled who have been followed for
at least the maximum follow-up time $\mathcal{T}_{F}$, is 0.257, whereas that
for $\widehat{\beta}_{AIPW2}(t)$ is 0.462, almost twice as large. This
striking difference is reflected in the corresponding stopping boundaries: at
the first interim analysis at 150 days, the test statistic based on
$\widehat{\beta}_{\mathcal{T}_{F}}(t)$ is 2.496, far from the boundary of
4.265, whereas that based on $\widehat{\beta}_{AIPW2}(t)$ is 2.966, almost
reaching the boundary of 3.099. Basing the analyses on
$\widehat{\beta}_{AIPW2}(t)$ results in sufficient evidence to stop the trial
at the second interim analysis at 195 days with 487 subjects enrolled, while
sufficient evidence to stop using $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ does
not emerge until the fourth interim analysis at 285 days, with all
$n_{max}=602$ subjects enrolled. Basing the analyses on
$\widehat{\beta}_{IPW}(t)$ and $\widehat{\beta}_{AIPW1}(t)$ results in
stopping the trial at 240 days, with again all 602 planned subjects enrolled.

Table 5: Interim analysis results for analyses at time $(t_{1},\ldots,t_{5})=(150,195,240,285,330)$ for the simulated TESICO trial; $n(t_{j})$ is the number of subjects enrolled at $t_{j}$. For each of the estimators $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ (the ML estimator based on data from all subjects followed for at least the maximum follow-up period $\mathcal{T}_{F}$ at $t$), the IPWCC estimator $\widehat{\beta}_{IPW}(t)$, and the AIPWCC estimators $\widehat{\beta}_{AIPW1}(t)$ and $\widehat{\beta}_{AIPW2}(t)$, Est (SE) are the estimate $\widehat{\beta}(t_{j})$ (standard error $SE\\{\widehat{\beta}(t_{j})\\}$) at $t_{j}$, $T$ is associated the Wald test statistic, $p(t_{j})$ is the proportion of information at $t_{j}$, and $b_{j}$ is the O’Brien-Fleming stopping boundary. Entries are boldfaced at the interim analysis at which the trial would be stopped using the indicated estimator.  |  | $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
---|---|---|---|---
$t_{j}$ | $n(t_{j})$ | Est (SE) | $T$ | $p(t_{j})$ | $b_{j}$ |  | Est (SE) | $T$ | $p(t)$ | $b_{j}$
150 | 368 | 0.730 (0.292) | 2.496 | 0.257 | 4.265 |  | 0.547 (0.230) | 2.380 | 0.408 | 3.318
195 | 487 | 0.619 (0.224) | 2.765 | 0.432 | 3.218 |  | 0.476 (0.193) | 2.473 | 0.581 | 2.733
240 | 602 | 0.457 (0.187) | 2.445 | 0.611 | 2.657 |  | 0.423 (0.166) | 2.551 | 0.785 | 2.313
285 | 602 | 0.459 (0.162) | 2.828 | 0.809 | 2.277 |  | – | – | – | –
330 | 602 | – | – | – | – |  | – | – | – | –
|  | $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{j}$ | $n(t_{j})$ | Est (SE) | $T$ | $p(t)$ | $b_{j}$ |  | Est (SE) | $T$ | $p(t)$ | $b_{j}$
150 | 368 | 0.565 (0.218) | 2.586 | 0.382 | 3.444 |  | 0.590 (0.199) | 2.966 | 0.462 | 3.099
195 | 487 | 0.497 (0.182) | 2.739 | 0.564 | 2.777 |  | 0.532 (0.167) | 3.185 | 0.670 | 2.521
240 | 602 | 0.409 (0.156) | 2.615 | 0.757 | 2.362 |  | – | – | – | –
285 | 602 | – | – | – | – |  | – | – | – | –
330 | 602 | – | – | – | – |  | – | – | – | –

##  7 Discussion

We have proposed a general framework for design and conduct of group
sequential trials in the common situation where the outcome is known with
certainty only after some time lag. The methods account for censoring at the
time of an interim analysis and incorporate baseline and time-dependent
evolving covariate information to improve efficiency over standard analyses,
facilitating earlier stopping with potentially smaller numbers of enrolled
subjects. We have demonstrated analytically and empirically that the proposed
test statistics possess the independent increments structure, so that standard
methods and software for specifying stopping boundaries can be used. The
methods can be applied under both information-based monitoring and fixed-
sample monitoring strategies. For the latter, we have proposed the idea of
effective sample size to characterize the proportion of information available
at an interim analysis. Simulation studies demonstrate that the methods
preserve the operating characteristics of a monitored trial and that
substantial reductions in expected sample size and stopping time can be
achieved.

As noted above, the proposed methodology is relevant in the large class of
problems where the outcome would be known with certainty for all subjects at
the final analysis. For some trials with possibly censored time-to-event
outcome, interest may focus on the hazard ratio under the assumption of
proportional hazards. Here, there is no prespecified, maximum follow-up time
$\mathcal{T}_{F}$ at which the outcome is known with certainty, so that the
proposed framework is not applicable.

The methods as presented are based on the assumption (12) that entry time is
independent of all other variables, including baseline covariates $X$, which
implies that any interim analysis time $t$ $C(t)\mathchoice{\mathrel{\hbox
to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\\{X,A,T,Y,\bar{L}(T)\\}$. This assumption is
made tacitly in any clinical trial that focuses on inference on an
unconditional treatment effect parameter. If the distribution of $X$ changes
over the course of a trial, then (12) is violated, and, intuitively, subjects
enrolled at the time of an interim analysis may not be representative of the
population of interest at the final analysis. This is a general phenomenon and
not unique to our methodology. If (12) is violated in this way, then the
treatment effect parameter may not be static over time. Under these
circumstances, conditional (on $X$) inference may be more appropriate; e.g.,
as in the case of the conditional proportional odds model for ordinal
categorical outcome in Section 2.1. Under the modified assumption
$E\mathchoice{\mathrel{\hbox to0.0pt{$\displaystyle\perp$\hss}\mkern
2.0mu{\displaystyle\perp}}}{\mathrel{\hbox
to0.0pt{$\textstyle\perp$\hss}\mkern 2.0mu{\textstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptstyle\perp$\hss}\mkern
2.0mu{\scriptstyle\perp}}}{\mathrel{\hbox
to0.0pt{$\scriptscriptstyle\perp$\hss}\mkern
2.0mu{\scriptscriptstyle\perp}}}\\{A,T,Y,\bar{L}(T)\\}|X$ (independence
conditional on $X$), the proposed methods can be extended to support such
conditional inference through incorporation of relevant influence functions
and modeling of the censoring distribution as a function of $X$.

## Acknowledgments

The authors thank Drs. Birgit Grund and Michael Hughes for helpful
discussions.

## Supporting information

R code implementing the simulations reported in Section 5 is available from
the authors

##  Appendix A Demonstration of Independent Increments Property

For definiteness, we consider AIPWCC estimators $\widehat{\beta}(t)$ with
influence function (13) with $f^{opt}(X)$ and $h^{opt}\\{u,X,A,\bar{L}(u)\\}$
in (19) substituted for $f(X)$ and $h\\{u,X,A,\bar{L}(u)\\}$, i.e., the
efficient influence function; results for the IPWCC estimator follow from the
argument.

We wish to describe the joint distribution of such estimators at interim
analysis times $t_{1}<\cdots<t_{K}$ (calendar time). It suffices to consider
the bivariate distribution of $\\{\widehat{\beta}(s),\widehat{\beta}(t)\\}$,
$s<t$. The large-sample properties of
$\\{\widehat{\beta}(s),\widehat{\beta}(t)\\}$ are determined by the covariance
matrix of the corresponding influence functions for $\widehat{\beta}(s)$ and
$\widehat{\beta}(t)$.

Using

$\frac{I(E\leq t)}{\mbox{pr}(E\leq
t)}\frac{\Delta(t)m(Y,A,X;\alpha_{0},\beta_{0})}{\mathcal{K}_{t}\\{U(t)\\}}=\frac{I(E\leq
t)}{\mbox{pr}(E\leq
t)}\left[m(Y,A,X;\alpha_{0},\beta_{0})-\int^{t}_{0}\frac{dM^{(t)}_{c}(u)}{\mathcal{K}_{t}(u)}\\{m(Y,A,X;\alpha_{0},\beta_{0})-\mu(m,u;\alpha_{0},\beta_{0})\\}\right],$
(A1) $\displaystyle\mu(h^{opt},u)$
$\displaystyle=E\big{[}h^{opt}\\{u,X,A,\bar{L}(u)\\}|T\geq
u\big{]}=\frac{E\left[E\\{m(Y,A,X;\alpha_{0},\beta_{0})|T\geq
u,X,A,\bar{L}(u)\\}|T\geq u\right]}{\mathcal{K}_{t}(u)}$
$\displaystyle=\frac{E\\{m(Y,A,X;\alpha_{0},\beta_{0})|T\geq
u\\}}{\mathcal{K}_{t}(u)}=\frac{\mu(m,u;\alpha_{0},\beta_{0})}{\mathcal{K}_{t}(u)},$

and denoting the available data at interim analysis times $s$ and $t$ by
$\mathcal{O}^{(s)}$ and $\mathcal{O}^{(t)}$ as in (11), the efficient
influence function for $\widehat{\beta}(t)$ is given by

$\displaystyle m_{t}(\mathcal{O}^{(t)};\alpha_{0},$
$\displaystyle\beta_{0})=\frac{I(E\leq t)}{\mbox{pr}(E\leq
t)}\left(\vphantom{\int^{t}_{0}\frac{dM^{(t)}_{c}(u)}{\mathcal{K}_{t}(u)}}\\{m(Y,A,X;\alpha_{0},\beta_{0})-(A-\pi)f^{opt}(X)\\}\right.$
(A2)
$\displaystyle-\left.\int^{t}_{0}\frac{dM^{(t)}_{c}(u)}{\mathcal{K}_{t}(u)}\big{[}m(Y,A,X;\alpha_{0},\beta_{0})-E\\{m(Y,A,X;\alpha_{0},\beta_{0})|T\geq
u,X,A,\bar{L}(u)\\}\big{]}\right).$ (A3)

Similarly, denote the efficient influence function for $\widehat{\beta}(s)$ as
$m_{s}(\mathcal{O}^{(s)};\alpha_{0},\beta_{0})$. For brevity, we suppress
dependence of these influence functions on $\alpha_{0},\beta_{0}$ henceforth.

To demonstrate the independent increments property, it suffices to show that

$E\\{m_{s}(\mathcal{O}^{(s)})m_{t}(\mathcal{O}^{(t)})\\}=\mbox{var}\\{m_{t}(\mathcal{O}^{(t)})\\}=E\\{m_{t}(\mathcal{O}^{(t)})^{2}\\}.$
(A4)

To this end, recalling that $C(t)=t-E$, it is convenient to define

$\widetilde{N}^{(t)}_{c}(u)=I\\{C(t)\leq u\\}=I(E\geq t-u),\hskip
10.84006pt\widetilde{\mathcal{Y}}^{(t)}(u)=I\\{C(t)\geq u\\}=I(E\leq t-u),$

and note that $\mathcal{K}_{t}(u)=\mbox{pr}\\{C(t)\geq u|E\leq
t\\}=\mbox{pr}(E\leq t-u|E\leq t)$. Denoting the distribution of $E$ by
$Q(u)=\mbox{pr}(E\leq u)$, with density $q(u)$, and recalling that
$\Lambda^{(t)}_{c}(u)=-\log\\{\mathcal{K}_{t}(u)\\}$, it follows that

$\mathcal{K}_{t}(u)=Q(t-u)/Q(t),\hskip
10.84006ptd\Lambda^{(t)}_{c}(u)=\\{q(t-u)/Q(t-u)\\}\,du.$

Thus, if we write

$d\widetilde{M}(t-u)=d\widetilde{N}^{(t)}_{c}(u)-d\Lambda^{(t)}_{c}(u)\widetilde{\mathcal{Y}}^{(t)}(u)=-dI(E\geq
t-u)-\frac{q(t-u)}{Q(t-u)}I(E\leq t-u)\,du,$ (A5)

then using $N^{(t)}_{c}(u)=I\\{C(t)\leq u,T\geq u\\}$ and
$\mathcal{Y}^{(t)}(u)=I\\{C(t)\geq u,T\geq u\\}$, the martingale integral (A3)
can be written as

$\displaystyle-\int^{t}_{0}\frac{d\widetilde{M}(t-u)I(T\geq
u)}{Q(t-u)/Q(t)}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq u,X,A,\bar{L}(u)\\}\big{]}$
$\displaystyle=-\int^{t}_{0}\frac{d\widetilde{M}(x)I(T\geq
t-x)}{Q(x)/Q(t)}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}$ (A6)

with change of variable $x=t-u$. If we define the filtration $\mathcal{F}(x)$
to be the sigma algebra generated by $\\{I(E\geq x),EI(E\geq
x),Y,A,X,\bar{L}(T)\\}$, then the integral (A6) is the realization of a
$\mathcal{F}(x)$-measurable martingale process. Note that the filtration
$\mathcal{F}(x)$ defines information about $E$ to the right (after) $x$ rather
than as for the usual filtration that defines information to the left (before)
$x$. Thus, (A3) can be written as

$\displaystyle-\frac{I(E\leq t)}{Q(t)}$
$\displaystyle\int^{t}_{0}\frac{d\widetilde{M}(x)I(T\geq
t-x)}{Q(x)/Q(t)}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}$
$\displaystyle=-\int^{t}_{0}\frac{d\widetilde{M}(x)I(T\geq
t-x)}{Q(x)}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]},$

and the variance of (A3) is

$\displaystyle E$
$\displaystyle\left(\int^{t}_{0}\frac{q(x)\,dx/Q(x)}{Q^{2}(x)}I(E\leq
x)I(T\geq t-x)\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}\right)$
$\displaystyle=\int^{t}_{0}\frac{q(x)\,dx}{Q^{2}(x)}E\left(\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right).$

Consequently,

$\displaystyle\mbox{var}\\{m_{t}(\mathcal{O}^{(t)})\\}$
$\displaystyle=\frac{\mbox{var}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}}{Q(t)}$ (A7)
$\displaystyle+\int^{t}_{0}\frac{q(x)\,dx}{Q^{2}(x)}E\left(\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right),$

and similarly for $\mbox{var}\\{m_{s}(\mathcal{O}^{(s)})\\}$.

From (A4), we thus wish to show that
$E\\{m_{s}(\mathcal{O}^{(s)})m_{t}(\mathcal{O}^{(t)})\\}$ is equal to
$\mbox{var}\\{m_{t}(\mathcal{O}^{(t)})\\}$ in (A7). Using the preceding
developments, we can write

$\displaystyle m_{t}(\mathcal{O}^{(t)})$ $\displaystyle=\frac{I(E\leq
t)}{Q(t)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}$ (A8)
$\displaystyle-\int^{t}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)$ (A9) $\displaystyle
m_{s}(\mathcal{O}^{(s)})$ $\displaystyle=\frac{I(E\leq
s)}{Q(s)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}$ (A10)
$\displaystyle-\int^{s}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
s-x,X,A,\bar{L}(s-x)\\}\big{]}I(T\geq s-x);$ (A11)

accordingly,

$E\\{m_{s}(\mathcal{O}^{(s)})m_{t}(\mathcal{O}^{(t)})\\}=(\ref{eq:termone})\times(\ref{eq:termthree})+(\ref{eq:termone})\times(\ref{eq:termfour})+(\ref{eq:termtwo})\times(\ref{eq:termthree})+(\ref{eq:termtwo})\times(\ref{eq:termfour}).$

We consider each of these terms in turn.

Using $s\leq t$, it is straightforward that

$\displaystyle E\\{(\ref{eq:termone})\times(\ref{eq:termthree})\\}$
$\displaystyle=E\left[\frac{I(E\leq
s)}{Q(s)Q(t)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}^{2}\right]$
$\displaystyle=\frac{\mbox{var}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}}{Q(t)}.$ (A12)

Similarly,

$\displaystyle E\\{(\ref{eq:termone})\times(\ref{eq:termfour})\\}$
$\displaystyle=E\left(\vphantom{\int^{s}_{0}\frac{d\widetilde{M}(x)}{Q(x)}}-\frac{I(E\leq
s)}{Q(t)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}\right.$ $\displaystyle\hskip
14.45377pt\times\left.\int^{s}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
s-x,X,A,\bar{L}(s-x)\\}\big{]}I(T\geq s-x)\right)$
$\displaystyle=E\left(\vphantom{\int^{s}_{0}\frac{d\widetilde{M}(x)}{Q(x)}}\frac{\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}}{Q(t)}\right.$
$\displaystyle\hskip
14.45377pt\times\left.\int^{s}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
s-x,X,A,\bar{L}(s-x)\\}\big{]}I(T\geq s-x)\right)$ $\displaystyle=0$ (A13)

because $\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}$ is $\mathcal{F}(s)$-predictable, so
that the expectation is zero by the martingale property of stochastic
integrals.

By the martingale property,

$\displaystyle E\\{(\ref{eq:termtwo})\times(\ref{eq:termfour})\\}$
$\displaystyle=\int^{s}_{0}\frac{q(x)dx}{Q^{2}(x)}E\Big{(}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}$ (A14) $\displaystyle\hskip
14.45377pt\times\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
s-x,X,A,\bar{L}(s-x)\\}\big{]}I(T\geq t-x)\Big{)}.$

The expectation in the intetgral in (A14) can be written as

$\displaystyle E\left(\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right)$
$\displaystyle-E\Big{(}\big{[}E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}-E\\{m(Y,A,X)|T\geq s-x,X,A,\bar{L}(s-x)\\}\big{]}$
(A15) $\displaystyle\times\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\Big{)}.$

The difference of conditional expectations in brackets in (A15) is some
function $g\\{X,A,\bar{L}(t-x)\\}$, in which case the last two lines of (A15)
can be written as

$\displaystyle E$
$\displaystyle\Big{(}g\\{X,A,\bar{L}(t-x)\\}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\Big{)}$
$\displaystyle=E\left\\{E\left(g\\{X,A,\bar{L}(t-x)\\}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right)\mid T\geq
t-x,X,A,\bar{L}(t-x)\right\\}$
$\displaystyle=E\left(g\\{X,A,\bar{L}(t-x)\\}\big{[}E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right)$ $\displaystyle=0.$

It follows that

$E\\{(\ref{eq:termtwo})\times(\ref{eq:termfour})\\}=\int^{s}_{0}\frac{q(x)dx}{Q^{2}(x)}E\left(\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right).$ (A16)

Finally, consider

$\displaystyle E\\{(\ref{eq:termtwo})\times(\ref{eq:termthree})\\}$
$\displaystyle=-E\left(\vphantom{\int^{t}_{0}\frac{d\widetilde{M}(x)}{Q(x)}}\frac{I(E\leq
s)}{Q(s)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}\right.$ (A17)
$\displaystyle\times\left.\int^{t}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right).$

We can write (A17) as

$\displaystyle-E$
$\displaystyle\left(\vphantom{\int^{t}_{0}\frac{d\widetilde{M}(x)}{Q(x)}}\frac{I(E\leq
s)}{Q(s)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}\right.$
$\displaystyle\times\left.\int^{s}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right)$ (A18) $\displaystyle-E$
$\displaystyle\left(\vphantom{\int^{t}_{0}\frac{d\widetilde{M}(x)}{Q(x)}}\frac{I(E\leq
s)}{Q(s)}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}\right.$
$\displaystyle\times\left.\int^{t}_{s}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right).$ (A19)

Because $\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}$ is $\mathcal{F}(s)$-predictable,
(A18) is equal to zero. Thus, consider (A19). Recalling from (A5) that

$d\widetilde{M}(x)=-dI(E\geq x)-\frac{q(x)}{Q(x)}I(E\leq x)dx$

and noting that for $x\geq s$

$I(E\leq s)\\{-dI(E\geq x)\\}=0,\hskip 10.84006ptI(E\leq s)I(E\leq x)=I(E\leq
s),$

it follows that (A19) can be written as

$\displaystyle E$
$\displaystyle\left(\int^{t}_{s}\frac{\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}}{Q(s)}\frac{q(x)dx}{Q^{2}(x)}I(E\leq
s)\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq
t-x)\right)$
$\displaystyle=\int^{t}_{s}\frac{q(x)dx}{Q^{2}(x)}E\left(\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right).$ (A20)

Write the expectation in the integrand of (A20) as

$\displaystyle E$ $\displaystyle\left(\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right)$
$\displaystyle+E\left(\big{[}E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}-(A-\pi)f^{opt}(X)\big{]}\right.$ (A21)
$\displaystyle\times\left.\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}I(T\geq t-x)\right).$

Because the term in brackets in (A21) is some function
$g^{*}\\{X,A,\bar{L}(t-x)\\}$, say, it follows by an argument similar to that
above that the last two lines are equal to zero, and thus (A17) is equal to

$\displaystyle
E\\{(\ref{eq:termtwo})\times(\ref{eq:termthree})\\}=\int^{t}_{s}\frac{q(x)dx}{Q^{2}(x)}E\left(\big{[}\\{m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right).$ (A22)

Combining the results in (A12), (A13), (A16), and (A22) demonstrates the
desired result that

$\displaystyle E\\{m_{s}(\mathcal{O}^{(s)})m_{t}(\mathcal{O}^{(t)})\\}$
$\displaystyle=\frac{\mbox{var}\\{m(Y,A,X)-(A-\pi)f^{opt}(X)\\}}{Q(t)}$
$\displaystyle+\int^{t}_{0}\frac{q(x)dx}{Q^{2}(x)}E\left(\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x,X,A,\bar{L}(t-x)\\}\big{]}^{2}I(T\geq t-x)\right),$

which is (A7).

Note that the influence function for the IPWCC estimator solving (15) is given
by (13) with $f(X)=h\\{u,X,A,\bar{L}(u)\\}\equiv 0$. Using the equality (A1)
and the definition of $\widetilde{M}(x)$, the influence function for the IPWCC
estimator can be written as

$\displaystyle m^{IPW}_{t}(\mathcal{O}^{(t)})=\frac{I(E\leq
t)}{Q(t)}m(Y,A,X)-\int^{t}_{0}\frac{d\widetilde{M}(x)}{Q(x)}\big{[}m(Y,A,X)-E\\{m(Y,A,X)|T\geq
t-x\\}\big{]}I(T\geq t-x).$

That
$E\\{m^{IPW}_{s}(\mathcal{O}^{(s)})m^{IPW}_{t}(\mathcal{O}^{(s)})\\}=\mbox{var}\\{m^{IPW}_{t}(\mathcal{O}^{(t)})\\}$
follows by an argument analogous to that above, demonstrating that the IPWCC
estimator also has the independent increments property.

In the practical implementation discussed in Section 4, the optimal choices
$f^{opt}(X)$ and $h^{opt}\\{u,X,A,\bar{L}(u)\\}$ are approximated using linear
combinations of basis functions. Accordingly, the resulting AIPWCC estimators
obtained via the two-step algorithm may not be fully efficient and thus are
not guaranteed to have the independent increments property. However, as
demonstrated in our simulation studies, because the approximations to
$f^{opt}(X)$ and $h^{opt}\\{u,X,A,\bar{L}(u)\\}$ are often quite good, the
estimators themselves are good approximations to the efficient estimator and
thus exhibit behavior very close to that of independent increments, so that
the operating characteristics of the trial are preserved.

##  Appendix B Additional Simulation Results

Simulation Scenario 1: Ordinal Categorical Outcome: Under the null hypothesis,
based on 10000 Monte Carlo data sets, the Monte Carlo sample covariance
matrices of
$\\{\widehat{\beta}(t_{1}),\ldots,\widehat{\beta}(t_{4}),\widehat{\beta}(t_{end})\\}$,
where $\widehat{\beta}(t)$ is each of $\widehat{\beta}_{\mathcal{T}_{F}}(t)$,
$\widehat{\beta}_{IPW}(t)$, $\widehat{\beta}_{AIPW1}(t)$, and
$\widehat{\beta}_{AIPW2}(t)$, are given by

    betahat_Tf
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.086 0.049 0.034 0.026 0.021
    [2,] 0.049 0.049 0.034 0.026 0.021
    [3,] 0.034 0.034 0.034 0.026 0.021
    [4,] 0.026 0.026 0.026 0.026 0.021
    [5,] 0.021 0.021 0.021 0.021 0.021

    betahat_IPW
    [1,] 0.054 0.036 0.027 0.023 0.022
    [2,] 0.036 0.036 0.027 0.023 0.022
    [3,] 0.027 0.027 0.028 0.023 0.022
    [4,] 0.023 0.023 0.023 0.023 0.022
    [5,] 0.022 0.022 0.022 0.022 0.022

    betahat_AIPW1
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.049 0.031 0.023 0.019 0.018
    [2,] 0.031 0.032 0.023 0.019 0.018
    [3,] 0.023 0.023 0.024 0.020 0.019
    [4,] 0.019 0.019 0.020 0.020 0.018
    [5,] 0.018 0.018 0.019 0.018 0.018

    betahat_AIPW2
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.041 0.027 0.022 0.019 0.019
    [2,] 0.027 0.028 0.021 0.019 0.018
    [3,] 0.022 0.021 0.022 0.019 0.019
    [4,] 0.019 0.019 0.019 0.019 0.018
    [5,] 0.019 0.018 0.019 0.018 0.018

Under the alternative $\beta_{A}=\log(1.5)$, the analogous Monte Carlo sample
covariance matrices are

    betahat_Tf
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.087 0.048 0.034 0.026 0.021
    [2,] 0.048 0.049 0.034 0.026 0.021
    [3,] 0.034 0.034 0.034 0.027 0.022
    [4,] 0.026 0.026 0.027 0.027 0.022
    [5,] 0.021 0.021 0.022 0.022 0.022

    betahat_IPW
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.055 0.036 0.028 0.023 0.022
    [2,] 0.036 0.036 0.028 0.023 0.022
    [3,] 0.028 0.028 0.028 0.024 0.022
    [4,] 0.023 0.023 0.024 0.024 0.022
    [5,] 0.022 0.022 0.022 0.022 0.022

    betahat_AIPW1
         [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.050 0.032 0.024 0.020 0.019
    [2,] 0.032 0.032 0.024 0.020 0.019
    [3,] 0.024 0.024 0.025 0.020 0.019
    [4,] 0.020 0.020 0.020 0.020 0.019
    [5,] 0.019 0.019 0.019 0.019 0.019

    betahat_AIPW2
    [1,] 0.042 0.027 0.022 0.019 0.019
    [2,] 0.027 0.029 0.022 0.019 0.019
    [3,] 0.022 0.022 0.022 0.019 0.019
    [4,] 0.019 0.019 0.019 0.019 0.019
    [5,] 0.019 0.019 0.019 0.019 0.019

These results clearly demonstrate that the independent increments property
holds approximately for all estimators.

Simulation Scenario 2: Binary Outcome: Under the null hypothesis, based on
10000 Monte Carlo data sets, the Monte Carlo sample covariance matrices of
$\\{\widehat{\beta}(t_{1}),\ldots,\widehat{\beta}(t_{4}),\widehat{\beta}(t_{end})\\}$,
where $\widehat{\beta}(t)$ is each of $\widehat{\beta}_{\mathcal{T}_{F}}(t)$,
$\widehat{\beta}_{IPW}(t)$, $\widehat{\beta}_{AIPW1}(t)$, and
$\widehat{\beta}_{AIPW2}(t)$, are given by

    betahat_Tf
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.039 0.022 0.015 0.012 0.009
    [2,] 0.022 0.021 0.015 0.011 0.009
    [3,] 0.015 0.015 0.015 0.011 0.009
    [4,] 0.012 0.011 0.011 0.011 0.009
    [5,] 0.009 0.009 0.009 0.009 0.009

    betahat_IPW
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.018 0.013 0.011 0.009 0.009
    [2,] 0.013 0.013 0.010 0.009 0.009
    [3,] 0.011 0.010 0.010 0.009 0.009
    [4,] 0.009 0.009 0.009 0.009 0.009
    [5,] 0.009 0.009 0.009 0.009 0.009

    betahat_AIPW1
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.017 0.012 0.009 0.008 0.008
    [2,] 0.012 0.012 0.009 0.008 0.008
    [3,] 0.009 0.009 0.009 0.008 0.008
    [4,] 0.008 0.008 0.008 0.008 0.008
    [5,] 0.008 0.008 0.008 0.008 0.008

    betahat_AIPW2
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.017 0.012 0.009 0.008 0.008
    [2,] 0.012 0.012 0.009 0.008 0.008
    [3,] 0.009 0.009 0.009 0.008 0.008
    [4,] 0.008 0.008 0.008 0.008 0.008
    [5,] 0.008 0.008 0.008 0.008 0.008

Under the alternative $\beta_{A}=\log(0.247/0.33)=-0.290$, the analogous Monte
Carlo sample covariance matrices are

    betahat_Tf
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.049 0.027 0.019 0.015 0.012
    [2,] 0.027 0.027 0.019 0.014 0.012
    [3,] 0.019 0.019 0.019 0.014 0.011
    [4,] 0.015 0.014 0.014 0.014 0.011
    [5,] 0.012 0.012 0.011 0.011 0.011

    betahat_IPW
          [,1]  [,2]  [,3]  [,4]  [,5]
    [1,] 0.023 0.017 0.013 0.012 0.012
    [2,] 0.017 0.017 0.013 0.012 0.011
    [3,] 0.013 0.013 0.013 0.011 0.011
    [4,] 0.012 0.012 0.011 0.011 0.011
    [5,] 0.012 0.011 0.011 0.011 0.011

    betahat_AIPW1
          [,1]  [,2]  [,3] [,4] [,5]
    [1,] 0.022 0.015 0.012 0.010 0.010
    [2,] 0.015 0.015 0.012 0.010 0.010
    [3,] 0.012 0.012 0.012 0.010 0.010
    [4,] 0.010 0.010 0.010 0.010 0.010
    [5,] 0.010 0.010 0.010 0.010 0.010

    betahat_AIPW2
          [,1]  [,2]  [,3] [,4] [,5]
    [1,] 0.022 0.015 0.012 0.010 0.010
    [2,] 0.015 0.015 0.012 0.010 0.010
    [3,] 0.012 0.012 0.012 0.010 0.010
    [4,] 0.010 0.010 0.010 0.010 0.010
    [5,] 0.010 0.010 0.010 0.010 0.010

These results clearly demonstrate that the independent increments property
holds approximately for all estimators.

Table B1 presents performance of the estimators under the null and alternative
hypotheses.

Table B1: For Scenario 2 with binary outcome, performance of estimators for $\beta$ under (a) the null hypothesis $\beta=0$ and (b) the alternative $\beta=\log(0.247/0.33)=-0.290$ at each interim analysis time $(t_{1},\ldots,t_{4})=(150,195,240,285)$ days and at the final analysis at $t_{end}=330$ days MC Mean is the mean of 10000 Monte Carlo estimates; MC SD is the Monte Carlo standard deviation, Ave MC SE is the mean of Monte Carlo standard errors, and MSE ratio is the ratio of Monte Carlo mean square error for the AIPW2 estimator divided by that for the indicated estimator. | MC Mean | MC SD | Ave MC SE | MSE ratio |  | MC Mean | MC SD | Ave MC SE | MSE ratio
---|---|---|---|---|---|---|---|---|---
| (a) Null Hypothesis
| $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
$t_{1}$ | 0.003 | 0.197 | 0.193 | 1.000 |  | 0.000 | 0.136 | 0.134 | 2.097
$t_{2}$ | 0.000 | 0.146 | 0.145 | 1.000 |  | 0.000 | 0.115 | 0.114 | 1.600
$t_{3}$ | 0.000 | 0.122 | 0.121 | 1.000 |  | -0.001 | 0.102 | 0.101 | 1.433
$t_{4}$ | 0.000 | 0.106 | 0.106 | 1.000 |  | 0.000 | 0096 | 0.095 | 1.237
$t_{end}$ | 0.000 | 0.096 | 0.095 | 1.000 |  | 0.000 | 0.096 | 0.095 | 1.000
| $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{1}$ | 0.000 | 0.130 | 0.128 | 2.302 |  | 0.001 | 0.130 | 0.128 | 2.300
$t_{2}$ | 0.000 | 0.110 | 0.109 | 1.761 |  | 0.000 | 0.110 | 0.109 | 1.759
$t_{3}$ | -0.001 | 0.097 | 0.097 | 1.587 |  | -0.001 | 0.097 | 0.096 | 1.591
$t_{4}$ | 0.000 | 0.090 | 0.090 | 1.389 |  | 0.000 | 0.090 | 0.090 | 1.389
$t_{end}$ | 0.000 | 0.090 | 0.090 | 1.123 |  | 0.000 | 0.090 | 0.090 | 1.123
| (b) Alternative Hypothesis
| $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
$t_{1}$ | -0.291 | 0.220 | 0.216 | 1.000 |  | -0.292 | 0.153 | 0.151 | 2.072
$t_{2}$ | -0.291 | 0.164 | 0.162 | 1.000 |  | -0.291 | 0.130 | 0.129 | 1.591
$t_{3}$ | -0.292 | 0.136 | 0.135 | 1.000 |  | -0.291 | 0.115 | 0.114 | 1.412
$t_{4}$ | -0.291 | 0.119 | 0.118 | 1.000 |  | -0.290 | 0.107 | 0.107 | 1.242
$t_{end}$ | -0.290 | 0.107 | 0.107 | 1.000 |  | -0.290 | 0.107 | 0.107 | 1.000
| $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{1}$ | -0.291 | 0.147 | 0.146 | 2.246 |  | -0.291 | 0.147 | 0.145 | 2.249
$t_{2}$ | -0.291 | 0.124 | 0.123 | 1.734 |  | -0.291 | 0.124 | 0.123 | 1.736
$t_{3}$ | -0.291 | 0.110 | 0.109 | 1.545 |  | -0.291 | 0.109 | 0.109 | 1.548
$t_{4}$ | -0.290 | 0.102 | 0.102 | 1.377 |  | -0.290 | 0.102 | 0.101 | 1.373
$t_{end}$ | -0.290 | 0.101 | 0.101 | 1.109 |  | -0.290 | 0.101 | 0.101 | 1.109

Simulation Scenario 3: Continuous Outcome: Under the null hypothesis, based on
10000 Monte Carlo data sets, the Monte Carlo sample covariance matrices of
$\\{\widehat{\beta}(t_{1}),\ldots,\widehat{\beta}(t_{4}),\widehat{\beta}(t_{end})\\}$,
where $\widehat{\beta}(t)$ is each of $\widehat{\beta}_{\mathcal{T}_{F}}(t)$,
$\widehat{\beta}_{IPW}(t)$, $\widehat{\beta}_{AIPW1}(t)$, and
$\widehat{\beta}_{AIPW2}(t)$, are given by

    betahat_Tf
          [,1] [,2] [,3] [,4] [,5]
    [1,] 11.67 7.84 5.85 4.65 3.85
    [2,]  7.84 7.89 5.87 4.69 3.90
    [3,]  5.85 5.87 5.83 4.65 3.87
    [4,]  4.65 4.69 4.65 4.63 3.86
    [5,]  3.85 3.90 3.87 3.86 3.88

    \betahat_IPW
          [,1] [,2] [,3] [,4] [,5]
    [1,] 11.67 7.84 5.85 4.65 3.85
    [2,]  7.84 7.89 5.87 4.69 3.90
    [3,]  5.85 5.87 5.83 4.65 3.87
    [4,]  4.65 4.69 4.65 4.63 3.86
    [5,]  3.85 3.90 3.87 3.86 3.88

    \betaht_AIPW1
          [,1] [,2] [,3] [,4] [,5]
    [1,] 10.65 6.81 4.82 3.82 3.14
    [2,]  6.81 7.08 5.04 3.86 3.19
    [3,]  4.82 5.04 5.13 3.96 3.16
    [4,]  3.82 3.86 3.96 3.94 3.16
    [5,]  3.14 3.19 3.16 3.16 3.17

    betahat_AIPW2
         [,1] [,2] [,3] [,4] [,5]
    [1,] 7.92 5.47 4.27 3.50 3.17
    [2,] 5.47 5.44 4.17 3.49 3.16
    [3,] 4.27 4.17 4.15 3.48 3.16
    [4,] 3.50 3.49 3.48 3.46 3.16
    [5,] 3.17 3.16 3.16 3.16 3.17

Under the alternative $\beta_{A}=6.24$, the analogous Monte Carlo sample
covariance matrices are

    betahat_Tf
          [,1] [,2] [,3] [,4] [,5]
    [1,] 11.71 7.87 5.88 4.67 3.86
    [2,]  7.87 7.92 5.90 4.71 3.91
    [3,]  5.88 5.90 5.85 4.67 3.89
    [4,]  4.67 4.71 4.67 4.65 3.88
    [5,]  3.86 3.91 3.89 3.88 3.89

    betahat_IPW
          [,1] [,2] [,3] [,4] [,5]
    [1,] 11.71 7.87 5.88 4.67 3.86
    [2,]  7.87 7.92 5.90 4.71 3.91
    [3,]  5.88 5.90 5.85 4.67 3.89
    [4,]  4.67 4.71 4.67 4.65 3.88
    [5,]  3.86 3.91 3.89 3.88 3.89

    betahat_AIPW1
          [,1] [,2] [,3] [,4] [,5]
    [1,] 10.69 6.84 4.84 3.83 3.15
    [2,]  6.84 7.10 5.06 3.87 3.20
    [3,]  4.84 5.06 5.15 3.97 3.18
    [4,]  3.83 3.87 3.97 3.95 3.17
    [5,]  3.15 3.20 3.18 3.17 3.18

    betahat_AIPW2
         [,1] [,2] [,3] [,4] [,5]
    [1,] 7.94 5.49 4.29 3.52 3.18
    [2,] 5.49 5.46 4.18 3.51 3.17
    [3,] 4.29 4.18 4.16 3.49 3.18
    [4,] 3.52 3.51 3.49 3.47 3.18
    [5,] 3.18 3.17 3.18 3.18 3.18

Table B2 presents performance of the estimators under the null and alternative
hypotheses.

Table B2: For Scenario 3 with continuous outcome, performance of estimators for $\beta$ under (a) the null hypothesis $\beta=0$ and (b) the alternative $\beta=6.24$ at each interim analysis time $(t_{1},\ldots,t_{4})=(104,130,156,182)$ days and at the final analysis at $t_{end}=208$ days MC Mean is the mean of 10000 Monte Carlo estimates; MC SD is the Monte Carlo standard deviation, Ave MC SE is the mean of Monte Carlo standard errors, and MSE ratio is the ratio of Monte Carlo mean square error for the AIPW2 estimator divided by that for the indicated estimator. | MC Mean | MC SD | Ave MC SE | MSE ratio |  | MC Mean | MC SD | Ave MC SE | MSE ratio
---|---|---|---|---|---|---|---|---|---
| (a) Null Hypothesis
| $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
$t_{1}$ | 0.012 | 3.416 | 3.415 | 1.000 |  | 0.012 | 3.416 | 3.380 | 1.000
$t_{2}$ | 0.010 | 2.809 | 2.781 | 1.000 |  | 0.010 | 2.809 | 2.763 | 1.000
$t_{3}$ | -0.001 | 2.414 | 2.407 | 1.000 |  | -0.001 | 2.414 | 2.395 | 1.000
$t_{4}$ | -0.005 | 2.152 | 2.151 | 1.000 |  | -0.005 | 2.152 | 2.142 | 1.000
$t_{end}$ | 0.005 | 1.969 | 1.962 | 1.000 |  | 0.005 | 1.969 | 1.956 | 1.000
| $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{1}$ | 0.007 | 3.264 | 3.222 | 1.095 |  | -0.009 | 2.813 | 2.721 | 1.474
$t_{2}$ | -0.006 | 2.660 | 2.609 | 1.115 |  | -0.003 | 2.332 | 2.284 | 1.452
$t_{3}$ | 0.002 | 2.265 | 2.247 | 1.136 |  | 0.001 | 2.037 | 2.014 | 1.405
$t_{4}$ | -0.001 | 1.985 | 1.975 | 1.176 |  | -0.001 | 1.859 | 1.839 | 1.340
$t_{end}$ | 0.008 | 1.780 | 1.772 | 1.225 |  | 0.008 | 1.780 | 1.772 | 1.225
| (b) Alternative Hypothesis
| $\widehat{\beta}_{\mathcal{T}_{F}}(t)$ |  | $\widehat{\beta}_{IPW}(t)$
$t_{1}$ | 6.230 | 3.422 | 3.421 | 1.000 |  | 6.230 | 3.422 | 3.386 | 1.000
$t_{2}$ | 6.208 | 2.815 | 2.786 | 1.000 |  | 6.208 | 2.815 | 2.768 | 1.000
$t_{3}$ | 6.216 | 2.419 | 2.411 | 1.000 |  | 6.216 | 2.419 | 2.399 | 1.000
$t_{4}$ | 6.213 | 2.157 | 2.154 | 1.000 |  | 6.213 | 2.157 | 2.146 | 1.000
$t_{end}$ | 6.223 | 1.973 | 1.966 | 1.000 |  | 6.223 | 1.973 | 1.959 | 1.000
| $\widehat{\beta}_{AIPW1}(t)$ |  | $\widehat{\beta}_{AIPW2}(t)$
$t_{1}$ | 6.225 | 3.269 | 3.227 | 1.096 |  | 6.210 | 2.818 | 2.726 | 1.474
$t_{2}$ | 6.212 | 2.665 | 2.613 | 1.115 |  | 6.216 | 2.336 | 2.288 | 1.452
$t_{3}$ | 6.220 | 2.269 | 2.250 | 1.137 |  | 6.219 | 2.041 | 2.017 | 1.405
$t_{4}$ | 6.217 | 1.989 | 1.979 | 1.176 |  | 6.217 | 1.863 | 1.842 | 1.341
$t_{end}$ | 6.226 | 1.783 | 1.775 | 1.225 |  | 6.226 | 1.783 | 1.775 | 1.225

## References

  * 1 Tsiatis A. Information-based monitoring of clinical trials. Stat Med 2006; 25: 3236–3244.
  * 2 Pocock S. Group sequential methods in the design and analysis of clinical trials. Biometrika 1977; 64: 191–199.
  * 3 O’Brien P, Fleming T. A multiple testing procedure for clinical trials. Biometrics 1979; 35: 549–556.
  * 4 ClinicalTrials.gov, National Library of Medicine (US) . ACTIV-3b: Therapeutics for Severely Ill Inpatients With COVID-19 (TESICO). NCT04843761. https://clinicaltrials.gov/ct2/show/NCT04843761; .
  * 5 ClinicalTrials.gov, National Library of Medicine (US) . ACTIV-2: A Study for Outpatients With COVID-19. NCT04518410. https://clinicaltrials.gov/ct2/show/NCT04518410; .
  * 6 Lu X, Tsiatis A. Semiparametric estimation of treatment effect with time-lagged response in the presence of informative censoring. Lifetime Data Anal 2011; 17: 566–593.
  * 7 Tsiatis A. Semiparametric Theory and Missing Data.  New York, NY: Springer .  2006\.
  * 8 Scharfstein D, Tsiatis A, Robins J. Semiparametric efficiency and its implication on the design and analysis of group sequential studies. J Am Stat Assoc 1997; 92: 1342–1350.
  * 9 Lan K, DeMets D. Discrete sequential boundaries for clinical trials. Biometrika 1983; 70: 659–663.
  * 10 Tsiatis A, Davidian M, Holloway S. Estimation of the odds ratio in a proportional odds model with censored time-lagged outcome in a randomized clinical trial. Biometrics 2021; 78: 1–13.
  * 11 Hellmich M. Monitoring clinical trials with multiple arms. Biometrics 2001; 57: 892–898.
  * 12 Wason J. Design of multi-arm, multi-stage trials in oncology. In: Halabi S, Michiels S. , eds. Textbook of Clinical Trials in Oncology: A Statistical Perspective. New York: Chapman and Hall/CRC Press. 2019 (pp. 155-182).
  * 13 Agresti A. An Introduction to Categorical Data Analysis.  New York, NY: Wiley .  2019\.
  * 14 Stefanski L, Boos D. The calculus of M-estimation. Amer Statist 2002; 56: 29–38.
  * 15 Zhang M, Tsiatis A, Davidian M. Improving efficiency of inferences in randomized clinical trials using auxiliary covariates. Biometrics 2008; 64: 707–715.
  * 16 Kim KM, Tsiatis AA. Independent increments in group sequential tests: a review. SORT - Statistics and Operations Research Transactions 2020; 44: 223–264.
  * 17 Casper C, Cook T, Perez OA. Package ldbounds: Lan-DeMets method for group sequential boundaries. Comprehensive R Archive Network. https://cran.r-project.org/package=ldbounds; .
  * 18 Mehta CR, Tsiatis AA. Flexible sample size considerations using information-based interim monitoring. Drug Info J 2001; 35: 1095–1112.
  * 19 Venables W, Ripley B. Modern Applied Statistics with S, 4th edition.  New York, NY: Springer .  2002\.