The next result addresses the case where $\hat{P}_{n}$ is a non-empirical
bootstrap sampling distribution. We let $\hat{Q}_{n}$ be the marginal
distribution of $Z$ under $\hat{P}_{n}$,
$\hat{g}_{n}(z):=\hat{P}_{n}(A=1|Z=z)$,
$\hat{\mu}_{n}(z):=E_{\hat{P}_{n}}(Y|A=0,Z=z)$, and
$\hat{\sigma}_{n}^{2}(z):=\mathrm{Var}_{\hat{P}_{n}}(Y\mid A=0,Z=z)$, all of
which we assume are well-defined. We note that $\hat{Q}_{n}$ need not be a
smooth distribution. We also define $\sigma_{0}^{2}(z):=\mathrm{Var}_{0}(Y\mid
A=0,Z=z)$. We then have the following result regarding conditional asymptotic
linearity of the bootstrap one-step and estimating equations-based estimators
when sampling from $\hat{P}_{n}$.

######  Proposition 5.8.

Suppose $P_{0}^{*}(\hat{P}_{n}\in\mathscr{P})\to 1$, where $\mathscr{P}$ is
such that $\lim_{M\to\infty}\sup_{P\in\mathscr{P}}E_{P}[Y^{2}I(Y^{2}>M)]=0$,
$\hat{\mu}_{n}\in\mathscr{F}_{\mu}$ and $\hat{g}_{n}\in\mathscr{F}_{g}$ with
probability tending to one, where $\mathscr{F}_{\mu}$ is uniformly bounded,
$\mathscr{F}_{g}$ is uniformly bounded away from zero, and $\mathscr{F}_{\mu}$
and $\mathscr{F}_{g}$ possess finite uniform entropy integrals,
$\|\hat{g}_{n}-g_{0}\|_{L_{2}(P_{0})}$,
$\|\hat{\mu}_{n}-\mu_{0}\|_{L_{2}(P_{0})}$,
$\|\hat{\sigma}_{n}^{2}-\sigma_{0}^{2}\|_{L_{2}(P_{0})}$ are each
$o_{P_{0}^{*}}(1)$, and each of the following is $o_{P_{0}^{*}}(1)$:

$\displaystyle\begin{split}\sup_{g,\bar{g}}\left|(\hat{Q}_{n}-Q_{0})\left[\frac{g\bar{g}(1-\hat{g}_{n})}{(1-g)(1-\bar{g})}(\hat{\sigma}_{n}^{2}+\hat{\mu}_{n}^{2})\right]\right|,\,\sup_{g,\bar{g},\mu,\bar{\mu}}\left|(\hat{Q}_{n}-Q_{0})\left[\frac{g\bar{g}(1-\hat{g}_{n})}{(1-g)(1-\bar{g})}\mu\bar{\mu}\right]\right|,\\\
\sup_{\mu}\left|(\hat{Q}_{n}-Q_{0})\left[\mu\hat{g}_{n}\right]\right|,\,\sup_{\mu,\bar{\mu}}\left|(\hat{Q}_{n}-Q_{0})\left[\mu\bar{\mu}\hat{g}_{n}\right]\right|,\,\sup_{\mu,g}\left|{(\hat{Q}_{n}-Q_{0})\left[\frac{g(1-\hat{g}_{n})}{1-g}\mu\right]}\right|,\,(\hat{Q}_{n}-Q_{0})\hat{g}_{n}.\end{split}$
(14)

where the suprema over $\mu$ and $\bar{\mu}$ are taken over
$\mathscr{F}_{\mu}$ and the suprema over $g$ and $\bar{g}$ are taken over
$\mathscr{F}_{g}$, and
$\hat{Q}_{n}\left\\{(\hat{g}_{n}-g_{n})(\hat{\mu}_{n}-\mu_{n})/(1-g_{n})\right\\}=o_{P_{0}^{*}}(n^{-1/2})$.
Suppose also that $\mu_{n}^{*}\in\mathscr{F}_{\mu}$ and
$g_{n}^{*}\in\mathscr{F}_{g}$ with conditional probability tending to one,
$\left\|g_{n}^{*}-g_{0}\right\|_{L_{2}(P_{0})}=o_{P_{W}^{*}}(1)$,
$\left\|\mu_{n}^{*}-\mu_{0}\right\|_{L_{2}(P_{0})}=o_{P_{W}^{*}}(1)$, and
$\hat{Q}_{n}\left\\{(g_{n}^{*}-\hat{g}_{n})(\mu_{n}^{*}-\hat{\mu}_{n})/(1-g_{n}^{*})\right\\}=o_{P_{W}^{*}}(n^{-1/2})$.
If
$(\pi_{n}^{*}-\hat{\pi}_{n})(\psi_{n}^{*}-\hat{\psi}_{n})=o_{P_{W}^{*}}(n^{-1/2})$
and
$(\pi_{n}-\hat{\pi}_{n})(\psi_{n}-\hat{\psi}_{n})=o_{P_{0}^{*}}(n^{-1/2})$,
then $\psi_{n,1}^{*}$ is conditionally asymptotically linear with influence
function $\phi_{0}$. If the conditions of 5.6 hold and
$\pi_{n}^{*}-\hat{\pi}_{n}=o_{P_{W}^{*}}(1)$, then $\psi_{n,2}^{*}$ is
conditionmally asymptotically linear with influence function $\phi_{0}$.

5.8 illustrates that the bootstrap sampling distribution $\hat{P}_{n}$ can
produce valid bootstrap confidence intervals even if it is not globally
consistent. In this case, it is sufficient that the propensity score,
conditional mean, and conditional variance functions induced by $\hat{P}_{n}$
be consistent, and that certain means of the marginal distribution of the
covariates $\hat{Q}_{n}$ be consistent. If $\hat{Q}_{n}=Q_{n}$ is the
empirical distribution of the observed covariates, then the conditions in (14)
hold by the assumption that $\mathscr{F}_{\mu}$ and $\mathscr{F}_{g}$ possess
finite uniform entropy integrals. However, it may be of interest to use
something other than the empirical distribution for $\hat{Q}_{n}$ in order to,
for instance, produce unique bootstrap covariate values. We also note that it
is possible that stronger notions of consistency of $\hat{P}_{n}$ have
implications for higher-order properties of bootstrap confidence intervals.
Finally, we note that some of the conditions of 5.8 hold automatically if
$\hat{\mu}_{n}=\mu_{n}$ or $\hat{g}_{n}=g_{n}$.

##  6 Numerical study

We conducted a simulation study to assess the finite-sample performance of the
methods of inference for the average density value parameter studied in
Section 5.1. We set $P_{0}$ as the standard normal distribution. For each
sample size $n\in\\{50,100,200,300,400,500,1000,2000,3000,4000,5000\\}$, we
simulated 1000 datasets of $n$ independent and identically distributed
observations from $P_{0}$. For each dataset, we considered the three estimator
constructions defined in Section 5: the one-step estimator
$\psi_{n,1}=T_{1}(\eta_{n},\mathbb{P}_{n})$, the plug-in estimator
$\psi_{n,2}=T_{2}(\eta_{n},\mathbb{P}_{n})$, and the empirical mean plug-in
estimator $\psi_{n,3}=T_{3}(\eta_{n},\mathbb{P}_{n})$. For each estimator, we
used three different nuisance estimators $\eta_{n}$: (1) a KDE with Gaussian
kernel and bandwidth $h$ selected at the optimal rate $h\propto n^{-1/5}$ for
twice-differentiable densities using the method of Sheather & Jones, (1991);
(2) a KDE with under-smoothed bandwidth $h/n^{1/10}$; and (3) TMLE using (1)
as the initial estimator. Hence, we constructed a total of nine distinct
estimators for each dataset.

We considered four bootstrap sampling distributions: the empirical
distribution, and three smooth distributions corresponding to the three
density estimators defined above. For each dataset, we generated $B=1000$
bootstrap datasets using these four bootstrap distributions. For each
bootstrap dataset, we then considered the same three estimator constructions
using the bootstrap data:
$\psi_{n,1}^{*}=T_{1}(\eta_{n}^{*},\mathbb{P}_{n}^{*})$,
$\psi_{n,2}^{*}=T_{2}(\eta_{n}^{*},\mathbb{P}_{n}^{*})$, and
$\psi_{n,3}^{*}=T_{3}(\eta_{n}^{*},\mathbb{P}_{n}^{*})$. We considered two
bootstrap nuisance estimators $\eta_{n}^{*}$: the same nuisance estimation
procedure used for the original data applied to the bootstrap sample, with
bandwidth fixed at the value selected using the original data, and using the
fixed nuisance estimator obtained from the original data, i.e.
$\eta_{n}^{*}=\eta_{n}$. Finally, we used all four methods of constructing
bootstrap confidence intervals defined in Section 3.4 to construct two-sided,
equi-tailed 95% confidence intervals for $\psi_{0}$ based on each bootstrap
sample. For the percentile $t$-method, we used the influence function-based
variance estimator. For comparison, we also constructed ordinary Wald-style
confidence intervals using the influence function-based variance estimator. We
evaluated the performance of these confidence intervals by computing their
empirical coverage and average width over the 1000 simulations.

Figure 1: Empirical coverage of 95% confidence intervals based on the
bootstrap one-step estimator when re-estimating the nuisance using the
bootstrap sample. “KDE” stands for kernel density estimator; “US” stands for
under-smoothed; TMLE stands for targeted maximum likelihood estimator. Figure
2: Scaled average width of 95% confidence intervals based on the bootstrap
one-step estimator when re-estimating the nuisance using the bootstrap sample.
Abbreviations as in Figure 1.

We now turn to the results of the simulation study. Figures 1, 3, and 4
display empirical coverage and Figures 2, 5 and 6 display the average width of
95% confidence intervals based on the one-step, plug-in, and empirical mean
plug-in estimators, respectively, when the bootstrap nuisance was re-estimated
using the bootstrap sample. Figures 7 and 8 display empirical coverage and
Figures 9 and 10 display average width based on the one-step and empirical
mean plug-in constructions, respectively, when the bootstrap nuisance was
fixed. The results for the plug-in construction with the nuisance fixed are
not displayed because the coverage in this case was always zero, as discussed
in Section 4.2. In each figure, the rows represent the bootstrap sampling
distribution and the columns represent the method of construction of the
nuisance estimator. For example, the top left panel of Figure 1 shows the
empirical coverage rate of confidence intervals based on the one-step
estimator where the bootstrap sampling distribution was the empirical
distribution $\mathbb{P}_{n}$ and the nuisance estimator was the KDE with
optimal bandwidth.

We first discuss the results displayed in Figure 1 for the one-step estimator
with re-estimated nuisance. Efron’s percentile method did not yield valid
confidence intervals at large sample sizes when the bootstrap sampling
distribution was based on a KDE with optimal bandwidth (second row from the
top). This was expected based on the results of Section 3.4. The bias
$T_{1}(\eta_{n},\hat{P}_{n})-T_{1}(\eta_{n},\mathbb{P}_{n})=(\mathbb{P}_{n}-\hat{P}_{n})\phi_{\eta_{n}}$
in this case is not $o_{P_{0}^{*}}(n^{-1/2})$ because $\hat{P}_{n}$ was not
under-smoothed. The coverage of all other confidence intervals for the
bootstrap one-step estimator approached 95% as the sample size increased,
which is in line with Propositions 5.2 and 5.4. Efron’s percentile confidence
intervals had poor coverage for small and moderate sample sizes in some cases,
which we hypothesize is due to excess bias in this method, as discussed in
Section 3.4. The coverage of (non-bootstrap) Wald-type confidence intervals
approached 95% in all cases, as expected, though its performance for smaller
sample sizes was not always good. An exception was when the nuisance estimator
was TMLE (third column from the left), in which case the coverage of the Wald-
type estimator was excellent at all sample sizes considered. Otherwise, there
was no clear and consistent best nuisance estimator or bootstrap sampling
distribution. The average widths scaled by $n^{1/2}$ displayed in Figure 2 all
converge to the same value.

Figure 3: Empirical coverage of 95% confidence intervals based on the
bootstrap plug-in estimator when re-estimating the nuisance using the
bootstrap sample. Abbreviations as in Figure 1. Figure 4: Empirical coverage
of 95% confidence intervals based on the bootstrap empirical mean plug-in
estimator when re-estimating the nuisance using the bootstrap sample.
Abbreviations as in Figure 1. Figure 5: Scaled average width of 95% confidence
intervals based on the bootstrap plug-in estimator when re-estimating the
nuisance using the bootstrap sample. Abbreviations as in Figure 1. Figure 6:
Scaled average width of 95% confidence intervals based on the bootstrap
empirical mean plug-in estimator when re-estimating the nuisance using the
bootstrap sample. Abbreviations as in Figure 1.

We next discuss the results displayed in Figures 3 and 4 for the plug-in and
empirical mean plug-in estimators with re-estimated nuisance. Using the KDE
with optimal bandwidth as nuisance estimator did not generally yield valid
Wald or bootstrap confidence intervals because neither estimator is
asymptotically linear in this case (first column from the left). However, the
percentile and percentile-$t$ confidence intervals had close to nominal
coverage in large samples for both estimators when both the nuisance estimator
and bootstrap sampling distribution were the KDE with optimal bandwidth
(second row from the top and first column from the left). This was expected
based on 5.5. Besides the bootstrap Wald method, using the KDE with optimal
bandwidth as the bootstrap sampling distribution did not yield valid bootstrap
confidence intervals in large samples for other nuisance estimators (second
row from the top and second and third columns from the left). All other
confidence intervals based on the plug-in and empirical mean plug-in
estimators with re-estimated nuisance had close to 95% coverage for large
sample sizes, which aligns with Propositions 5.1, 5.2, and 5.4. Among the
methods with good large-sample coverage, using TMLE as the nuisance estimator
and bootstrap sampling distribution along with the Wald, bootstrap Wald,
percentile, or percentile $t$-confidence interval yielded the best coverage
for small and moderate sample sizes. In many cases, Efron’s method again had
poor coverage in small and moderate samples. The average widths scaled by
$n^{1/2}$ displayed in Figures 5 and 6 appear to again converge to roughly the
same value.

Figure 7: Empirical coverage of 95% confidence intervals based on the
bootstrap one-step estimator when the nuisance was not re-estimated using the
bootstrap sample. Abbreviations as in Figure 1. Figure 8: Empirical coverage
of 95% confidence intervals based on the bootstrap empirical mean plug-in
estimator when the nuisance was not re-estimated using the bootstrap sample.
Abbreviations as in Figure 1. Figure 9: Scaled average width of 95% confidence
intervals based on the bootstrap one-step when the nuisance was not re-
estimated using the bootstrap sample. Abbreviations as in Figure 1. Figure 10:
Scaled average width of 95% confidence intervals based on the bootstrap
empirical mean plug-in estimator when the nuisance was not re-estimated using
the bootstrap sample. Notes as in Figure 1.

We now turn to the results displayed in Figure 7 for the one-step estimator
with fixed bootstrap nuisance. Efron’s percentile method again did not yield
valid coverage in large samples when the bootstrap sampling distribution was
based on a KDE with optimal bandwidth (second row from the top). As discussed
following Figure 1, this was expected based on the results of Section 3.4. All
other confidence intervals for the bootstrap one-step estimator with fixed
bootstrap nuisance estimator had good coverage in large samples, as expected
based on 4.1. Compared to the results in Figure 1 for the bootstrap one-step
estimator with re-estimated nuisance, the coverage of bootstrap Wald,
percentile, and percentile $t$-confidence intervals were mostly worse when
fixing the bootstrap nuisance than when re-estimating it. However,
interestingly, the coverage of Efron’s percentile intervals was often better.
For the empirical bootstrap, percentile $t$-confidence intervals had slightly
better coverage in small samples than other bootstrap intervals, but in many
other cases the various bootstrap methods had very similar coverage when the
nuisance was fixed. For many of the cases considered, bootstrap intervals had
better coverage in small and moderate samples than Wald-type intervals,
indicating there may still be a benefit of the bootstrap even when fixing the
bootstrap nuisance. There was not a substantial difference between the
bootstrap sampling distributions, but the KDE with optimal bandwidth had the
best overall performance in small and moderate samples. As with the bootstrap
one-step estimator with the nuisance re-estimated, TMLE had the best
performance among the nuisance estimators considered, with excellent coverage
even for small sample sizes. The average widths scaled by $n^{1/2}$ displayed
in Figures 9 appear to again converge to roughly the same value.

Finally, from Figures 8 and 10, all of the bootstrap confidence intervals for
the bootstrap empirical mean plug-in estimator with fixed bootstrap nuisance
had poor coverage at all sample sizes because they were on average too narrow.
This is in line with the results and discussion of Section 4.3.

##  7 Conclusion

In this paper, we studied the problem of bootstrapping asymptotically
estimators that rely on a data-adaptive nuisance estimator. We proposed a
framework that encompasses many approaches to constructing asymptotically
linear estimators and bootstrapping them, and we provided high-level
conditions for consistency of the bootstrap in this framework. We also
provided more detailed conditions for several bootstrap distributions and
estimator constructions. We used our general results to demonstrate that a
wide variety of bootstrap confidence intervals are asymptotically valid in
this setting, and our simulation study confirmed this. It is our hope that the
generality of our framework and theory ensures that there are many potential
future applications of our results.

An important area of future research is establishing rates of convergence for
the bootstrap methods considered here. We focused on consistency of the
bootstrap because it is an important first step, and because we expect that
rates of convergence will require stronger assumptions than we used here.
However, understanding how the different components of the original estimator
and the bootstrap sampling scheme contribute to the accuracy of bootstrap
confidence intervals is crucial for deciding which method to use in practice.
For instance, while we showed that the precise behavior of the bootstrap
nuisance estimator does not play a role in the first-order asymptotic behavior
of the bootstrap estimator as long as our high-level conditions hold, we
expect that it plays an important role in the finite-sample accuracy of the
bootstrap. This was sometimes, but not always, the case in our numerical
studies. Similarly, while we showed that both the empirical and smooth
bootstraps can yield asymptotically valid bootstrap confidence intervals, our
results did not reveal which approach will yield better finite-sample
coverage.

## Acknowledgements

The authors gratefully acknowledge support from the University of
Massachusetts Amherst Department of Mathematics and Statistics startup fund
and NSF Award 2113171.

## References

  * Austern & Syrgkanis, (2020) Austern, M. & Syrgkanis, V. (2020).  Asymptotics of the empirical bootstrap method beyond asymptotic normality.  arXiv preprint arXiv:2011.11248.
  * Beran, (1997) Beran, R. (1997).  Diagnosing bootstrap success.  Annals of the Institute of Statistical Mathematics, 49, 1–24.
  * Beutner & Zähle, (2023) Beutner, E. & Zähle, H. (2023).  Donsker results for the empirical process indexed by functions of locally bounded variation and applications to the smoothed empirical process.  Bernoulli, 29(1), 205–228.
  * Bickel, (1982) Bickel, P. J. (1982).  On adaptive estimation.  The Annals of Statistics, 10(3), 647–671.
  * Bickel et al., (1997) Bickel, P. J., Götze, F., & van Zwet, W. R. (1997).  Resampling fewer than $n$ observations: gains, losses, and remedies for losses.  Statistica Sinica, 7(1), 1–31.
  * Cai & van der Laan, (2020) Cai, W. & van der Laan, M. (2020).  Nonparametric bootstrap inference for the targeted highly adaptive least absolute shrinkage and selection operator (LASSO) estimator.  The International Journal of Biostatistics, 16(2), 20170070.
  * Carone et al., (2019) Carone, M., Luedtke, A. R., & van der Laan, M. J. (2019).  Toward computerized efficient estimation in infinite-dimensional models.  Journal of the American Statistical Association, 114(527), 1174–1190.  PMID: 32405108.
  * Cattaneo & Jansson, (2018) Cattaneo, M. D. & Jansson, M. (2018).  Kernel-based semiparametric estimators: Small bandwidth asymptotics and bootstrap consistency.  Econometrica, 86(3), 955–995.
  * Cattaneo & Jansson, (2022) Cattaneo, M. D. & Jansson, M. (2022).  Average density estimators: Efficiency and bootstrap consistency.  Econometric Theory, 38(6), 1140–1174.
  * Cattaneo et al., (2020) Cattaneo, M. D., Jansson, M., & Nagasawa, K. (2020).  Bootstrap-based inference for cube root asymptotics.  Econometrica, 88(5), 2203–2219.
  * Cheng & Huang, (2010) Cheng, G. & Huang, J. Z. (2010).  Bootstrap consistency for general semiparametric $M$-estimation.  The Annals of Statistics, 38(5), 2884–2915.
  * Chernozhukov et al., (2018) Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018).  Double/debiased machine learning for treatment and structural parameters.  The Econometrics Journal, 21(1), C1–C68.
  * Chernozhukov et al., (2017) Chernozhukov, V., Chetverikov, D., & Kato, K. (2017).  Central limit theorems and bootstrap in high dimensions.  The Annals of Probability, 45(4), 2309–2352.
  * Chernozhukov et al., (2022) Chernozhukov, V., Chetverikov, D., Kato, K., & Koike, Y. (2022).  Improved central limit theorem and bootstrap approximations in high dimensions.  The Annals of Statistics, 50(5), 2562–2586.
  * Coyle & van der Laan, (2018) Coyle, J. & van der Laan, M. J. (2018).  Targeted bootstrap.  In Targeted learning in data science chapter 28, (pp. 523–539). Springer Cham.
  * Cuevas & Romo, (1997) Cuevas, A. & Romo, J. (1997).  Differentiable functionals and smoothed bootstrap.  Annals of the Institute of Statistical Mathematics, 49(2), 355–370.
  * Dabrowska, (1989) Dabrowska, D. M. (1989).  Uniform consistency of the kernel conditional kaplan-meier estimate.  The Annals of Statistics, 17(3), 1157–1167.
  * Diciccio & Romano, (1988) Diciccio, T. J. & Romano, J. P. (1988).  A review of bootstrap confidence intervals.  Journal of the Royal Statistical Society: Series B (Methodological), 50(3), 338–354.
  * Dixon et al., (2005) Dixon, J. R., Kosorok, M. R., & Lee, B. L. (2005).  Functional inference in semiparametric models using the piggyback bootstrap.  Annals of the Institute of statistical Mathematics, 57(2), 255–277.
  * Efron, (1982) Efron, B. (1982).  The Jackknife, the Bootstrap and Other Resampling Plans.  Society for Industrial and Applied Mathematics.
  * Efron & Tibshirani, (1994) Efron, B. & Tibshirani, R. J. (1994).  An introduction to the Bootstrap.  Chapman and Hall/CRC.
  * Gaenssler & Rost, (2000) Gaenssler, P. & Rost, D. (2000).  On uniform laws of large numbers for smoothed empirical measures.  In E. Giné, D. M. Mason, & J. A. Wellner (Eds.), High Dimensional Probability II (pp. 79–87). Boston, MA: Birkhäuser Boston.
  * Gaenssler & Rost, (2003) Gaenssler, P. & Rost, D. (2003).  Smoothed empirical processes and the bootstrap.  In J. Hoffmann-Jørgensen, J. A. Wellner, & M. B. Marcus (Eds.), High Dimensional Probability III (pp. 313–320). Basel: Birkhäuser Basel.
  * Geman & Hwang, (1982) Geman, S. & Hwang, C.-R. (1982).  Nonparametric maximum likelihood estimation by the method of sieves.  The Annals of Statistics, 10(2), 401–414.
  * Geskus & Groeneboom, (1999) Geskus, R. & Groeneboom, P. (1999).  Asymptotically optimal estimation of smooth functionals for interval censoring, case $2$.  The Annals of Statistics, 27(2), 627–674.
  * Gill & Robins, (2001) Gill, R. D. & Robins, J. M. (2001).  Causal Inference for Complex Longitudinal Data: The Continuous Case.  The Annals of Statistics, 29(6), 1785–1811.
  * Giné & Nickl, (2008) Giné, E. & Nickl, R. (2008).  Uniform central limit theorems for kernel density estimators.  Probability Theory and Related Fields, 141(3), 333–387.
  * Giné & Zinn, (1990) Giné, E. & Zinn, J. (1990).  Bootstrapping general empirical measures.  The Annals of Probability, 18(2), 851–869.
  * Giné & Zinn, (1991) Giné, E. & Zinn, J. (1991).  Gaussian characterization of uniform donsker classes of functions.  The Annals of Probability, 19(2), 758–782.
  * Giné & Guillou, (2002) Giné, E. & Guillou, A. (2002).  Rates of strong uniform consistency for multivariate kernel density estimators.  Annales de l’Institut Henri Poincare (B) Probability and Statistics, 38(6), 907–921.
  * Hall, (1988) Hall, P. (1988).  Theoretical comparison of bootstrap confidence intervals.  The Annals of Statistics, 16(3), 927–953.
  * Hall, (1992) Hall, P. (1992).  The Bootstrap and Edgeworth Expansion.  Springer New York, NY.
  * Hall et al., (1989) Hall, P., DiCiccio, T. J., & Romano, J. P. (1989).  On smoothing and the bootstrap.  The Annals of Statistics, 17(2), 692–704.
  * Hall & Kang, (2001) Hall, P. & Kang, K.-H. (2001).  Bootstrapping nonparametric density estimators with empirically chosen bandwidths.  The Annals of Statistics, 29(5), 1443–1468.
  * Han et al., (2018) Han, F., Xu, S., & Zhou, W.-X. (2018).  On Gaussian comparison inequality and its application to spectral analysis of large random matrices.  Bernoulli, 24(3), 1787–1833.
  * Hardin & Hilbe, (2002) Hardin, J. W. & Hilbe, J. M. (2002).  Generalized Estimating Equations.  Chapman and Hall/CRC.
  * Holland, (1986) Holland, P. W. (1986).  Statistics and causal inference.  Journal of the American Statistical Association, 81(396), 945–960.
  * Klaassen, (1987) Klaassen, C. A. J. (1987).  Consistent estimation of the influence function of locally asymptotically linear estimators.  The Annals of Statistics, 15(4), 1548–1562.
  * Kosorok, (2008) Kosorok, M. R. (2008).  Introduction to Empirical Processes and Semiparametric Inference.  Springer New York, NY.
  * Kosorok et al., (2004) Kosorok, M. R., Lee, B. L., & Fine, J. P. (2004).  Robust inference for univariate proportional hazards frailty regression models.  The Annals of Statistics, 32(4), 1448 – 1491.
  * Liang & Zeger, (1986) Liang, K.-Y. & Zeger, S. L. (1986).  Longitudinal data analysis using generalized linear models.  Biometrika, 73(1), 13–22.
  * Ma & Kosorok, (2005) Ma, S. & Kosorok, M. R. (2005).  Robust semiparametric $m$-estimation and the weighted bootstrap.  Journal of Multivariate Analysis, 96(1), 190–217.
  * Murphy & van der Vaart, (2000) Murphy, S. A. & van der Vaart, A. W. (2000).  On profile likelihood.  Journal of the American Statistical Association, 95(450), 449–465.
  * Newey et al., (1998) Newey, W. K., Hsieh, F., & Robins, J. M. (1998).  Undersmoothing and bias corrected functional estimation.  Working papers.
  * Newey et al., (2004) Newey, W. K., Hsieh, F., & Robins, J. M. (2004).  Twicing kernels and a small bias property of semiparametric estimators.  Econometrica, 72(3), 947–962.
  * Pfanzagl, (1982) Pfanzagl, J. (1982).  Contributions to A General Asymptotic Statistical Theory.  Springer New York, NY.
  * Pfanzagl, (1990) Pfanzagl, J. (1990).  Estimation in Semiparametric Models.  Springer New York, NY.
  * Powell et al., (1989) Powell, J. L., Stock, J. H., & Stoker, T. M. (1989).  Semiparametric estimation of index coefficients.  Econometrica: Journal of the Econometric Society, 57(6), 1403–1430.
  * Praestgaard & Wellner, (1993) Praestgaard, J. & Wellner, J. A. (1993).  Exchangeably weighted bootstraps of the general empirical process.  The Annals of Probability, 21(4), 2053 – 2086.
  * Quale et al., (2006) Quale, C. M., van der Laan, M. J., & Robins, J. R. (2006).  Locally efficient estimation with bivariate right-censored data.  Journal of the American Statistical Association, 101(475), 1076–1084.
  * Radulović & Wegkamp, (2000) Radulović, D. & Wegkamp, M. (2000).  Weak convergence of smoothed empirical processes: Beyond donsker classes.  In E. Giné, D. M. Mason, & J. A. Wellner (Eds.), High Dimensional Probability II (pp. 89–105). Boston, MA: Birkhäuser Boston.
  * Radulović & Wegkamp, (2009) Radulović, D. & Wegkamp, M. (2009).  Uniform central limit theorems for pregaussian classes of functions.  In C. Houdré, V. Koltchinskii, D. M. Mason, & M. Peligrad (Eds.), High Dimensional Probability V: The Luminy Volume (pp. 84–102).
  * Robins, (1986) Robins, J. (1986).  A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect.  Mathematical Modelling, 7(9), 1393–1512.
  * Rost, (2000) Rost, D. (2000).  Limit theorems for smoothed empirical processes.  In E. Giné, D. M. Mason, & J. A. Wellner (Eds.), High Dimensional Probability II (pp. 107–113). Boston, MA: Birkhäuser Boston.
  * Sheather & Jones, (1991) Sheather, S. J. & Jones, M. C. (1991).  A Reliable Data-Based Bandwidth Selection Method for Kernel Density Estimation.  Journal of the Royal Statistical Society: Series B (Methodological), 53(3), 683–690.
  * Sheehy & Wellner, (1992) Sheehy, A. & Wellner, J. A. (1992).  Uniform donsker classes of functions.  The Annals of Probability, 20(4), 1983–2030.
  * Shen, (1997) Shen, X. (1997).  On methods of sieves and penalization.  The Annals of Statistics, 25(6), 2555–2591.
  * Silverman, (1986) Silverman, B. W. (1986).  Density Estimation for Statistics and Data Analysis.  London: Chapman and Hall.
  * van der Laan & Robins, (2003) van der Laan, M. J. & Robins, J. M. (2003).  Unified Methods for Censored Longitudinal Data and Causality.  Springer New York, NY.
  * van der Laan & Rose, (2011) van der Laan, M. J. & Rose, S. (2011).  Targeted Learning: Causal Inference for Observational and Experimental Data.  Springer New York, NY.
  * van der Laan & Rubin, (2006) van der Laan, M. J. & Rubin, D. (2006).  Targeted maximum likelihood learning.  The International Journal of Biostatistics, 2(1).
  * van der Vaart, (1991) van der Vaart, A. W. (1991).  On differentiable functionals.  The Annals of Statistics, 19(1), 178–204.
  * van der Vaart, (1994) van der Vaart, A. W. (1994).  Weak convergence of smoothed empirical processes.  Scandinavian Journal of Statistics, 21(4), 501–504.
  * van der Vaart, (2000) van der Vaart, A. W. (2000).  Asymptotic Statistics, volume 3.  Cambridge University Press.
  * van der Vaart & Wellner, (1996) van der Vaart, A. W. & Wellner, J. A. (1996).  Weak Convergence and Empirical Processes: With Applications to Statistics.  Springer New York, NY.
  * Wellner & Zhan, (1996) Wellner, J. A. & Zhan, Y. (1996).  Bootstrapping $z$-estimators.  University of Washington Department of Statistics Technical Report, 308(5).
  * Yukich, (1992) Yukich, J. E. (1992).  Weak convergence of smoothed empirical processes.  Scandinavian Journal of Statistics, 19(3), 271–279.

Supplementary Material

##  Supplement A Proof of results in Section 2

See 3.1

###### Proof of Theorem 3.1.

Consider the decomposition

$\displaystyle
T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})=\mathbb{P}_{n}\phi_{0}$
$\displaystyle+(\mathbb{P}_{n}-P_{0})(\phi_{n}-\phi_{0})+[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})-(\mathbb{P}_{n}-P_{0})\phi_{n}].$

By conditions (A1)–(A2) and Lemma 19.24 of van der Vaart, (2000),
$(\mathbb{P}_{n}-P_{0})(\phi_{n}-\phi_{0})=o_{P_{0}^{*}}(n^{-1/2})$. By
condition (A3), the remainder term is $o_{P_{0}^{*}}(n^{-1/2})$. The second
statement follows by the central limit theorem since $P_{0}\phi_{0}=0$ and
$P_{0}\phi_{0}^{2}<\infty$ by assumption. $\hfill\blacksquare$

See 3.2

###### Proof of Proposition 3.2.

By adding and subtracting terms, we have

$\mathbb{P}_{n}\phi_{n}^{2}-\sigma_{0}^{2}=(\mathbb{P}_{n}-P_{0})\phi_{n}^{2}+P_{0}(\phi_{n}^{2}-\phi_{0}^{2}).$

The second term is $o_{P_{0}^{*}}(1)$ by assumption. For the first term, for
any $\varepsilon>0$, we have

$\displaystyle
P_{0}^{*}\left(\left|{(\mathbb{P}_{n}-P_{0})\phi_{n}^{2}}\right|\geq\varepsilon\right)$
$\displaystyle=P_{0}^{*}\left(\left|{(\mathbb{P}_{n}-P_{0})\phi_{n}^{2}}\right|\geq\epsilon,\,\phi_{n}^{2}\in\mathscr{G}\right)+P_{0}^{*}\left(\left|{(\mathbb{P}_{n}-P_{0})\phi_{n}^{2}}\right|\geq\varepsilon,\,\phi_{n}^{2}\notin\mathscr{G}\right)$
$\displaystyle\leq
P_{0}^{*}\left(\sup_{g\in\mathscr{G}}\left|{(\mathbb{P}_{n}-P_{0})g}\right|\geq\varepsilon\right)+P_{0}^{*}\left(\phi_{n}^{2}\notin\mathscr{G}\right).$

Since $\mathscr{G}$ is a $P_{0}$-Glivenko Cantelli class, the first term is
$o(1)$, and since $\phi_{n}^{2}\in\mathscr{G}$ with probability tending to one
the second term is $o(1)$. Hence,
$(\mathbb{P}_{n}-P_{0})\phi_{n}^{2}=o_{P_{0}^{*}}(1)$, which completes the
proof of the first statement.

By Theorem 2.10.14 of van der Vaart & Wellner, (1996), since
$\sup_{f\in\mathscr{F}}|P_{0}f|<\infty$, condition (A1) implies that
$\mathscr{F}^{2}=\\{f^{2}:f\in\mathscr{F}\\}$ is $P_{0}$-Glivenko-Cantelli in
probability, and since $P_{0}^{*}(\phi_{n}^{2}\in\mathscr{F}^{2})\to 1$, this
implies (i). By Cauchy-Schwarz and Minkowski’s inequalities, we also have

$\displaystyle\left|P_{0}(\phi_{n}^{2}-\phi_{0}^{2})\right|$
$\displaystyle\leq P_{0}|\phi_{n}^{2}-\phi_{0}^{2}|$
$\displaystyle=P_{0}\left|(\phi_{n}-\phi_{0})(\phi_{n}+\phi_{0})\right|$
$\displaystyle\leq\left\|{\phi_{n}-\phi_{0}}\right\|_{L_{2}(P_{0})}\left\|{\phi_{n}+\phi_{0}}\right\|_{L_{2}(P_{0})}$
$\displaystyle\leq\left\|{\phi_{n}-\phi_{0}}\right\|_{L_{2}(P_{0})}[\left\|{\phi_{n}-\phi_{0}}\right\|_{L_{2}(P_{0})}+2\left\|{\phi_{0}}\right\|_{L_{2}(P_{0})}],$

which is $o_{P_{0}^{*}}(1)$ by (A2). $\hfill\blacksquare$

##  Supplement B Proof of results in Section 3

See 3.5

###### Proof of Lemma 3.5.

A standard way to prove results of this type outside the setting of the
bootstrap is the continuous mapping theorem (see, e.g., van der Vaart,, 2000,
Theorem 19.24). However, the continuous mapping theorem might not be
applicable in the bootstrap setting as the map $(w_{1},\dotsc,w_{n})\mapsto
h(\mathbb{G}_{n}^{*}(x_{1},\dotsc,x_{n},w_{1},\dotsc,w_{n}))$ might not be
measurable given almost $x_{1},\dotsc,x_{n}$ for all $h\in
C_{b}(\ell^{\infty}(\mathscr{F}))$.

For any $\tau>0$, we define
$\mathscr{F}_{\tau}=\\{f-f_{\infty}:f\in\mathscr{F},\rho(f,f_{\infty})<\tau\\}$.
We then have

$\displaystyle\left\\{|\mathbb{G}_{n}^{*}(f_{n}^{*}-f_{\infty})|\geq\varepsilon\right\\}$
$\displaystyle\subseteq\left\\{\|\mathbb{G}_{n}^{*}(f_{n}^{*}-f_{\infty})|\geq\varepsilon,\
\rho(f_{n}^{*},f_{\infty})<\tau,\ f_{n}^{*}\in\mathscr{F}\right\\}$
$\displaystyle\qquad\qquad\cup\left\\{\rho(f_{n}^{*},f_{\infty})\geq\tau\
\text{or}\ f_{n}^{*}\notin\mathscr{F}\right\\}$
$\displaystyle\subseteq\left\\{\|\mathbb{G}_{n}^{*}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right\\}\cup\left\\{\rho(f_{n}^{*},f_{\infty})\geq\tau\right\\}\cup\left\\{f_{n}^{*}\notin\mathscr{F}\right\\}$

for any $\varepsilon,\tau>0$. Thus,

$\displaystyle
P_{W}^{*}\left(|\mathbb{G}_{n}^{*}(f_{n}^{*}-f_{\infty})|\geq\varepsilon\right)$
$\displaystyle\leq
P^{*}_{W}\left(\|\mathbb{G}_{n}^{*}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right)+P^{*}_{W}\left(\rho(f_{n}^{*},f_{\infty})\geq\tau\right)+P^{*}_{W}\left(f_{n}^{*}\notin\mathscr{F}\right).$

The second and third terms on the right-hand side conditionally converge to
zero in outer probability for any $\tau>0$ by assumption. Hence, the proof is
complete if we can show that for all $\varepsilon,\delta,\gamma>0$ there
exists $\tau>0$ such that
$P_{0}^{*}\left(P^{*}_{W}\left(\|\mathbb{G}_{n}^{*}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right)>\delta\right)<\gamma$
for all $n$ large enough.

For each $\varepsilon,\tau>0$ there exists a sequence of functions
$h_{m}:\ell^{\infty}(\mathscr{F})\to\mathbb{R}$ such that $h_{m}$ is
$m$-Lipschitz for all $m$,
$1\\{\|z\|_{\mathscr{F}_{\tau}}\geq\varepsilon\\}\leq h_{m}(z)\leq 1$ for all
$z\in\ell^{\infty}(\mathscr{F})$ and $m$, and $h_{m}(z)$ monotonically
decreases to $1\\{\|z\|_{\mathscr{F}_{\tau}}\geq\varepsilon\\}$ as
$m\to\infty$ for each $z\in\ell^{\infty}(\mathscr{F})$. For instance,
$h_{m}:z\mapsto\min\\{\max(m[\|z\|_{\mathscr{F}_{\tau}}-\varepsilon]+1,0),1\\}$
satisfies these criteria. For any $\varepsilon,\tau>0$ and
$m\in\\{1,2,\dots\\}$, we can now write

$\displaystyle
P^{*}_{W}\left(\|\mathbb{G}_{n}^{*}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right)$
$\displaystyle=\left[E^{*}_{W}1\left\\{\|\mathbb{G}_{n}^{*}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right\\}-E^{*}_{W}h_{m}(\mathbb{G}_{n}^{*})\right]+\left[E^{*}_{W}h_{m}(\mathbb{G}_{n}^{*})-E_{0}h_{m}(\mathbb{G}_{0})\right]$
$\displaystyle\qquad+\left[E_{0}h_{m}(\mathbb{G}_{0})-E_{0}1\left\\{\|\mathbb{G}_{0}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right\\}\right]+P_{0}\left(\left\|{\mathbb{G}_{0}}\right\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right)$
$\displaystyle\leq\left[E^{*}_{W}h_{m}(\mathbb{G}_{n}^{*})-E_{0}h_{m}(\mathbb{G}_{0})\right]+\left[E_{0}h_{m}(\mathbb{G}_{0})-E_{0}1\left\\{\|\mathbb{G}_{0}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right\\}\right]$
$\displaystyle\qquad+P_{0}\left(\left\|{\mathbb{G}_{0}}\right\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right),$

where the second inequality follows because
$E^{*}_{W}1\left\\{\|\mathbb{G}_{n}^{*}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right\\}\leq
E^{*}_{W}h_{m}\left(\mathbb{G}_{n}^{*}\right)$ for all $\varepsilon,\tau>0$
and $m\in\\{1,2,\dots\\}$ by assumption. For the final term on the right-hand
side, since almost all sample paths of $\mathbb{G}_{0}$ are uniformly
continuous in $\ell^{\infty}(\mathscr{F})$ with respect to $\rho$, for any
$\varepsilon,\delta>0$, we can choose $\tau>0$ such that
$P_{0}\left(\left\|{\mathbb{G}_{0}}\right\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right)<\delta$.
For the second term on the right-hand side, by the monotone convergence
theorem,
$\lim_{m\to\infty}E_{0}h_{m}(\mathbb{G}_{0})=E_{0}1\\{\|\mathbb{G}_{0}\|_{\mathscr{F_{\tau}}}\geq\varepsilon\\}$.
Hence, for any $\varepsilon,\delta,\tau>0$, we can choose $m$ such that
$E_{0}h_{m}(\mathbb{G}_{0})-E_{0}1\left\\{\|\mathbb{G}_{0}\|_{\mathscr{F}_{\tau}}\geq\varepsilon\right\\}<\delta$.
Finally, for the first term on the right-hand side, since $h_{m}$ is bounded
and $m$-Lipschitz, $h_{m}/m\in\mathrm{BL}_{1}(\ell^{\infty}(\mathscr{F}))$ for
each $m$. Therefore, by the assumed conditional weak convergence of
$\mathbb{G}_{n}^{*}$ to $\mathbb{G}_{0}$ in $\ell^{\infty}(\mathscr{F})$,
$E^{*}_{W}\frac{1}{m}h_{m}\left(\mathbb{G}_{n}^{*}\right)\xrightarrow{P_{0}^{*}}E_{0}\frac{1}{m}h_{m}(\mathbb{G}_{0})$
for each $m$, so that
$E^{*}_{W}h_{m}\left(\mathbb{G}_{n}^{*}\right)\xrightarrow{P_{0}^{*}}E_{0}h_{m}(\mathbb{G}_{0})$
as well. Hence, for any $\varepsilon,\delta,\gamma,\tau>0$ and
$m\in\\{1,2,\dots\\}$,
$P_{0}^{*}\left(E^{*}_{W}h_{m}\left(\mathbb{G}_{n}^{*}\right)-E_{0}h_{m}(\mathbb{G}_{0})>\delta\right)<\gamma$
for all $n$ large enough. This completes the proof.

$\hfill\blacksquare$

See 3.4

###### Proof of Theorem 3.4.

We let $h$ be an arbitrary element of $\mathrm{BL}_{1}(\mathbb{R})$. By adding
and subtracting terms and the triangle inequality, we have

$\displaystyle\begin{split}&\left|{E_{W}^{*}h\left(n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})\right]\right)-E_{0}h\left(\mathbb{G}_{0}\phi_{0}\right)}\right|\\\
&\qquad\leq
E_{W}^{*}\left|{h\left(n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})\right]\right)-h\left(\mathbb{G}_{n}^{*}\phi_{0}\right)}\right|+\left|{E_{W}^{*}h\left(\mathbb{G}_{n}^{*}\phi_{0}\right)-E_{0}h\left(\mathbb{G}_{0}\phi_{0}\right)}\right|.\end{split}$
(15)

For the first term on the right-hand side of (15), we note that for any
$x_{1},x_{2}\in\mathbb{R}$, $\left|{h(x_{1})-h(x_{2})}\right|\leq
2\wedge|x_{1}-x_{2}|$, and for any $\varepsilon>0$,
$2\wedge|x_{1}-x_{2}|\leq\varepsilon+2I(|x_{1}-x_{2}|>\varepsilon)$, which
implies that for any $\varepsilon>0$,

$\displaystyle\begin{split}&E_{W}^{*}\left|{h\left(n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})\right]\right)-h\left(\mathbb{G}_{n}^{*}\phi_{0}\right)}\right|\\\
&\qquad\leq\varepsilon+2P^{*}_{W}\left(\left|n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})\right]-\mathbb{G}_{n}^{*}\phi_{0}\right|\geq\varepsilon\right).\end{split}$
(16)

We now write

$\displaystyle
n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})\right]-\mathbb{G}_{n}^{*}\phi_{0}$
$\displaystyle=\mathbb{G}_{n}^{*}(\phi_{n}^{*}-\phi_{0})+n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*}\right]$
$\displaystyle=\mathbb{G}_{n}^{*}(\phi_{n}^{*}-\phi_{0})+n^{1/2}R_{n}^{*}.$

Note that $\mathbb{G}_{n}^{*}(\phi_{n}^{*}-\phi_{0})=o_{P_{W}^{*}}(1)$ by
conditions (B1)–(B2) and Lemma 3.5, and $n^{1/2}R_{n}^{*}=o_{P_{W}^{*}}(1)$ by
condition (B3). This implies the first term on the right-hand side of (15) is
$o_{P_{0}^{*}}(1)$. For the second term on the right hand side of (15), we
define the function $g:\ell^{\infty}(\mathscr{F})\to\mathbb{R}$ as
$g(z):=h(z(\phi_{0}))$. For any $z_{1},z_{2}\in\ell^{\infty}(\mathscr{F})$
with $\|z_{1}-z_{2}\|_{\mathscr{F}}>0$, we have

$\frac{|g(z_{1})-g(z_{2})|}{\|z_{1}-z_{2}\|_{\mathscr{F}}}=\frac{|h(z_{1}(\phi_{0}))-h(z_{2}(\phi_{0}))|}{\|z_{1}-z_{2}\|_{\mathscr{F}}}\leq\frac{|z_{1}(\phi_{0})-z_{2}(\phi_{0})|}{\|z_{1}-z_{2}\|_{\mathscr{F}}}\leq
1$

because $h\in\mathrm{BL}_{1}(\mathbb{R})$ and $\phi_{0}\in\mathscr{F}$. Hence,
$g\in\mathrm{BL}_{1}(\ell^{\infty}(\mathscr{F}))$. Therefore,

$\left|{E_{W}^{*}h\left(\mathbb{G}_{n}^{*}\phi_{0}\right)-E_{0}h\left(\mathbb{G}_{0}\phi_{0}\right)}\right|\leq\sup_{g\in\mathrm{BL}_{1}(\ell^{\infty}(\mathscr{F}))}\left|{E_{W}^{*}g\left(\mathbb{G}_{n}^{*}\right)-E_{0}g\left(\mathbb{G}_{0}\right)}\right|\xrightarrow{P_{0}^{*}}0$

by condition (B1). This implies conditional asymptotic linearity of
$T(\eta_{n}^{*},\mathbb{P}_{n}^{*})$. Conditional weak convergence of
$n^{1/2}\left[T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})\right]$
is then implied by (16). $\hfill\blacksquare$

See 3.7

###### Proof of Lemma 3.7.

We argue along subsequences using a standard argument structure. By Lemma
1.9.2 of van der Vaart & Wellner, (1996), every subsequence
$\\{n_{k}:k=1,2,\dots\\}$ has a further subsequence
$\\{n_{k_{l}}:l=1,2,\dots\\}$ such

$\sup_{f,g\in\mathscr{F}}\left|\|f-g\|_{L_{2}(\hat{P}_{n_{k_{l}}})}-\|f-g\|_{L_{2}(P_{0})}\right|\stackrel{{\scriptstyle\text{a.s.*}}}{{\longrightarrow}}0.$

By Theorem 4.5 of Sheehy & Wellner, (1992), we then have
$n_{k_{l}}^{1/2}(\mathbb{P}_{n_{k_{l}}}^{*}-\hat{P}_{n_{k_{l}}})\underset{W}{\stackrel{{\scriptstyle\text{a.s.*}}}{{\leadsto}}}\mathbb{G}_{0}$.
Hence, every $\\{n_{k}\\}$ has a further subsequence $\\{n_{k_{l}}\\}$ such
that

$\sup_{h\in\mathrm{BL}_{1}(\ell^{\infty}(\mathscr{F}))}\left|{E_{W}^{*}h(\mathbb{G}_{n_{k_{l}}}^{*})-E_{0}h(\mathbb{G}_{0})}\right|\stackrel{{\scriptstyle\text{a.s.*}}}{{\longrightarrow}}0,$

which by Lemma 1.9.2 of van der Vaart & Wellner, (1996) again implies that

$\sup_{h\in\mathrm{BL}_{1}(\ell^{\infty}(\mathscr{F}))}\left|{E_{W}^{*}h(\mathbb{G}_{n}^{*})-E_{0}h(\mathbb{G}_{0})}\right|\xrightarrow{P_{0}^{*}}0,$

i.e.,
$\mathbb{G}_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}$.

$\hfill\blacksquare$

See 3.8

###### Proof of Proposition 3.8.

Since $\mathscr{F}$ has uniformly bounded envelope function and finite uniform
entropy integral, by Theorem 2.8.3 of van der Vaart & Wellner, (1996),
conditions (i)–(ii) of Lemma 3.7 are satisfied. We next show the uniform
convergence of semi-metric. For simplicity, we denote
$\mathscr{F}_{\infty}:=\\{f-g:f,g\in\mathscr{F}\\}$ and
$\mathscr{F}_{\infty}^{2}:=\\{(f-g)^{2}:f,g\in\mathscr{F}\\}$. Since
$|\sqrt{x}-\sqrt{y}|\leq\sqrt{|x-y|}$ for all $x,y\in\mathbb{R}$, we have

$\displaystyle\sup_{f,g\in\mathscr{F}}\left|{\left\|{f-g}\right\|_{L_{2}(\hat{P}_{n})}-\left\|{f-g}\right\|_{L_{2}(P_{0})}}\right|\leq\sup_{f,g\in\mathscr{F}}\left|{\int(f-g)^{2}\,\mathrm{d}(\hat{P}_{n}-P_{0})}\right|^{1/2}$

Hence, condition (iii) of Lemma 3.7 is satisfied if
$\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}_{\infty}^{2}}=o_{P_{0}^{*}}(1)$.

We show that
$P_{0}^{*}(\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}_{\infty}^{2}}\geq\delta)\to 0$
for all $\delta>0$ using Theorem 2.2 of Gaenssler & Rost, (2000). To do so, we
verify their conditions (2.2)–(2.5) with $\mu_{n}=L_{n}$ and
$\nu_{n}=\mathbb{P}_{n}$. Since $\mathscr{F}_{\infty}^{2}$ is uniformly
bounded by the uniform boundedness of $\mathscr{F}$, conditions (2.2) and
(2.3) are satisfied, as discussed following Theorem 2.2 of Gaenssler & Rost,
(2000). To show condition (2.4), we firstly note that following inequality
holds for all $\varepsilon>0$ and measures $Q$ on $(\mathscr{X},\mathscr{B})$,

$N(4\varepsilon,\mathscr{F}_{\infty}^{2},L_{1}(Q))\leq
N(4\varepsilon,\mathscr{F}_{\infty}^{2},L_{2}(Q))\leq
N(2\varepsilon,\mathscr{F}_{\infty},L_{2}(Q))\leq
N(\varepsilon,\mathscr{F},L_{2}(Q))^{2}.$

The first inequality is because
$\left\|{\,\cdot\,}\right\|_{L_{1}(Q)}\leq\left\|{\,\cdot\,}\right\|_{L_{2}(Q)}$
by Hölder’s inequality. The second inequality is because for any
$f_{\infty},g_{\infty}\in\mathscr{F}_{\infty}$, we have
$Q(f_{\infty}^{2}-g_{\infty}^{2})^{2}=Q(f_{\infty}-g_{\infty})^{2}(f_{\infty}+g_{\infty})^{2}\leq
4Q(f_{\infty}-g_{\infty})^{2}$ by uniform boundedness of $\mathscr{F}$. The
third inequality is because for any $f_{1},g_{1},f_{2},g_{2}\in\mathscr{F}$,
we have $Q((f_{1}-g_{1})-(f_{2}-g_{2}))^{2}\leq
Q(f_{1}-f_{2})^{2}+Q(g_{1}-g_{2})^{2}+2\|f_{1}-f_{2}\|_{L_{2}(Q)}\|g_{1}-g_{2}\|_{L_{2}(Q)}$.
Since $\mathscr{F}$ has uniformly integrable $L_{2}$ entropy, we have
$\int_{0}^{\infty}\sup_{Q}\left\\{\log
N(\varepsilon,\mathscr{F},L_{2}(Q))\right\\}^{1/2}\,\mathrm{d}\varepsilon<\infty,$
where the supremum is taken over all finitely discrete probability measures
$Q$ on $(\mathscr{X},\mathscr{B})$. By exercise 1 in Chapter 2.5 of van der
Vaart & Wellner, (1996), the supremum over finitely discrete $Q$ can be
replaced by the supremum over all probability measures $Q$ such that
$0<QF^{2}<\infty$ without changing the assumption of uniformly integrable
$L_{2}$ entropy of $\mathscr{F}$. So, we have

$\int_{0}^{\infty}\sup_{Q}\left\\{\log
N(\varepsilon,\mathscr{F}_{\infty}^{2},L_{1}(Q))\right\\}^{1/2}\,\mathrm{d}\varepsilon<\infty,$
(17)

where the supremum is taken over all probability measures $Q$ on
$(\mathscr{X},\mathscr{B})$. Since
$N(\varepsilon,\mathscr{F}_{\infty}^{2},L_{1}(\hat{P}_{n}))\leq\sup_{Q}N(\varepsilon,\mathscr{F}_{\infty}^{2},L_{1}(Q))$
almost surely, and the latter is finite for $\varepsilon>0$ by (17),
$N(\varepsilon,\mathscr{F}_{\infty}^{2},L_{1}(\hat{P}_{n}))$ is stochastically
bounded for all $\varepsilon>0$. Furthermore, since
$\|\cdot\|_{d_{\tilde{\nu}_{n}}^{(1)}}\leq\|\cdot\|_{L_{1}(\hat{P}_{n})}$ for
$d_{\tilde{\nu}_{n}}^{(1)}$ defined in Gaenssler & Rost, (2000), this further
implies that condition (2.4) of Theorem 2.2 of Gaenssler & Rost, (2000) holds.

Lastly, to show condition (2.5) in Theorem 2.2 of Gaenssler & Rost, (2000)
holds, we show that

$\displaystyle\lim_{n\to\infty}\sup_{f_{\infty}\in\mathscr{F}_{\infty}}\left|{P_{0}*L_{n}(f_{\infty}^{2})-P_{0}(f_{\infty}^{2})}\right|$
$\displaystyle\qquad=\lim_{n\to\infty}\sup_{f_{\infty}\in\mathscr{F}_{\infty}}\left|{\iint
f_{\infty}(x+y)^{2}\,\mathrm{d}P_{0}(x)\,\mathrm{d}L_{n}(y)-\iint
f_{\infty}(x+y)^{2}\,\mathrm{d}P_{0}(x)\,\mathrm{d}\delta_{0}(y)}\right|=0.$

The function $y\mapsto\int f_{\infty}^{2}(x+y)\,\mathrm{d}P_{0}(x)$ is
uniformly bounded by 1 since $\mathscr{F}_{\infty}^{2}$ is uniformly bounded
by 1. Furthermore, for any $y_{1},y_{2}\in\mathscr{X}$ and
$f_{\infty}\in\mathscr{F}_{\infty}$, using the assumption that $p_{0}$ is
Lipschitz, we have

$\displaystyle\left|{\int f_{\infty}(x+y_{1})^{2}\,\mathrm{d}P_{0}(x)-\int
f_{\infty}(x+y_{2})^{2}\,\mathrm{d}P_{0}(x)}\right|$
$\displaystyle\qquad=\left|{\int
f_{\infty}(x+y_{1})^{2}p_{0}(x)\,\mathrm{d}\lambda(x)-\int
f_{\infty}(x+y_{2})^{2}p_{0}(x)\,\mathrm{d}\lambda(x)}\right|$
$\displaystyle\qquad=\left|{\int
f_{\infty}(z)^{2}[p_{0}(z-y_{1})-p_{0}(z-y_{2})]\,\mathrm{d}\lambda(z)}\right|$
$\displaystyle\qquad\leq\gamma\left|{y_{1}-y_{2}}\right|\int
f_{\infty}^{2}\,\mathrm{d}\lambda$ $\displaystyle\qquad\leq
4\gamma\left|{y_{1}-y_{2}}\right|\int F^{2}\,\mathrm{d}\lambda.$

for some $\gamma>0$. Since $\int F^{2}\,\mathrm{d}\lambda<\infty$ by
assumption, this implies $y\mapsto\int
f_{\infty}^{2}(x+y)\,\mathrm{d}P_{0}(x)$ is uniformly bounded by 1 and
$\gamma^{\prime}$-Lipschitz for $\gamma^{\prime}:=4\gamma\int
F^{2}\,\mathrm{d}\lambda$ for all $f_{\infty}\in\mathscr{F}_{\infty}$. Thus,
the function $\min(\gamma^{\prime-1},1)\times\int
f_{\infty}^{2}(x+\cdot)\,\mathrm{d}P_{0}(x)\in\mathrm{BL}_{1}(\mathbb{R})$,
which implies that

$\displaystyle\sup_{f_{\infty}\in\mathscr{F}_{\infty}}\left|{\iint
f_{\infty}(x+y)^{2}\,\mathrm{d}P_{0}(x)\,\mathrm{d}L_{n}(y)-\iint
f_{\infty}(x+y)^{2}\,\mathrm{d}P_{0}(x)\,\mathrm{d}\delta_{0}(y)}\right|$
$\displaystyle\qquad\leq\frac{1}{\min(\gamma^{\prime-1},1)}\sup_{h\in\mathrm{BL}_{1}(\mathscr{X})}\left|{\int
h(y)\,\mathrm{d}L_{n}(y)-\int h(y)\,\mathrm{d}\delta_{0}(y)}\right|,$

which goes to $0$ as $n\to\infty$ because $L_{n}$ converges weakly to
$\delta_{0}$ by assumption. We have now checked all the conditions of Theorem
2.2 of Gaenssler & Rost,, 2000, so we conclude that
$E_{0}^{*}\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}_{\infty}^{2}}^{p}\to 0$ for all
$p\geq 1$. This demonstrates that condition (iii) of Lemma 3.7 holds, so the
result follows.

$\hfill\blacksquare$

See 3.11

###### Proof of Lemma 3.11.

By adding and subtracting terms,

$\mathbb{P}_{n}^{*}\phi_{n}^{*2}-\sigma_{0}^{2}=(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*2}+(\hat{P}_{n}-P_{0})\phi_{n}^{*2}+P_{0}(\phi_{n}^{*2}-\phi_{0}^{2}).$
(18)

For the first term on the right-hand side of (18), for any $\varepsilon>0$, we
have

$\displaystyle
P_{W}^{*}\left(\left|{(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*2}}\right|\geq\varepsilon\right)$
$\displaystyle=P_{W}^{*}\left(\left|{(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*2}}\right|\geq\varepsilon,\,\phi_{n}^{*2}\in\mathscr{G}\right)$
$\displaystyle\qquad+P_{W}^{*}\left(\left|{(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*2}}\right|\geq\varepsilon,\,\phi_{n}^{*2}\notin\mathscr{G}\right)$
$\displaystyle\leq
P_{W}^{*}\left(\sup_{g\in\mathscr{G}}\left|{(\mathbb{P}_{n}^{*}-\hat{P}_{n})g}\right|\geq\varepsilon\right)+P_{W}^{*}\left(\phi_{n}^{*2}\notin\mathscr{G}\right)$
$\displaystyle=o_{P_{0}^{*}}(1).$

For the second term on the right-hand side of (18), for any $\varepsilon>0$,
we have

$\displaystyle
P_{W}^{*}\left(\left|{(\hat{P}_{n}-P_{0})\phi_{n}^{*2}}\right|\geq\varepsilon\right)$
$\displaystyle=P_{W}^{*}\left(\left|{(\hat{P}_{n}-P_{0})\phi_{n}^{*2}}\right|\geq\varepsilon,\,\phi_{n}^{*2}\in\mathscr{G}\right)$
$\displaystyle\qquad+P_{W}^{*}\left(\left|{(\hat{P}_{n}-P_{0})\phi_{n}^{*2}}\right|\geq\varepsilon,\,\phi_{n}^{*2}\notin\mathscr{G}\right)$
$\displaystyle\leq
P_{W}^{*}\left(\sup_{g\in\mathscr{G}}\left|{(\hat{P}_{n}-P_{0})g}\right|\geq\varepsilon\right)+P_{W}^{*}\left(\phi_{n}^{*2}\notin\mathscr{G}\right)$
$\displaystyle=o_{P_{0}^{*}}(1).$

Therefore, $\mathbb{P}_{n}^{*}\phi_{n}^{*}-\sigma_{0}^{2}=o_{P_{W}^{*}}(1)$ by
assumption. Furthermore, $P_{0}(\phi_{n}^{*2}-\phi_{0}^{2})=o_{P_{W}^{*}}(1)$
is implied by condition (B2). This is because by Cauchy-Schwarz and
Minkowski’s inequalities,

$\displaystyle\left|{P_{0}(\phi_{n}^{*2}-\phi_{0}^{2})}\right|$
$\displaystyle\leq P_{0}|\phi_{n}^{*2}-\phi_{0}^{2}|$
$\displaystyle=P_{0}\left|{(\phi_{n}^{*}-\phi_{0})(\phi_{n}^{*}+\phi_{0})}\right|$
$\displaystyle\leq\left\|{\phi_{n}^{*}-\phi_{0}}\right\|_{L_{2}(P_{0})}\left\|{\phi_{n}^{*}+\phi_{0}}\right\|_{L_{2}(P_{0})}$
$\displaystyle\leq\left\|{\phi_{n}^{*}-\phi_{0}}\right\|_{L_{2}(P_{0})}[\left\|{\phi_{n}^{*}-\phi_{0}}\right\|_{L_{2}(P_{0})}+2\left\|{\phi_{0}}\right\|_{L_{2}(P_{0})}]$
$\displaystyle=o_{P_{W}^{*}}(1).$

Specifically, for the empirical bootstrap $\hat{P}_{n}=\mathbb{P}_{n}$,
condition (B1)(b) implies $\mathscr{F}$ is $P_{0}$-Donsker by Theorem 3.6.1 of
van der Vaart & Wellner, (1996). Hence,
$\mathscr{G}=\\{f^{2}:f\in\mathscr{F}\\}$ is $P_{0}$-Glivenko-Cantelli in
probability by Lemma 2.10.14 of van der Vaart, (1991), which implies (ii) and
(iii) by Theorem 2.6 of Giné & Zinn, (1990). Meanwhile, (i) is implied by
condition (B1)(a), where $\mathscr{G}=\\{f^{2}:f\in\mathscr{F}\\}$.
$\hfill\blacksquare$

######  Lemma B.1 (Bootstrap Slutsky’s Theorem).

Let $X_{n}^{*}$ and $Y_{n}^{*}$ be two sequences of real-valued random
variables defined on the product probability space
$(\mathscr{X},\mathscr{B},P_{0})^{n}\times(\mathscr{W}_{n},\mathscr{C}_{n},Q_{n})^{n}$.
Assume that $X_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}X$ and
$Y_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}c$ in $\mathbb{R}$, where $X$
is a tight, Borel measurable limit and $c$ is a constant. We then have (i)
$Y_{n}^{*}X_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}cX$ in
$\mathbb{R}$, (ii)
$(Y_{n}^{*})^{-1}X_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}c^{-1}X$
in $\mathbb{R}$ provided $c\neq 0$, and (iii)
$X_{n}^{*}+Y_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}X+Y$ in
$\mathbb{R}$.

###### Proof of Lemma B.1.

We only show (i); a similar argument can be applied to yield (ii) and (iii).
As in the proof of Lemma 3.5, a standard way to prove results of this type
outside the setting of the bootstrap is the continuous mapping theorem (e.g.,
Lemma 2.8 of van der Vaart,, 2000). However, the continuous mapping theorem
might not be applicable in the bootstrap setting as the map
$(w_{1},\dotsc,w_{n})\mapsto
h(\mathbb{G}_{n}^{*}(x_{1},\dotsc,x_{n},w_{1},\dotsc,w_{n}))$ might not be
measurable given almost every $x_{1},\dotsc,x_{n}$ for all $h\in
C_{b}(\ell^{\infty}(\mathscr{F}))$.

We let $h$ be an arbitrary element of $\mathrm{BL}_{1}(\mathbb{R})$. By adding
and subtracting terms and the triangle inequality, for any $\varepsilon>0$,

$\displaystyle|E_{W}^{*}h(X_{n}^{*}Y_{n}^{*})-E_{0}h(Xc)|$
$\displaystyle\leq|E_{W}^{*}h(X_{n}^{*}Y_{n}^{*})-E_{W}^{*}h(X_{n}^{*}c)|+|E_{W}^{*}h(X_{n}^{*}c)-E_{0}h(Xc)|$
$\displaystyle\leq\varepsilon+2P_{W}^{*}(|X_{n}^{*}Y_{n}^{*}-X_{n}^{*}c|\geq\varepsilon)+|E_{W}^{*}h(X_{n}^{*}c)-E_{0}h(Xc)|,$
(19)

where the second inequality is because $\left|{h(x_{1})-h(x_{2})}\right|\leq
2\wedge|x_{1}-x_{2}|\leq\varepsilon+2I(|x_{1}-x_{2}|>\varepsilon)$ for any
$x_{1},x_{2}\in\mathbb{R}$. By defining $g:\mathbb{R}\mapsto\mathbb{R}$ as
$g(x)=cx$, we note that $\min(|c|{-1},1)\times(h\circ
g)\in\mathrm{BL}_{1}(\mathbb{R})$ because for any $x_{1},x_{2}\in\mathbb{R}$
and $x_{1}\neq x_{2}$,

$\displaystyle\frac{|\min(|c|^{-1},1)\times(h\circ
g)(x_{1})-\min(|c|^{-1},1)\times(h\circ g)(x_{2})|}{|x_{1}-x_{2}|}$
$\displaystyle\qquad=\min(|c|^{-1},1)\frac{|(h\circ g)(x_{1})-(h\circ
g)(x_{2})|}{|g(x_{1})-g(x_{2})|}\frac{|g(x_{1})-g(x_{2})|}{|x_{1}-x_{2}|}$
$\displaystyle\qquad\leq\min(|c|,1)\frac{|(h\circ g)(x_{1})-(h\circ
g)(x_{2})|}{|g(x_{1})-g(x_{2})|}$ $\displaystyle\qquad\leq 1$

because $h\in\mathrm{BL}_{1}(\mathbb{R})$. Since in addition
$|\min(|c|^{-1},1)\times(h\circ g)|\leq 1$, $\min(|c|^{-1},1)\times(h\circ
g)\in\mathrm{BL}_{1}(\mathbb{R})$. Hence, for the third term on the right-hand
side of (19), we have

$\displaystyle|E_{W}^{*}h(X_{n}^{*}c)-E_{0}h(Xc)|$
$\displaystyle=\frac{1}{\min(|c|^{-1},1)}\left|{E_{W}^{*}(\min(|c|^{-1},1)\times(h\circ
g))(X_{n}^{*})-E_{0}(\min(|c|^{-1},1)\times(h\circ g))(X)}\right|$
$\displaystyle\leq\frac{1}{\min(|c|^{-1},1)}\sup_{h\in\mathrm{BL}_{1}(\mathbb{R})}|E_{W}^{*}h(X_{n}^{*})-E_{0}h(X)|,$

which converges to 0 in outer probability since
$X_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}X$ by assumption. For the
second term on the right-hand side of (19), for any $M>0$,

$\displaystyle P_{W}^{*}(|X_{n}^{*}Y_{n}^{*}-X_{n}^{*}c|\geq\varepsilon)$
$\displaystyle=P_{W}^{*}(|X_{n}^{*}Y_{n}^{*}-X_{n}^{*}c|\geq\varepsilon,|X_{n}^{*}|<M)$
$\displaystyle\qquad+P_{W}^{*}(|X_{n}^{*}Y_{n}^{*}-X_{n}^{*}c|\geq\varepsilon,|X_{n}^{*}|\geq
M)$ $\displaystyle\leq
P_{W}^{*}(|Y_{n}^{*}-c|\geq\varepsilon/M)+P_{W}^{*}(|X_{n}^{*}|\geq M).$

By assumption, $Y_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}c$ so that
$P_{W}^{*}(|Y_{n}^{*}-c|\geq\varepsilon/M)=o_{P_{0}^{*}}(1)$. Hence, the proof
is complete if we can show that given any $\varepsilon,\gamma>0$, there exists
$M>0$ such that $P_{0}^{*}(P_{W}^{*}(|X_{n}^{*}|\geq
M)\geq\varepsilon)<\gamma$ for all $n$ large enough.

For each $\varepsilon,\gamma>0$ there exists a sequence of functions
$h_{m}:\mathbb{R}\to\mathbb{R}$ such that $h_{m}$ is $m$-Lipschitz for all
$m$, $1\\{|x|\geq M\\}\leq h_{m}(z)\leq 1$ for all $x\in\mathbb{R}$ and $m$,
and $h_{m}(z)$ monotonically decreases to $1\\{|x|\geq M\\}$ as $m\to\infty$
for each $x\in\mathbb{R}$. For instance,
$h_{m}:x\mapsto\min\\{\max(m[|x|-M]+1,0),1\\}$ satisfies these criteria. For
any $\varepsilon>0$ and $m\in\\{1,2,\dots\\}$, we can now write

$\displaystyle P^{*}_{W}\left(|X_{n}^{*}|\geq M\right)$
$\displaystyle=\left[E^{*}_{W}1\left\\{|X_{n}^{*}|\geq
M\right\\}-E^{*}_{W}h_{m}\left(X_{n}^{*}\right)\right]+\left[E^{*}_{W}h_{m}\left(X_{n}^{*}\right)-E_{0}h_{m}(X)\right]$
$\displaystyle\qquad+\left[E_{0}h_{m}(X)-E_{0}1\left\\{|X|\geq
M\right\\}\right]+P_{0}\left(|X|\geq M\right)$
$\displaystyle\leq\left[E^{*}_{W}h_{m}\left(X_{n}^{*}\right)-E_{0}h_{m}(X)\right]+\left[E_{0}h_{m}(X)-E_{0}1\left\\{|X|\geq
M\right\\}\right]$ $\displaystyle\qquad+P_{0}\left(|X|\geq M\right),$

where the second inequality follows because $E^{*}_{W}1\left\\{|X_{n}^{*}|\geq
M\right\\}\leq E^{*}_{W}h_{m}\left(X_{n}^{*}\right)$ for all $\varepsilon>0$
and $m\in\\{1,2,\dots\\}$ by assumption. We therefore have

$\displaystyle P_{0}^{*}(P_{W}^{*}(|X_{n}^{*}|\geq M)\geq\varepsilon)$
$\displaystyle\qquad\leq
P_{0}^{*}\left([E^{*}_{W}h_{m}\left(X_{n}^{*}\right)-E_{0}h_{m}(X)]\geq\varepsilon/3\right)+P_{0}^{*}\left([E_{0}h_{m}(X)-E_{0}1\left\\{|X|\geq
M\right\\}]\geq\varepsilon/3\right)$
$\displaystyle\qquad\qquad+P_{0}^{*}(P_{0}\left(|X|\geq
M\right)\geq\varepsilon/3)$
$\displaystyle\qquad=P_{0}^{*}\left([E^{*}_{W}h_{m}\left(X_{n}^{*}\right)-E_{0}h_{m}(X)]\geq\varepsilon/3\right)+1\left\\{[E_{0}h_{m}(X)-E_{0}1\left\\{|X|\geq
M\right\\}]\geq\varepsilon/3\right\\}$
$\displaystyle\qquad\qquad+1\left\\{P_{0}\left(|X|\geq
M\right)\geq\varepsilon/3\right\\}.$ (20)

First, since $X$ is tight by assumption, we can choose $M$ suh that the third
term on the right-hand side of (B) is 0. Next, for this choice of $M$, by the
monotone convergence theorem we can choose $m$ large enough so that the second
term on the right-hand side of (B) is 0. Finally, for these choices of $m$ and
$M$, for the first term on the right-hand side of (B), we have
$\min(m^{-1},1)\times h_{m}\in\mathrm{BL}_{1}(\mathbb{R})$ because $h_{m}$ is
bounded and $m$-Lipschitz. Therefore, by the assumed conditional weak
convergence of $X_{n}^{*}$ to $X$ in $\mathbb{R}$,
$E^{*}_{W}[\min(m^{-1},1)\times
h_{m}\left(X_{n}^{*}\right)]\xrightarrow{P_{0}^{*}}E_{0}[\min(m^{-1},1)\times
h_{m}(X)]$ for each $m$, so that
$E^{*}_{W}h_{m}\left(X_{n}^{*}\right)\xrightarrow{P_{0}^{*}}E_{0}h_{m}(X)$ as
well. Hence, the first term on the right-hand side of (B) converges to 0 for
fixed $m$ and $\varepsilon$, so we can make it as small as we like for $n$
large enough. Hence, for any $\varepsilon$, there exist $M>0$ and
$m\in\\{1,2,\dots\\}$ such that $P_{W}^{*}(|X_{n}^{*}|\geq
M)=o_{P_{0}^{*}}(1)$ for $n$ large enough. $\hfill\blacksquare$

We next have a result that generalizes Theorem 23.4 of van der Vaart, (2000)
to include smooth bootstraps.

######  Lemma B.2 (Conditional CLT).

Let $\phi_{0}:\mathbb{R}^{d}\mapsto\mathbb{R}$ satisfy $P_{0}\phi_{0}=0$ and
$P_{0}\phi_{0}^{2}<\infty$. Let $X_{1},X_{2},\dots$ be IID random vectors with
mean $\mu$ and covariance matrix $\Sigma$. If
$\hat{P}_{n}\phi_{0}^{2}\xrightarrow{P_{0}^{*}}P_{0}\phi_{0}^{2}$ and
$(\hat{P}_{n}-P_{0})[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]\xrightarrow{P_{0}^{*}}0$
for any $M>0$, then conditionally on every sequence $X_{1},X_{2},\dots$, in
outer probabiliy,

$n^{1/2}\left(\mathbb{P}_{n}^{*}-\hat{P}_{n}\right)\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}N(0,P_{0}\phi_{0}^{2}).$

###### Proof of Lemma B.2.

We use the Lindeberg-Feller CLT for triangular arrays (e.g., Proposition 2.27
of van der Vaart,, 2000). Since the bootstrap sample
$X_{1}^{*},\dotsc,X_{n}^{*}$ is drawn IID from $\hat{P}_{n}$ given
$X_{1},\dotsc,X_{n}$, the conditional mean and variance of
$\phi_{0}(X_{i}^{*})$ are given by

$\displaystyle E_{W}\phi_{0}(X_{i}^{*})$
$\displaystyle=\int\phi_{0}(x)\,\mathrm{d}\hat{P}_{n}(x)=\hat{P}_{n}\phi_{0},\text{
and}$ $\displaystyle E_{W}[\phi_{0}(X_{i}^{*})-E_{W}\phi_{0}(X_{i}^{*})]^{2}$
$\displaystyle=E_{W}\phi_{0}^{2}(X_{i}^{*})-[E_{W}\phi_{0}(X_{i}^{*})]^{2}=\hat{P}_{n}\phi_{0}^{2}-(\hat{P}_{n}\phi_{0})^{2}\xrightarrow{P_{0}^{*}}P_{0}\phi_{0}^{2}.$

We next verify the Lindeberg condition. For any $\varepsilon,\gamma>0$, there
exists $M>0$ such that
$\int\phi_{0}(x)^{2}1\\{|\phi_{0}(x)|>M\\}\,\mathrm{d}P_{0}(x)<\varepsilon/2$
and for all $n$ large enough, $M\leq\gamma\sqrt{n}$. Therefore,

$\displaystyle
P_{0}\left(E_{W}[\phi_{0}(X_{i}^{*})^{2}1\\{|\phi_{0}(X_{i}^{*})|>\gamma\sqrt{n}\\}]\geq\varepsilon\right)$
$\displaystyle\qquad=P_{0}\left(\hat{P}_{n}[\phi_{0}^{2}1\\{|\phi_{0}|>\gamma\sqrt{n}\\}]\geq\varepsilon\right)$
$\displaystyle\qquad\leq
P_{0}\left(\hat{P}_{n}[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]\geq\varepsilon\right)$
$\displaystyle\qquad\leq
P_{0}\left(|(\hat{P}_{n}-P_{0})[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]|\geq\varepsilon/2\right)+P_{0}\left(P_{0}[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]\geq\varepsilon/2\right)$
$\displaystyle\qquad=P_{0}\left(|(\hat{P}_{n}-P_{0})[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]|\geq\varepsilon/2\right)+1\left\\{P_{0}[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]\geq\varepsilon/2\right\\}$
$\displaystyle\qquad=P_{0}\left(|(\hat{P}_{n}-P_{0})[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]|\geq\varepsilon/2\right)$
$\displaystyle\qquad\longrightarrow 0$

Thus,
$E_{W}[\phi_{0}(X_{i}^{*})^{2}1\\{|\phi_{0}(X_{i}^{*})|>\varepsilon\sqrt{n}\\}]\xrightarrow{P_{0}^{*}}0$.
Therefore, the conditions of the Lindeberg-Feller CLT holds conditionally on
every sequence $X_{1},X_{2},\dots$, in outer probability, which yields the
result. $\hfill\blacksquare$

See 3.9

###### Proof of 3.9.

We prove the result for the percentile $t$-method; the result for the
percentile method follows the same argument setting
$\sigma_{n}=\sigma_{n}^{*}=\sigma_{0}=1$ (so that the additional conditions
for the percentile $t$-method hold automatically).

We first show that
$\sigma_{n}^{*-1}(S_{n}^{*}+R_{n}^{*})-\sigma_{n}^{-1}(S_{n}+R_{n})=o_{P_{W}^{*}}(n^{-1/2})$.
By adding and subtracting terms, we note that

$\displaystyle\frac{S_{n}^{*}+R_{n}^{*}}{\sigma_{n}^{*}}-\frac{S_{n}+R_{n}}{\sigma_{n}}$
$\displaystyle=\frac{\sigma_{n}(S_{n}^{*}+R_{n}^{*})-\sigma_{n}(S_{n}+R_{n})+\sigma_{n}(S_{n}+R_{n})-\sigma_{n}^{*}(S_{n}+R_{n})}{\sigma_{n}\sigma_{n}^{*}}$
$\displaystyle=\frac{(S_{n}^{*}-S_{n})+(R_{n}^{*}-R_{n})}{\sigma_{n}^{*}}+\frac{(\sigma_{n}-\sigma_{n}^{*})(S_{n}+R_{n})}{\sigma_{n}\sigma_{n}^{*}}.$
(21)

Since $\sigma_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\sigma_{0}>0$, we
can show that $\sigma_{n}^{*-1}=O_{P_{W}^{*}}(1)$ using the same logic as the
unconditional result. Hence, the above equals
$o_{P_{W}^{*}}(n^{-1/2})O_{P_{W}^{*}}(1)+o_{P_{W}^{*}}(n^{-1/2})O_{P_{W}^{*}}(1)$
by assumption.

For any $t\in\mathbb{R}$, by definition of $S_{n}$ and $R_{n}$, we have

$\displaystyle
P_{0}^{*}\left(\frac{T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})}{\sigma_{n}}\leq
t\right)$
$\displaystyle=P_{0}^{*}\left(\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}+n^{1/2}\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]\leq
n^{1/2}t\right)$
$\displaystyle=P_{0}^{*}\left(\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}\leq
T_{n}\right),$

where $T_{n}:=n^{1/2}t-n^{1/2}\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]$. Thus,
by definition of $S_{n}^{*}$ and $R_{n}^{*}$,

$\displaystyle
P_{W}^{*}\left(\frac{T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})}{\sigma_{n}^{*}}\leq
t\right)$
$\displaystyle=P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+n^{1/2}\sigma_{n}^{*-1}\left[S_{n}^{*}+R_{n}^{*}\right]\leq
n^{1/2}t\right)$
$\displaystyle=P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+n^{1/2}\sigma_{n}^{*-1}\left[S_{n}^{*}+R_{n}^{*}\right]\leq
T_{n}+n^{1/2}\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]\right)$
$\displaystyle=P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+A_{n}^{*}\leq
T_{n}\right),$

where
$A_{n}^{*}:=n^{1/2}\left\\{\sigma_{n}^{*-1}\left[S_{n}^{*}+R_{n}^{*}\right]-\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]\right\\}$.
Thus,

$\displaystyle\sup_{t\in\mathbb{R}}$
$\displaystyle\left|P_{W}^{*}\left(\frac{T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})}{\sigma_{n}^{*}}\leq
t\right)-P_{0}^{*}\left(\frac{T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})}{\sigma_{n}}\leq
t\right)\right|$
$\displaystyle=\sup_{t\in\mathbb{R}}\left|P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+A_{n}^{*}\leq
t\right)-P_{0}^{*}\left(\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}\leq
t\right)\right|$
$\displaystyle\leq\sup_{t\in\mathbb{R}}\left|P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+A_{n}^{*}\leq
t\right)-P_{0}^{*}\left(\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}\leq
t\right)\right|+\sup_{\xi\in\mathbb{R}}\left|P_{0}^{*}\left(\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}\leq
t\right)-P_{0}^{*}\left(\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}\leq
t\right)\right|.$

By Lemma B.2, we have
$\mathbb{G}_{n}^{*}\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$.
Since $\sigma_{n}^{*2}\underset{W}{\xrightarrow{P_{0}^{*}}}\sigma_{0}^{2}$,
$\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}$
by Lemma B.1. Since $A_{n}^{*}=o_{P_{W}^{*}}(1)$, it follows that
$\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+A_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}$.
Hence, by the Portmanteau theorem (e.g., Lemma 1.3.4 of van der Vaart &
Wellner,, 1996),

$\sup_{t\in\mathbb{R}}\left|P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+A_{n}^{*}\leq
t\right)-P_{0}^{*}\left(\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}\leq
t\right)\right|\xrightarrow{P_{0}^{*}}0.$

Similarly,
$\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}\leadsto\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}$,
so

$\sup_{t\in\mathbb{R}}\left|P_{0}^{*}\left(\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}\leq\xi\right)-P_{0}^{*}\left(\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}\leq
t\right)\right|\to 0.$

The first statement follows.

By definition of $\xi_{n,p}^{*},$ $S_{n}^{*}$, and $R_{n}^{*}$, we have

$\displaystyle\xi_{n,p}^{*}:=$
$\displaystyle\inf\left\\{\xi\in\mathbb{R}:P_{W}^{*}\left(\frac{T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\hat{P}_{n})}{\sigma_{n}^{*}}\leq\xi\right)\geq
p\right\\}$ $\displaystyle=$
$\displaystyle\inf\left\\{\xi\in\mathbb{R}:P_{W}^{*}\left(n^{-1/2}\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+\sigma_{n}^{*-1}\left[S_{n}^{*}+R_{n}^{*}\right]\leq\xi\right)\geq
p\right\\}.$

Since $\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]$ is a function of the original
data, we have

$\displaystyle
n^{1/2}\left(\xi_{n,p}^{*}-\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]\right)$
$\displaystyle\qquad=\inf\left\\{\xi\in\mathbb{R}:P_{W}^{*}\left(n^{-1/2}\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+\sigma_{n}^{*-1}\left[S_{n}^{*}+R_{n}^{*}\right]\leq
n^{-1/2}\xi+\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]\right)\geq p\right\\}$
$\displaystyle\qquad=\inf\left\\{\xi\in\mathbb{R}:P_{W}^{*}\left(\sigma_{n}^{*-1}\mathbb{G}_{n}^{*}\phi_{0}+A_{n}^{*}\leq\xi\right)\geq
p\right\\}.$

By the above derivations and van der Vaart,, 2000, Lemma 21.2, for any
$p\in(0,1)$, we then have

$n^{1/2}\left(\xi_{n,p}^{*}-\sigma_{n}^{-1}\left[S_{n}+R_{n}\right]\right)\xrightarrow{P_{0}^{*}}\Phi^{-1}(p).$

Therefore,

$\displaystyle
P_{0}^{*}\left(T(\eta_{n},\mathbb{P}_{n})-\xi_{n,1-\alpha}^{*}\sigma_{n}\leq
T(\eta_{0},P_{0})\leq
T(\eta_{n},\mathbb{P}_{n})-\xi_{n,\beta}^{*}\sigma_{n}\right)$
$\displaystyle\qquad=P_{0}^{*}\left(n^{1/2}\xi_{n,\beta}^{*}\leq
n^{1/2}[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})]/\sigma_{n}\leq
n^{1/2}\xi_{n,1-\alpha}^{*}\right)$
$\displaystyle\qquad=P_{0}^{*}\left(n^{1/2}\left[\xi_{n,\beta}^{*}-\sigma_{n}^{-1}(S_{n}+R_{n})\right]\leq\sigma_{n}^{-1}\mathbb{G}_{n}\phi_{0}\leq
n^{1/2}\left[\xi_{n,1-\alpha}^{*}-\sigma_{n}^{-1}(S_{n}+R_{n})\right]\right)$
$\displaystyle\qquad\longrightarrow
P_{0}^{*}\left(\Phi^{-1}(\beta)\leq\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}\leq\Phi^{-1}(1-\alpha)\right)$
$\displaystyle\qquad=1-\alpha-\beta.$

$\hfill\blacksquare$

See 3.10

###### Proof of 3.10.

We note that in the proof of 3.9, the conditions
$\hat{P}_{n}\phi_{0}^{2}\xrightarrow{P_{0}^{*}}P_{0}\phi_{0}^{2}$ and
$(\hat{P}_{n}-P_{0})[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]\xrightarrow{P_{0}^{*}}0$
were only used to establish that
$\mathbb{G}_{n}^{*}\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$
using Lemma B.2. However, condition (B1) directly implies
$\mathbb{G}_{n}^{*}\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$,
so it suffices to check the remainder of the conditions of 3.9. By conditions
(A1)–(A3), we have $S_{n}=o_{P_{0}^{*}}(n^{-1/2})$ and
$R_{n}=o_{P_{0}^{*}}(n^{-1/2})$. By conditions (B1)–(B3), we have
$S_{n}^{*}=o_{P_{W}^{*}}(n^{-1/2})$ and $R_{n}^{*}=o_{P_{W}^{*}}(n^{-1/2})$.
Hence, $S_{n}^{*}-S_{n}=o_{P_{W}^{*}}(n^{-1/2})$ and
$R_{n}^{*}-R_{n}=o_{P_{W}^{*}}(n^{-1/2})$, which implies the consistency of
the bootstrap percentile confidence interval by 3.9. For the percentile $t$
interval, $\sigma_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\sigma_{0}$ and
$\sigma_{n}\xrightarrow{P_{0}^{*}}\sigma_{0}$ by assumption, which also
implies
$(S_{n}+R_{n})(\sigma_{n}^{*}-\sigma_{n})=o_{P_{W}^{*}}(n^{-1/2})o_{P_{W}^{*}}(1)=o_{P_{W}^{*}}(n^{-1/2})$.
$\hfill\blacksquare$

See 3.12

###### Proof of 3.12.

For any $t\in\mathbb{R}$, by definition of $S_{n}$ and $R_{n}$, we have

$\displaystyle
P_{0}^{*}\left(-\left[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})\right]\leq
t\right)$
$\displaystyle=P_{0}^{*}\left(-\mathbb{G}_{n}\phi_{0}-n^{1/2}\left[S_{n}+R_{n}\right]\leq
n^{1/2}t\right)=P_{0}^{*}\left(-\mathbb{G}_{n}\phi_{0}\leq T_{n}\right),$

where $T_{n}:=n^{1/2}t+n^{1/2}\left[S_{n}+R_{n}\right]$. Thus, by definition
of $S_{n}^{*}$ and $R_{n}^{*}$,

$\displaystyle
P_{W}^{*}\left(T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\mathbb{P}_{n})\leq
t\right)$
$\displaystyle=P_{W}^{*}\left(\mathbb{G}_{n}^{*}\phi_{0}+n^{1/2}\left[S_{n}^{*}+R_{n}^{*}+T(\eta_{n},\hat{P}_{n})-T(\eta_{n},\mathbb{P}_{n})\right]\leq
n^{1/2}t\right)$
$\displaystyle=P_{W}^{*}\left(\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\leq
T_{n}\right),$

where
$B_{n}^{*}:=n^{1/2}\left[S_{n}^{*}+S_{n}+R_{n}^{*}+R_{n}+T(\eta_{n},\hat{P}_{n})-T(\eta_{n},\mathbb{P}_{n})\right]$.
Thus,

$\displaystyle\sup_{t\in\mathbb{R}}$
$\displaystyle\left|P_{W}^{*}\left(T(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T(\eta_{n},\mathbb{P}_{n})\leq
t\right)-P_{0}^{*}\left(-\left[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})\right]\leq
t\right)\right|$
$\displaystyle=\sup_{t\in\mathbb{R}}\left|P_{W}^{*}\left(\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\leq
t\right)-P_{0}^{*}\left(-\mathbb{G}_{n}\phi_{0}\leq t\right)\right|$
$\displaystyle\leq\sup_{t\in\mathbb{R}}\left|P_{W}^{*}\left(\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\leq
t\right)-P_{0}^{*}\left(\mathbb{G}_{0}\phi_{0}\leq
t\right)\right|+\sup_{t\in\mathbb{R}}\left|P_{0}^{*}\left(-\mathbb{G}_{n}\phi_{0}\leq
t\right)-P_{0}^{*}\left(\mathbb{G}_{0}\phi_{0}\leq t\right)\right|.$

By Lemma B.2 and since $B_{n}^{*}=o_{P_{W}^{*}}(1)$ by assumption, we have
$\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$.
Hence, by the Portmanteau theorem (e.g., Lemma 1.3.4 of van der Vaart &
Wellner,, 1996),

$\sup_{t\in\mathbb{R}}\left|P_{W}^{*}\left(\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\leq
t\right)-P_{0}^{*}\left(\mathbb{G}_{0}\phi_{0}\leq
t\right)\right|\xrightarrow{P_{0}^{*}}0.$

Similarly, $\mathbb{G}_{n}\phi_{0}\leadsto\mathbb{G}_{0}\phi_{0}$, and since
$\mathbb{G}_{0}\phi_{0}$ follows a mean-zero normal distribution,
$\mathbb{G}_{n}\phi_{0}\leadsto-\mathbb{G}_{0}\phi_{0}$ as well. Therefore,

$\sup_{t\in\mathbb{R}}\left|P_{0}^{*}\left(\mathbb{G}_{n}\phi_{0}\leq
t\right)-P_{0}^{*}\left(-\mathbb{G}_{0}\phi_{0}\leq t\right)\right|\to 0.$

The first statement follows.

By definition of $\zeta_{n,p}^{*},S_{n}^{*}$ and $R_{n}^{*}$, we have

$\displaystyle\zeta_{n,p}^{*}:=$
$\displaystyle\inf\left\\{\zeta\in\mathbb{R}:P_{W}^{*}\left(T(\eta_{n}^{*},\mathbb{P}_{n}^{*})\leq\zeta\right)\geq
p\right\\}$ $\displaystyle=$
$\displaystyle\inf\left\\{\zeta\in\mathbb{R}:P_{W}^{*}\left(n^{-1/2}\mathbb{G}_{n}^{*}\phi_{0}+(S_{n}^{*}+R_{n}^{*})+T(\eta_{n},\hat{P}_{n})\leq\zeta\right)\geq
p\right\\}.$

Since $[S_{n}+R_{n}-T(\eta_{n},\mathbb{P}_{n})]$ is a function of the original
data, we have

$\displaystyle
n^{1/2}\left\\{\zeta_{n,p}^{*}+[S_{n}+R_{n}-T(\eta_{n},\mathbb{P}_{n})]\right\\}$
$\displaystyle\qquad=\inf\left\\{\zeta\in\mathbb{R}:P_{W}^{*}\left(n^{-1/2}\mathbb{G}_{n}^{*}\phi_{0}+(S_{n}^{*}+R_{n}^{*})+T(\eta_{n},\hat{P}_{n})\leq
n^{-1/2}\zeta-[S_{n}+R_{n}-T(\eta_{n},\mathbb{P}_{n})]\right)\geq p\right\\}$
$\displaystyle\qquad=\inf\left\\{\zeta\in\mathbb{R}:P_{W}^{*}\left(\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\leq\zeta\right)\geq
p\right\\},$

As argued above,
$\mathbb{G}_{n}^{*}\phi_{0}+B_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$,
so by Lemma 21.2 of van der Vaart, (2000), for any $p\in(0,1)$, we then have

$n^{1/2}\left\\{\zeta_{n,p}^{*}+\left[S_{n}+R_{n}-T(\eta_{n},\mathbb{P}_{n})\right]\right\\}\xrightarrow{P_{0}^{*}}\sigma_{0}\Phi^{-1}(p).$

Therefore,

$\displaystyle P_{0}^{*}\left(\zeta_{n,1-\alpha}^{*}\leq
T(\eta_{0},P_{0})\leq\zeta_{n,\beta}^{*}\right)$
$\displaystyle\qquad=P_{0}^{*}\left(-n^{1/2}[\zeta_{n,\beta}^{*}-T(\eta_{n},\mathbb{P}_{n})]\leq
n^{1/2}[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})]\leq-n^{1/2}[\zeta_{n,1-\alpha}^{*}-T(\eta_{n},\mathbb{P}_{n})]\right)$
$\displaystyle\qquad=P_{0}^{*}\left(-n^{1/2}[\zeta_{n,\beta}^{*}-T(\eta_{n},\mathbb{P}_{n})]\leq\mathbb{G}_{n}\phi_{0}+n^{1/2}(S_{n}+R_{n})\leq-n^{1/2}[\zeta_{n,1-\alpha}^{*}-T(\eta_{n},\mathbb{P}_{n})]\right)$
$\displaystyle\qquad=P_{0}^{*}\left(-n^{1/2}[\zeta_{n,\beta}^{*}+S_{n}+R_{n}-T(\eta_{n},\mathbb{P}_{n})]\leq\mathbb{G}_{n}\phi_{0}\leq-n^{1/2}[\zeta_{n,1-\alpha}^{*}+S_{n}+R_{n}-T(\eta_{n},\mathbb{P}_{n})]\right)$
$\displaystyle\qquad\longrightarrow
P_{0}^{*}\left(-\Phi^{-1}(\beta)\leq\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}\leq-\Phi^{-1}(1-\alpha)\right)$
$\displaystyle\qquad=1-\alpha-\beta,$

where the last equality is due to the symmetry of the standard normal
distribution. This shows the consistency of Efron’s percentile method.

$\hfill\blacksquare$

See 3.13

###### Proof of 3.13.

As in the proof of Corollary 3.10, we note that in the proof of 3.12, the
conditions $\hat{P}_{n}\phi_{0}^{2}\xrightarrow{P_{0}^{*}}P_{0}\phi_{0}^{2}$
and
$(\hat{P}_{n}-P_{0})[\phi_{0}^{2}1\\{|\phi_{0}|>M\\}]\xrightarrow{P_{0}^{*}}0$
were only used to establish that
$\mathbb{G}_{n}^{*}\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$
using Lemma B.2. However, condition (B1) directly implies
$\mathbb{G}_{n}^{*}\phi_{0}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$,
so it suffices to check the remainder of the conditions of 3.12.

By conditions (A1)–(A3), we have $S_{n}=o_{P_{0}^{*}}(n^{-1/2})$ and
$R_{n}=o_{P_{0}^{*}}(n^{-1/2})$. By conditions (B1)–(B3), we have
$S_{n}^{*}=o_{P_{W}^{*}}(n^{-1/2})$ and $R_{n}^{*}=o_{P_{W}^{*}}(n^{-1/2})$.
Since
$T(\eta_{n},\hat{P}_{n})-T(\eta_{n},\mathbb{P}_{n})=o_{P_{0}^{*}}(n^{-1/2})$
by assumption, we then have
$S_{n}^{*}+S_{n}+R_{n}^{*}+R_{n}+T(\eta_{n},\hat{P}_{n})-T(\eta_{n},\mathbb{P}_{n})=o_{P_{W}^{*}}(n^{-1/2})$.
Hence, the result follows by 3.12.

$\hfill\blacksquare$

See 3.14

###### Proof of 3.14.

Let $h(x):=x^{2}$. There exists a sequence of functions
$h_{m}:\mathbb{R}\mapsto\mathbb{R}$, $m=1,2,\dots$ such that
$h_{m}\in\mathrm{BL}_{1}(\mathbb{R})$ for all $m$ and $mh_{m}$ monotonically
increases to $h$ as $m\to\infty$. For instance, $h_{m}(x):=\min\\{x^{2}/(m\vee
4),1\\}$ satisfies these criteria. We then write

$\displaystyle
E_{W}^{*}h\left(T_{n}^{*}\right)-E_{0}h(\mathbb{G}_{0}\phi_{0})$
$\displaystyle=[E_{W}^{*}h\left(T_{n}^{*}\right)-mE_{W}^{*}h_{m}\left(T_{n}^{*}\right)]+[mE_{W}^{*}h_{m}\left(T_{n}^{*}\right)-mE_{0}h_{m}\left(\mathbb{G}_{0}\phi_{0}\right)]$
$\displaystyle\qquad\qquad+[mE_{0}h_{m}\left(\mathbb{G}_{0}\phi_{0}\right)-E_{0}h(\mathbb{G}_{0}\phi_{0})].$

Hence, for any $\varepsilon>0$, we have

$\displaystyle
P_{0}^{*}\left(\left|{E_{W}^{*}h\left(T_{n}^{*}\right)-E_{0}h(\mathbb{G}_{0}\phi_{0})}\right|\geq\varepsilon\right)$
$\displaystyle\qquad\leq
P_{0}^{*}\left(\left|{E_{W}^{*}h\left(T_{n}^{*}\right)-mE_{W}^{*}h_{m}\left(T_{n}^{*}\right)}\right|\geq\varepsilon/3\right)+P_{0}^{*}\left(\left|{E_{W}^{*}h_{m}\left(T_{n}^{*}\right)-E_{0}h_{m}\left(\mathbb{G}_{0}\phi_{0}\right)}\right|\geq\varepsilon/(3m)\right)$
$\displaystyle\qquad\qquad+P_{0}^{*}\left(\left|{mE_{0}h_{m}\left(\mathbb{G}_{0}\phi_{0}\right)-E_{0}h(\mathbb{G}_{0}\phi_{0})}\right|\geq\varepsilon/3\right)$
$\displaystyle\qquad\leq
P_{0}^{*}\left(\left|{E_{W}^{*}h\left(T_{n}^{*}\right)-mE_{W}^{*}h_{m}\left(T_{n}^{*}\right)}\right|\geq\varepsilon/3\right)+P_{0}^{*}\left(\sup_{g\in\mathrm{BL}_{1}(\mathbb{R})}\left|{E_{W}^{*}g\left(T_{n}^{*}\right)-E_{0}g\left(\mathbb{G}_{0}\phi_{0}\right)}\right|\geq\varepsilon/(3m)\right)$
$\displaystyle\qquad\qquad+1\left\\{E_{0}\left|{mh_{m}\left(\mathbb{G}_{0}\phi_{0}\right)-h(\mathbb{G}_{0}\phi_{0})}\right|\geq\varepsilon/3\right\\}.$

By conditions (B1)-(B3), we have
$E_{W}^{*}T_{n}^{*}\underset{W}{\overset{P_{0}^{*}}{\leadsto}}\mathbb{G}_{0}\phi_{0}$,
i.e.,
$\sup_{g\in\mathrm{BL}_{1}(\mathbb{R})}\left|{E_{W}^{*}g\left(T_{n}^{*}\right)-E_{0}g(\mathbb{G}_{0}\phi_{0})}\right|\xrightarrow{P_{0}^{*}}0$.
Therefore, for each fixed $m$, the second term on the right-hand side of
previous display converges to 0. By the monotone convergence theorem, the
third term on the right-hand side of previous display converges to 0 as
$m\longrightarrow\infty$. For the first term on the right hand side of
previous display, we note that

$\displaystyle
P_{0}^{*}\left(\left|{E_{W}^{*}h\left(T_{n}^{*}\right)-mE_{W}^{*}h_{m}\left(T_{n}^{*}\right)}\right|\geq\varepsilon/3\right)$
$\displaystyle=P_{0}^{*}\left(\left|{E_{W}^{*}T_{n}^{*2}-E_{W}^{*}\min\\{T_{n}^{*2},m\\}}\right|\geq\varepsilon/3\right)$
$\displaystyle=P_{0}^{*}\left(\left|{E_{W}^{*}(T_{n}^{*2}-m)1\\{T_{n}^{*2}\geq
m\\}}\right|\geq\varepsilon/3\right)$ $\displaystyle\leq
P_{0}^{*}\left(\left|{E_{W}^{*}T_{n}^{*2}1\\{T_{n}^{*2}\geq
m\\}}\right|\geq\varepsilon/3\right)$ $\displaystyle\leq
E_{0}^{*}E_{W}^{*}T_{n}^{*2}1\\{T_{n}^{*2}\geq m\\}/(\varepsilon/3),$

and $\lim_{m\to\infty}\limsup_{n\to\infty}$ of this latter expression is 0 by
assumption. Therefore, we have

$\limsup_{n\to\infty}P_{0}^{*}\left(\left|{E_{W}^{*}h\left(T_{n}^{*}\right)-E_{0}h(\mathbb{G}_{0}\phi_{0})}\right|\geq\varepsilon\right)=\lim_{m\to\infty}\limsup_{n\to\infty}P_{0}^{*}\left(\left|{E_{W}^{*}h\left(T_{n}^{*}\right)-E_{0}h(\mathbb{G}_{0}\phi_{0})}\right|\geq\varepsilon\right)=0$

for every $\varepsilon>0$. Hence,
$\bar{\sigma}_{n}^{2}=E_{W}^{*}T_{n}^{*2}\xrightarrow{P_{0}^{*}}E_{0}(\mathbb{G}_{0}\phi_{0})^{2}=\sigma_{0}^{2}$.
By conditions (A1)–(A3), we have
$n^{1/2}[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})]\leadsto\mathbb{G}_{0}\phi_{0}$.
By Slutsky’s theorem, we have
$n^{1/2}\bar{\sigma}_{n}^{-1}[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})]\leadsto\sigma_{0}^{-1}\mathbb{G}_{0}\phi_{0}$.
We then have

$\displaystyle
P_{0}\left(T(\eta_{n},\mathbb{P}_{n})+z_{\beta}\bar{\sigma}_{n}n^{-1/2}\leq
T(\eta_{0},P_{0})\leq
T(\eta_{n},\mathbb{P}_{n})+z_{1-\alpha}\bar{\sigma}_{n}n^{-1/2}\right)$
$\displaystyle\qquad=P_{0}\left(-z_{1-\alpha}\leq
n^{1/2}[T(\eta_{n},\mathbb{P}_{n})-T(\eta_{0},P_{0})]/\bar{\sigma}_{n}\leq-
z_{\beta}\right)$ $\displaystyle\qquad\to
P_{0}\left(-z_{1-\alpha}\leq\mathbb{G}_{0}\phi_{0}/\sigma_{0}\leq-
z_{\beta}\right)$ $\displaystyle\qquad=1-\alpha-\beta,$

and the result follows.

$\hfill\blacksquare$

##  Supplement C Proof of results in Section 4

We first introduce a Lemma providing conditions under which the (bootstrap)
estimating equations-based estimator is (conditionally) consistent, which may
be useful in its own right.

######  Lemma C.1.

If $\psi_{0}$ is a well-separated solution of $\psi\mapsto
G_{0,\eta_{0}}(\psi)$,
$\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}=o_{P_{0}^{*}}(1)$,
$\phi_{\psi_{n},\eta_{n}}$ is contained in a $P_{0}$-Glivenko Cantelli class
with probability tending to one, $\sup_{|\psi|\leq
M}\left|P_{0}(\phi_{\psi,\eta_{n}}-\phi_{\psi,\eta_{0}})\right|=o_{P_{0}^{*}}(1)$
for all $M>0$, and $\psi_{n}=O_{P_{0}^{*}}(1)$, then
$\psi_{n}-\psi_{0}=o_{P_{0}^{*}}(1)$. If $\psi_{0}$ is a well-separated
solution of $\psi\mapsto G_{0,\eta_{0}}(\psi)$,
$\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{W}^{*}}(1)$,
$\phi_{\psi_{n}^{*},\eta_{n}^{*}}$ is contained in a $P_{0}$-Glivenko Cantelli
class $\mathscr{F}$ with conditional probability tending to one,
$\sup_{|\psi|\leq
M}\left|P_{0}(\phi_{\psi,\eta_{n}^{*}}-\phi_{\psi,\eta_{0}})\right|=o_{P_{W}^{*}}(1)$
for all $M>0$, $\psi_{n}^{*}=O_{P_{W}^{*}}(1)$, and
$\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}}=o_{P_{0}^{*}}(1)$, then
$\psi_{n}^{*}-\psi_{0}=o_{P_{W}^{*}}(1)$.

###### Proof of Lemma C.1.

By the well-separated assumption, for every $\varepsilon>0$ there exists
$\delta>0$ such that $|P_{0}\phi_{\psi,\eta_{0}}|\geq\delta$ for every
$\psi\in\mathbb{R}$ such that $|\psi-\psi_{0}|\geq\varepsilon$. Therefore, the
event $\\{|\psi_{n}-\psi_{0}|\geq\varepsilon\\}$ is contained in the event
$\\{|P_{0}\phi_{\psi_{n},\eta_{0}}|\geq\delta\\}$. We then write

$P_{0}\phi_{\psi_{n},\eta_{0}}=-(\mathbb{P}_{n}-P_{0})\phi_{\psi_{n},\eta_{n}}+\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}-P_{0}\left(\phi_{\psi_{n},\eta_{n}}-\phi_{\psi_{n},\eta_{0}}\right).$

By the assumption that $\phi_{\psi_{n},\eta_{n}}$ is contained in a
$P_{0}$-Glivenko Cantelli class with probability tending to one,
$(\mathbb{P}_{n}-P_{0})\phi_{\psi_{n},\eta_{n}}=o_{P_{0}^{*}}(1)$, and
$\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}=o_{P_{0}^{*}}(1)$ by assumption. Since
$\psi_{n}=O_{P_{0}^{*}}(1)$, for all $\gamma>0$ there exists $M$ such that
$P_{0}^{*}(|\psi_{n}|>M)<\gamma$ for all $n$ large enough. Thus,

$P_{0}\left(\left|P_{0}\left(\phi_{\psi_{n},\eta_{n}}-\phi_{\psi_{n},\eta_{0}}\right)\right|\geq\delta\right)\leq
P_{0}\left(\sup_{|\psi|\leq
M}\left|P_{0}\left(\phi_{\psi,\eta_{n}}-\phi_{\psi,\eta_{0}}\right)\right|\geq\delta\right)+P_{0}(|\psi_{n}|>M),$

which is less than $2\gamma$ for all $n$ large enough since $\sup_{|\psi|\leq
M}\left|P_{0}(\phi_{\psi,\eta_{n}}-\phi_{\psi,\eta_{0}})\right|=o_{P_{0}^{*}}(1)$
by assumption. Therefore, for any $\varepsilon>0$,

$\displaystyle
P_{0}^{*}\left(\left|\psi_{n}-\psi_{0}\right|\geq\varepsilon\right)\leq
P_{0}^{*}\left(\left|P_{0}\phi_{\psi_{n},\eta_{0}}\right|\geq\delta\right)\longrightarrow
0.$

We show that $\psi_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\psi_{0}$ using
a similar method. By the well-separated condition, the event
$\\{|\psi_{n}^{*}-\psi_{0}|\geq\varepsilon\\}$ is contained in the event
$\\{|P_{0}\phi_{\psi_{n}^{*},\eta_{0}}|\geq\delta\\}$. We then write

$P_{0}\phi_{\psi_{n}^{*},\eta_{0}}=-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{\psi_{n}^{*},\eta_{n}^{*}}-(\hat{P}_{n}-P_{0})\phi_{\psi_{n}^{*},\eta_{n}^{*}}+\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-P_{0}\left(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{n}^{*},\eta_{0}}\right).$

By the assumption that $\phi_{\psi_{n}^{*},\eta_{n}^{*}}$ is contained in a
$P_{0}$-Glivenko Cantelli class $\mathscr{F}$ with conditional probability
tending to one,
$(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{W}^{*}}(1)$
and
$\left|(\hat{P}_{n}-P_{0})\phi_{\psi_{n}^{*},\eta_{n}^{*}}\right|\leq\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}}=o_{P_{W}^{*}}(1)$,
and $\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{W}^{*}}(1)$ by
assumption. Since $\psi_{n}^{*}=O_{P_{W}^{*}}(1)$, for all $\gamma>0$ there
exists $M$ such that
$P_{0}^{*}(P_{W}^{*}(|\psi_{n}^{*}|>M)\geq\gamma/2)=o(1)$. Thus,

$\displaystyle
P_{0}^{*}\left(P_{W}^{*}\left(\left|P_{0}\left(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{n}^{*},\eta_{0}}\right)\right|\geq\delta\right)\geq\gamma\right)$
$\displaystyle\leq P_{0}^{*}\left(P_{W}^{*}\left(\sup_{|\psi|\leq
M}\left|P_{0}\left(\phi_{\psi,\eta_{n}^{*}}-\phi_{\psi,\eta_{0}}\right)\right|\geq\delta\right)>\gamma/2\right)$
$\displaystyle\qquad+P_{0}^{*}\left(P_{W}^{*}(|\psi_{n}^{*}|>M)>\gamma/2\right),$

and both terms are $o(1)$. The result follows. $\hfill\blacksquare$

See 4.2

###### Proof of Lemma 4.2.

We first show that the assumptions imply that
$\psi_{n}\xrightarrow{P_{0}^{*}}\psi_{0}$. By assumption, $\psi_{0}$ is a
well-separated solution of $\psi\mapsto G_{0,\eta_{0}}(\psi)$ and
$\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}=o_{P_{0}^{*}}(1)$. By condition (A1),
$\phi_{n}:=\phi_{\psi_{n},\eta_{n}}$ is contained in a $P_{0}$-Donsker class
with probability tending to one, which implies that it is contained in a
$P_{0}$-Glivenko Cantelli class with probability tending to one. By adding and
subtracting terms, we can write

$\displaystyle P_{0}\left(\phi_{\psi,\eta_{n}}-\phi_{\psi,\eta_{0}}\right)$
$\displaystyle=\Gamma_{0,\eta_{n}}(\psi)-\Gamma_{0,\eta_{0}}(\psi)+P_{0}\phi_{\psi_{0},\eta_{n}}+\left[G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right]\left(\psi-\psi_{0}\right)$

By assumption, $\sup_{|\psi|\leq
M}\left|\Gamma_{0,\eta_{n}}(\psi)-\Gamma_{0,\eta_{0}}(\psi)\right|=o_{P_{0}^{*}}(1)$,
$P_{0}\phi_{\psi_{0},\eta_{n}}=o_{P_{0}^{*}}(1)$, and

$\sup_{|\psi|\leq
M}\left|\left[G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right]\left(\psi-\psi_{0}\right)\right|=o_{P_{0}^{*}}(1)$

for every $M>0$ since $\eta_{n}\xrightarrow{P_{0}^{*}}\eta_{0}$. Finally,
$\psi_{n}=O_{P_{0}^{*}}(1)$ by assumption. Thus, the conditions of Lemma C.1
are satisfied, so $\psi_{n}\xrightarrow{P_{0}^{*}}\psi_{0}$.

We next show that $\psi_{n}-\psi_{0}=O_{P_{0}^{*}}(n^{-1/2})$ under the
assumptions of the lemma. We have

$n^{1/2}(\psi_{n}-\psi_{0})=\mathbb{G}_{n}\phi_{\psi_{n},\eta_{n}}-n^{1/2}\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}+n^{1/2}P_{0}\phi_{\psi_{0},\eta_{n}}+n^{1/2}\left[P_{0}(\phi_{\psi_{n},\eta_{n}}-\phi_{\psi_{0},\eta_{n}})+(\psi_{n}-\psi_{0})\right].$

Now, $\mathbb{G}_{n}\phi_{\psi_{n},\eta_{n}}=O_{P_{0}^{*}}(1)$ by (A1), and
$n^{1/2}\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}$ and
$n^{1/2}P_{0}\phi_{\psi_{0},\eta_{n}}$ are both $o_{P_{0}^{*}}(1)$ by
assumption. Since
$\left\|{\eta_{n}-\eta_{0}}\right\|_{\mathscr{H}}=o_{P_{0}^{*}}(1)$ by
assumption, with probability tending to one it holds that

$\displaystyle\left|P_{0}(\phi_{\psi_{n},\eta_{n}}-\phi_{\psi_{0},\eta_{n}})+(\psi_{n}-\psi_{0})\right|$
$\displaystyle=\left|G_{0,\eta_{n}}(\psi_{n})-G_{0,\eta_{n}}(\psi_{0})+(\psi_{n}-\psi_{0})\right|$
$\displaystyle\leq\left|\Gamma_{0,\eta_{n}}(\psi_{n})\right|+\left|G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right||\psi_{n}-\psi_{0}|$
$\displaystyle\leq\sup_{\eta:\|\eta-\eta_{0}\|_{\mathscr{H}}<\delta}\left|\Gamma_{0,\eta}(\psi_{n})\right|+\left|G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right|\left|\psi_{n}-\psi_{0}\right|.$

Now since $\|\eta_{n}-\eta_{0}\|_{\mathscr{H}}=o_{P_{0}^{*}}(1)$ and
$G_{0,\eta}(\psi_{0})$ is continuous in $\eta$ at $\eta_{0}$ with
$G_{0,\eta}(\psi_{0})=-1$,
$\left|G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right|=o_{P_{0}^{*}}(1)$. Since
$\psi_{n}\xrightarrow{P_{0}^{*}}\psi_{0}$ as established above, we then have
$\sup_{\eta:\|\eta-\eta_{0}\|_{\mathscr{H}}<\delta}\left|\Gamma_{0,\eta}(\psi_{n})\right|=o_{P_{0}^{*}}(|\psi_{n}-\psi_{0}|)$
by the assumed differentiability. Hence, we have

$\displaystyle
n^{1/2}|\psi_{n}-\psi_{0}|=O_{P_{0}^{*}}(1)+n^{1/2}o_{P_{0}^{*}}(|\psi_{n}-\psi_{0}|),$

which implies that $\psi_{n}-\psi_{0}=O_{P_{0}^{*}}(n^{-1/2})$.

Now since $G_{0,\eta_{n}}^{\prime}(\psi_{0})+1=o_{P_{0}^{*}}(1)$,
$P_{0}\phi_{\psi_{0},\eta_{n}}=o_{P_{0}^{*}}(n^{-1/2})$, and
$\psi_{n}-\psi_{0}=O_{P_{0}^{*}}(n^{-1/2})$, we have

$\displaystyle\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}$
$\displaystyle=\left[P_{0}(\phi_{\psi_{n},\eta_{n}}-\phi_{\psi_{0},\eta_{n}})-G_{0,\eta_{n}}^{\prime}(\psi_{0})(\psi_{n}-\psi_{0})\right]$
$\displaystyle\qquad+\left[G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right]\left(\psi_{n}-\psi_{0}\right)+P_{0}\phi_{\psi_{0},\eta_{n}}$
$\displaystyle=\Gamma_{0,\eta_{n}}(\psi_{n})+o_{P_{0}^{*}}(n^{-1/2}).$

The first term is $o_{P_{0}^{*}}(n^{-1/2})$ by the differentiability
assumption since $\|\eta_{n}-\eta_{0}\|_{\mathscr{H}}=o_{P_{0}^{*}}(1)$ and
$\psi_{n}-\psi_{0}=O_{P_{0}^{*}}(n^{-1/2})$ as discussed above. The result
follows. $\hfill\blacksquare$

See 4.3

###### Proof of Lemma 4.3.

We first show that the assumptions imply that
$\psi_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\psi_{0}$. By assumption,
$\psi_{0}$ is a well-separated solution of $\psi\mapsto G_{0,\eta_{0}}(\psi)$
and $\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{0}^{*}}(1)$. By
condition (A1), $\phi_{n}^{*}:=\phi_{\psi_{n}^{*},\eta_{n}^{*}}$ is contained
in a $P_{0}$-Donsker class with probability tending to one, which implies that
it is contained in a $P_{0}$-Glivenko Cantelli class with probability tending
to one. Adding and subtracting terms, we can write

$\displaystyle
P_{0}\left(\phi_{\psi,\eta_{n}^{*}}-\phi_{\psi,\eta_{0}}\right)$
$\displaystyle=\Gamma_{0,\eta_{n}^{*}}(\psi)-\Gamma_{0,\eta_{0}}(\psi)+P_{0}\phi_{\psi_{0},\eta_{n}^{*}}+\left[G_{0,\eta_{n}^{*}}^{\prime}(\psi_{0})+1\right]\left(\psi-\psi_{0}\right)$

By assumption, $\sup_{|\psi|\leq
M}\left|\Gamma_{0,\eta_{n}^{*}}(\psi)-\Gamma_{0,\eta_{0}}(\psi_{0})\right|=o_{P_{W}^{*}}(1)$,
$P_{0}\phi_{\psi_{0},\eta_{n}^{*}}=o_{P_{0}^{*}}(1)$, and

$\sup_{|\psi|\leq
M}\left|\left[G_{0,\eta_{n}^{*}}^{\prime}(\psi_{0})+1\right]\left(\psi-\psi_{0}\right)\right|=o_{P_{0}^{*}}(1)$

for every $M>0$ since
$\eta_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\eta_{0}$. Finally,
$\psi_{n}^{*}=O_{P_{W}^{*}}(1)$ and
$\|\mathbb{P}_{n}-P_{0}\|_{\mathscr{F}}=o_{P_{W}^{*}}(1)$ by condition (A1).
Thus, the conditions of Lemma C.1 are satisfied, so
$\psi_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\psi_{0}$.

We next show that
$|P_{0}(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{0},\eta_{n}^{*}})+(\psi_{n}^{*}-\psi_{0})|=o_{P_{W}^{*}}(|\psi_{n}^{*}-\psi_{0}|)$
under the assumptions of the lemma. Since
$\|\eta_{n}^{*}-\eta_{0}\|_{\mathscr{H}}=o_{P_{W}^{*}}(1)$ by assumption, with
conditional probability tending to one it holds that

$\displaystyle\left|P_{0}(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{0},\eta_{n}^{*}})+(\psi_{n}^{*}-\psi_{0})\right|$
$\displaystyle=\left|G_{0,\eta_{n}^{*}}(\psi_{n}^{*})-G_{0,\eta_{n}^{*}}(\psi_{0})+(\psi_{n}^{*}-\psi_{0})\right|$
$\displaystyle\leq\left|\Gamma_{0,\eta_{n}^{*}}(\psi_{n}^{*})\right|+\left|G_{0,\eta_{n}^{*}}^{\prime}(\psi_{0})+1\right||\psi_{n}^{*}-\psi_{0}|$
$\displaystyle\leq\sup_{\eta:\|\eta-\eta_{0}\|_{\mathscr{H}}<\delta}\left|\Gamma_{0,\eta}(\psi_{n}^{*})\right|+\left|G_{0,\eta_{n}^{*}}^{\prime}(\psi_{0})+1\right|\left|\psi_{n}^{*}-\psi_{0}\right|.$

Now since $\|\eta_{n}^{*}-\eta_{0}\|_{\mathscr{H}}=o_{P_{W}^{*}}(1)$ and
$G_{0,\eta}(\psi_{0})$ is continuous in $\eta$ at $\eta_{0}$ with
$G_{0,\eta_{0}}(\psi_{0})=-1$,
$\left|G_{0,\eta_{n}^{*}}^{\prime}(\psi_{0})+1\right|=o_{P_{W}^{*}}(1)$. Since
$\psi_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\psi_{0}$ as shown above, we
also have
$\sup_{\eta:\|\eta-\eta_{0}\|_{\mathscr{H}}<\delta}\left|\Gamma_{0,\eta}(\psi_{n}^{*})\right|=o_{P_{W}^{*}}(|\psi_{n}^{*}-\psi_{0}|)$.
The result follows.

We now show that $\psi_{n}^{*}-\psi_{0}=O_{P_{W}^{*}}(n^{-1/2})$ under the
assumptions of the lemma. We have

$\displaystyle n^{1/2}(\psi_{n}^{*}-\psi_{0})$
$\displaystyle=\mathbb{G}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}+\mathbb{G}_{n}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-n^{1/2}\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}+n^{1/2}P_{0}\phi_{\psi_{0},\eta_{n}^{*}}$
$\displaystyle\qquad+n^{1/2}\left[P_{0}(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{0},\eta_{n}^{*}})+(\psi_{n}^{*}-\psi_{0})\right].$

Now, $\mathbb{G}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=O_{P_{W}^{*}}(1)$ by
(B1), $\mathbb{G}_{n}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=O_{P_{W}^{*}}(1)$ by
(A1), and $n^{1/2}\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}$ and
$n^{1/2}P_{0}\phi_{\psi_{0},\eta_{n}^{*}}$ are both $o_{P_{W}^{*}}(1)$ by
assumption. We therefore have

$\displaystyle n^{1/2}|\psi_{n}^{*}-\psi_{0}|$
$\displaystyle=O_{P_{W}^{*}}(1)+n^{1/2}\left|P_{0}(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{0},\eta_{n}^{*}})+(\psi_{n}^{*}-\psi_{0})\right|$
$\displaystyle=O_{P_{W}^{*}}(1)+n^{1/2}o_{P_{W}^{*}}(|\psi_{n}^{*}-\psi_{0}|),$

which implies that $\psi_{n}^{*}-\psi_{0}=O_{P_{W}^{*}}(n^{-1/2})$.

We now show the first statement of Lemma 4.3. We write

$\displaystyle\begin{split}\psi_{n}^{*}-\psi_{n}-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\phi_{\psi_{n}^{*},\eta_{n}^{*}}&=[\psi_{n}^{*}-\psi_{0}+P_{0}\phi_{\psi_{n}^{*},\eta_{n}^{*}}]-[\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}]\\\
&\qquad+(\mathbb{P}_{n}-P_{0})(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{n},\eta_{n}})+\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}-\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}.\end{split}$
(22)

By Lemma 4.2, we have
$\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}=o_{P_{0}^{*}}(n^{-1/2})$. By
the definitions of $\psi_{n}$ and $\psi_{n}^{*}$, we have
$\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}=o_{P_{W}^{*}}(n^{-1/2})$ and
$\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$.
By conditions (A1) and (A2) and Lemma 19.24 of van der Vaart, (2000), we have
$(\mathbb{P}_{n}-P_{0})(\phi_{\psi_{n},\eta_{n}}-\phi_{\psi_{0},\eta_{0}})=o_{P_{0}^{*}}(n^{-1/2})$.
Similarly, by (A1)(b), (B1)(a), (B2) and a minor modification of Lemma 19.24
van der Vaart, (2000), we have
$(\mathbb{P}_{n}-P_{0})(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{0},\eta_{0}})=o_{P_{W}^{*}}(n^{-1/2})$.
Hence,
$(\mathbb{P}_{n}-P_{0})(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{n},\eta_{n}})=o_{P_{W}^{*}}(n^{-1/2})$.
Finally, since $G_{0,\eta_{n}^{*}}^{\prime}(\psi_{0})+1=o_{P_{W}^{*}}(1)$,
$P_{0}\phi_{\psi_{0},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$, and
$\psi_{n}^{*}-\psi_{0}=O_{P_{W}^{*}}(n^{-1/2})$, we have

$\displaystyle|\psi_{n}^{*}-\psi_{0}+P_{0}\phi_{\psi_{n}^{*},\eta_{n}^{*}}|$
$\displaystyle\leq\left|P_{0}(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{0},\eta_{n}^{*}})+(\psi_{n}^{*}-\psi_{0})\right|+\left|{P_{0}\phi_{\psi_{0},\eta_{n}^{*}}}\right|$
$\displaystyle=o_{P_{W}^{*}}(|\psi_{n}^{*}-\psi_{0}|)+o_{P_{W}^{*}}(n^{-1/2})$
$\displaystyle=o_{P_{W}^{*}}(n^{-1/2}).$

The result follows.

We next show the second statement in Lemma 4.3. Since
$R_{n}=\psi_{n}-\psi_{0}-(\mathbb{P}_{n}-P_{0})\phi_{\psi_{n},\eta_{n}}$ and
$R_{n}^{*}=\psi_{n}^{*}-\psi_{n}-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\phi_{\psi_{n}^{*},\eta_{n}^{*}}$,
by adding and subtracting terms, we have

$\displaystyle R_{n}^{*}-R_{n}$
$\displaystyle=\left(\psi_{n}^{*}-\psi_{0}+P_{0}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-P_{0}\phi_{\psi_{0},\eta_{n}^{*}}\right)-2\left(\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}-P_{0}\phi_{\psi_{0},\eta_{n}}\right)$
$\displaystyle\qquad+(P_{0}\phi_{\psi_{0},\eta_{n}^{*}}-2P_{0}\phi_{\psi_{0},\eta_{n}})+\left(\mathbb{P}_{n}-P_{0}\right)\left(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{n},\eta_{n}}\right)+\left(2\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}-\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}\right).$

As we showed above,
$\psi_{n}^{*}-\psi_{0}+P_{0}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-P_{0}\phi_{\psi_{0},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$
and
$(\mathbb{P}_{n}-P_{0})(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\psi_{n},\eta_{n}})=o_{P_{W}^{*}}(n^{-1/2})$.
By assumption,
$P_{0}\phi_{\psi_{0},\eta_{n}^{*}}-2P_{0}\phi_{\psi_{0},\eta_{n}}=o_{P_{W}^{*}}(n^{-1/2})$.
By the definitions of $\psi_{n}^{*}$ and $\psi_{n}$,
$2\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}$ and
$\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}$ are both
$o_{P_{W}^{*}}(n^{-1/2})$. Therefore,

$\displaystyle R_{n}^{*}-R_{n}$
$\displaystyle=-2\left(\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}-P_{0}\phi_{\psi_{0},\eta_{n}}\right)+o_{P_{W}^{*}}(n^{-1/2})$
$\displaystyle=-2\Gamma_{0,\eta_{n}}(\psi_{n})-2\left[G_{0,\eta_{n}}^{\prime}(\psi_{0})+1\right](\psi_{n}-\psi_{0})+o_{P_{W}^{*}}(n^{-1/2}).$

The first two terms are $o_{P_{0}^{*}}(n^{-1/2})$ as discussed above. The
result follows.

$\hfill\blacksquare$

See 4.4

###### Proof of Lemma 4.4.

By the same argument as used in Lemma 4.3, and since
$\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}}=o_{P_{W}^{*}}(1)$ by assumption, we can
show that the conditions of Lemma C.1 are satisfied, so that
$\psi_{n}^{*}\underset{W}{\xrightarrow{P_{0}^{*}}}\psi_{0}$.

We recall that $\hat{\psi}_{n}:=T(\hat{\eta}_{n},\hat{P}_{n})$. By the same
argument as used in Lemma 4.3,
$P_{0}\phi_{\hat{\psi}_{n},\eta_{0}}=o_{P_{0}^{*}}(1)$ implies that
$\hat{\psi}_{n}-\psi_{0}=o_{P_{0}^{*}}(1)$. We write

$P_{0}\phi_{\hat{\psi}_{n},\eta_{0}}=\hat{P}_{n}\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}-\left(\hat{P}_{n}-P_{0}\right)\left(\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}\right)-P_{0}\left(\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}-\phi_{\hat{\psi}_{n},\eta_{0}}\right).$

By the definition of $\hat{\psi}_{n}$, we have
$\hat{P}_{n}\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}=0$. Under the assumptions
$\|\hat{P}_{n}-P_{0}\|_{\mathscr{F}}=o_{P_{0}^{*}}(1)$ and
$\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}$ is contained in $\mathscr{F}$ with
probability tending to one, we have
$(\hat{P}_{n}-P_{0})\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}=o_{P_{0}^{*}}(1)$.
Finally, since $\hat{\psi}_{n}=O_{P_{0}^{*}}(1)$ and $\sup_{|\psi|\leq
M}\left|\Gamma_{0,\hat{\eta}_{n}}(\psi)-\Gamma_{0,\eta_{0}}(\psi)\right|=o_{P_{0}^{*}}(1)$
for every $M>0$, $P_{0}\phi_{\psi_{0},\hat{\eta}_{n}}=o_{P_{0}^{*}}(1)$, and
$\|\hat{\eta}_{n}-\eta_{0}\|_{\mathscr{H}}=o_{P_{0}^{*}}(1)$, by the same
argument as in Lemma 4.2,
$P_{0}\left(\phi_{\hat{\psi}_{n},\hat{\eta}_{n}}-\phi_{\hat{\psi}_{n},\eta_{0}}\right)=o_{P_{0}^{*}}(1)$.
Thus, $P_{0}\phi_{\hat{\psi}_{n},\eta_{0}}=o_{P_{0}^{*}}(1)$, so that
$\hat{\psi}_{n}\xrightarrow{P_{0}^{*}}\psi_{0}$. Hence, we also have
$\psi_{n}^{*}-\hat{\psi}_{n}=(\psi_{n}^{*}-\psi_{0})-(\hat{\psi}_{n}-\psi_{0})\underset{W}{\xrightarrow{P_{0}^{*}}}0$.

We now show that
$\hat{P}_{n}\left(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\hat{\psi}_{n},\eta_{n}^{*}}\right)+(\psi_{n}^{*}-\hat{\psi}_{n})=o_{P_{W}^{*}}(\psi_{n}^{*}-\hat{\psi}_{n})$
under the assumptions of the lemma. With conditional probability tending to
one, it holds that

$\displaystyle\left|{\hat{P}_{n}\big{(}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\hat{\psi}_{n},\eta_{n}^{*}}\big{)}+(\psi_{n}^{*}-\hat{\psi}_{n})}\right|$
$\displaystyle\qquad=\left|{\hat{G}_{n,\eta_{n}^{*}}(\psi_{n}^{*})-\hat{G}_{n,\eta_{n}^{*}}(\hat{\psi}_{n})+(\psi_{n}^{*}-\hat{\psi}_{n})}\right|$
$\displaystyle\qquad\leq\left|{\hat{G}_{n,\eta_{n}^{*}}(\psi_{n}^{*})-\hat{G}_{n,\eta_{n}^{*}}(\hat{\psi}_{n})-\hat{G}_{n,\eta_{n}^{*}}^{\prime}(\hat{\psi}_{n})(\psi_{n}^{*}-\hat{\psi}_{n})}\right|+\left|{\hat{G}_{n,\eta_{n}^{*}}^{\prime}(\hat{\psi}_{n})+1}\right|\left|{\psi_{n}^{*}-\hat{\psi}_{n}}\right|$
$\displaystyle\qquad\leq\sup_{\eta:\left\|{\eta-\hat{\eta}_{n}}\right\|_{\mathscr{H}}<\delta}\left|{\hat{\Gamma}_{n,\eta}(\psi_{n}^{*})}\right|+\left|{\hat{G}_{n,\eta_{n}^{*}}^{\prime}(\hat{\psi}_{n})+1}\right|\left|{\psi_{n}^{*}-\hat{\psi}_{n}}\right|$

Since by assumption
$\hat{G}_{n,\eta_{n}^{*}}^{\prime}(\psi_{n})+1=o_{P_{W}^{*}}(1)$, the second
term on the right hand side of previous display is
$o_{P_{W}^{*}}(\psi_{n}^{*}-\hat{\psi}_{n})$. The result follows.

We next show that $\psi_{n}^{*}-\hat{\psi}_{n}=O_{P_{W}^{*}}(n^{-1/2})$ under
the assumptions of the lemma. We have

$\displaystyle n^{1/2}(\psi_{n}^{*}-\hat{\psi}_{n})$
$\displaystyle=\mathbb{G}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-n^{1/2}\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}+n^{1/2}\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}+n^{1/2}\left[\hat{P}_{n}\left(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\hat{\psi}_{n},\eta_{n}^{*}}\right)+\psi_{n}^{*}-\hat{\psi}_{n}\right].$

Now, $\mathbb{G}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=O_{P_{W}^{*}}(1)$
because (B1) holds for $\phi_{n}^{*}=\phi_{\psi_{n}^{*},\eta_{n}^{*}}$ by
assumption, and $n^{1/2}\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}$
and $n^{1/2}\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}$ are both
$o_{P_{W}^{*}}(1)$ by assumption. As discussed above,
$\hat{P}_{n}\left(\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\phi_{\hat{\psi}_{n},\eta_{n}^{*}}\right)+\psi_{n}^{*}-\hat{\psi}_{n}=o_{P_{W}^{*}}(\psi_{n}^{*}-\hat{\psi}_{n})$.
Hence, we have

$\displaystyle
n^{1/2}|\psi_{n}^{*}-\hat{\psi}_{n}|=O_{P_{W}^{*}}(1)+n^{1/2}o_{P_{W}^{*}}(|\psi_{n}^{*}-\hat{\psi}_{n}|),$

which implies that $\psi_{n}^{*}-\hat{\psi}_{n}=O_{P_{W}^{*}}(n^{-1/2})$.

We can now show the first statement of Lemma 4.4. We recall that
$\psi_{n}^{\circ}:=T(\eta_{n},\hat{P}_{n})$. We can write

$\displaystyle R_{n}^{*}$
$\displaystyle=\psi_{n}^{*}-\psi_{n}^{\circ}-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{\psi_{n}^{*},\eta_{n}^{*}}$
$\displaystyle=\left(\psi_{n}^{*}-\hat{\psi}_{n}+\hat{P}_{n}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}\right)-\left(\psi_{n}^{\circ}-\hat{\psi}_{n}\right)+\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}-\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}};$

We have $\psi_{n}^{\circ}-\hat{\psi}_{n}=o_{P_{0}^{*}}(n^{-1/2})$ and
$\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$ by
assumption,
$\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$
by definition of $\psi_{n}^{*}$, and

$\psi_{n}^{*}-\hat{\psi}_{n}+\hat{P}_{n}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}=o_{P_{W}^{*}}\left(\psi_{n}^{*}-\hat{\psi}_{n}\right)=o_{P_{W}^{*}}(n^{-1/2})$

by the derivations above. Hence, $R_{n}^{*}=o_{P_{W}^{*}}(n^{-1/2})$.

Lastly, we show the second statement in Lemma 4.4. Recall that
$R_{n}=\psi_{n}-\psi_{0}-(\mathbb{P}_{n}-P_{0})\phi_{\psi_{n},\eta_{n}}$ and
$R_{n}^{*}=\psi_{n}^{*}-\psi_{n}^{\circ}-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{\psi_{n}^{*},\eta_{n}^{*}}$.
By adding and subtracting terms, we can write

$\displaystyle R_{n}^{*}-R_{n}$
$\displaystyle=\left(\psi_{n}^{*}-\hat{\psi}_{n}+\hat{P}_{n}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}\right)-\left(\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}-P_{0}\phi_{\psi_{0},\eta_{n}}\right)$
$\displaystyle\qquad+\left(\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}-P_{0}\phi_{\psi_{0},\eta_{n}}\right)+\left(\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}-\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}\right)+\left(\hat{\psi}_{n}-\psi_{n}^{\circ}\right).$

We have
$\psi_{n}^{*}-\hat{\psi}_{n}+\hat{P}_{n}\phi_{\psi_{n}^{*},\eta_{n}^{*}}-\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$
by the derivations above,
$\psi_{n}-\psi_{0}+P_{0}\phi_{\psi_{n},\eta_{n}}-P_{0}\phi_{\psi_{0},\eta_{n}}=o_{P_{0}^{*}}(n^{-1/2})$
by Lemma 4.2, $\hat{\psi}_{n}-\psi_{n}^{\circ}=o_{P_{W}^{*}}(n^{-1/2})$ and
$\hat{P}_{n}\phi_{\hat{\psi}_{n},\eta_{n}^{*}}-P_{0}\phi_{\psi_{0},\eta_{n}}=o_{P_{W}^{*}}(n^{-1/2})$
by assumption, and
$\mathbb{P}_{n}\phi_{\psi_{n},\eta_{n}}-\mathbb{P}_{n}^{*}\phi_{\psi_{n}^{*},\eta_{n}^{*}}=o_{P_{W}^{*}}(n^{-1/2})$
by the definitions of $\psi_{n}$ and $\psi_{n}^{*}$. $\hfill\blacksquare$

##  Supplement D Proof of results in Section 5

See 5.1

###### Proof of 5.1.

We use 3.1. Condition (A1) holds because $\eta_{n}$ falls in a $P_{0}$-Donsker
class by assumption. For any $\varepsilon>0$, by the triangle and
Cauchy–Schwarz inequalities, we have

$\displaystyle
P_{0}^{*}\left(\left\|{\phi_{n}-\phi_{0}}\right\|_{L_{2}(P_{0})}\geq\varepsilon\right)$
$\displaystyle\qquad\leq
P_{0}^{*}\left(2\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(P_{0})}\geq\varepsilon/2\right)+P_{0}^{*}\left(2\left|{\int\eta_{n}^{2}-\int\eta_{0}^{2}}\right|\geq\varepsilon/2\right)$
$\displaystyle\qquad\leq
P_{0}^{*}\left(2M\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{0}^{*}\left(2\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}\left\|{\eta_{n}+\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)$
$\displaystyle\qquad\leq
P_{0}^{*}\left(2M\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{0}^{*}\left(4M\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{0}^{*}\left(\left\|{\eta_{n}}\right\|_{L_{2}(\lambda)}\geq
M\right).$

Each term on the right-hand side converges to 0, which implies condition (A2).

For condition (A3) for $\psi_{n,1}$, we have

$\displaystyle
T_{1}(\eta_{n},\mathbb{P}_{n})-T_{1}(\eta_{0},P_{0})-(\mathbb{P}_{n}-P_{0})\phi_{n}$
$\displaystyle=\left(2\mathbb{P}_{n}\eta_{n}-\int\eta_{n}^{2}\right)-\int\eta_{0}^{2}-2(\mathbb{P}_{n}-P_{0})\eta_{n}$
$\displaystyle=-\int(\eta_{n}-\eta_{0})^{2}$
$\displaystyle=-\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}^{2}$

which is $o_{P_{0}^{*}}(n^{1/2})$ by assumption. For $\psi_{n,2}$ and
$\psi_{n,3}$, we have

$\displaystyle
T_{2}(\eta_{n},\mathbb{P}_{n})-T_{2}(\eta_{0},P_{0})-(\mathbb{P}_{n}-P_{0})\phi_{n}$
$\displaystyle=\int\eta_{n}^{2}-\int\eta_{0}^{2}-2(\mathbb{P}_{n}-P_{0})\eta_{n}$
$\displaystyle=-\int(\eta_{n}-\eta_{0})^{2}+2\left[\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}\right]$
$\displaystyle
T_{3}(\eta_{n},\mathbb{P}_{n})-T_{3}(\eta_{0},P_{0})-(\mathbb{P}_{n}-P_{0})\phi_{n}$
$\displaystyle=\mathbb{P}_{n}\eta_{n}-P_{0}\eta_{0}-2(\mathbb{P}_{n}-P_{0})\eta_{n}$
$\displaystyle=-\int(\eta_{n}-\eta_{0})^{2}+\left[\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}\right],$

which are both $o_{P_{0}^{*}}(n^{-1/2})$ when
$\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}=o_{P_{0}^{*}}(n^{-1/2})$ and
$\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}^{2}=o_{P_{0}^{*}}(n^{-1/4})$.
$\hfill\blacksquare$

See 5.2

###### Proof of 5.2.

We use 3.4. Condition (B1) holds by Lemma 3.6 because $\eta_{n}^{*}$ is
contained in a $P_{0}$-Donsker class with $P_{W}^{*}$ probability tending to
one. For any $\varepsilon>0$, by the triangle and Cauchy–Schwarz inequalities,
we have

$\displaystyle
P_{W}^{*}\left(\left\|{\phi_{n}^{*}-\phi_{0}}\right\|_{L_{2}(P_{0})}\geq\varepsilon\right)$
$\displaystyle\qquad\leq
P_{W}^{*}\left(2\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(P_{0})}\geq\varepsilon/2\right)+P_{W}^{*}\left(2\left|{\int\eta_{n}^{*2}-\int\eta_{0}^{2}}\right|\geq\varepsilon/2\right)$
$\displaystyle\qquad\leq
P_{W}^{*}\left(2M\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{W}^{*}\left(2\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\left\|{\eta_{n}^{*}+\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)$
$\displaystyle\qquad\leq
P_{W}^{*}\left(2M\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{W}^{*}\left(4M\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{W}^{*}\left(\left\|{\eta_{n}^{*}}\right\|_{L_{2}(\lambda)}\geq
M\right),$

Each term on the right-hand side is $o_{P_{0}^{*}}(1)$, which implies
condition (B2).

For condition (B3) for $\psi_{n,1}^{*}$, we write

$\displaystyle\left|{T_{1}(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T_{1}(\eta_{n},\mathbb{P}_{n})-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\phi_{n}^{*}}\right|$
$\displaystyle\qquad=\left|{\left(2\mathbb{P}_{n}^{*}\eta_{n}^{*}-\int\eta_{n}^{*2}\right)-\left(2\mathbb{P}_{n}\eta_{n}-\int\eta_{n}^{2}\right)-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})2\eta_{n}^{*}}\right|$
$\displaystyle\qquad=\left|{-\int(\eta_{n}^{*}-\eta_{n})^{2}-2\int\eta_{n}(\eta_{n}^{*}-\eta_{n})+2(\mathbb{P}_{n}-P_{0})(\eta_{n}^{*}-\eta_{n})+2P_{0}(\eta_{n}^{*}-\eta_{n})}\right|$
$\displaystyle\qquad=\left|{-\int(\eta_{n}^{*}-\eta_{n})^{2}-2\int(\eta_{n}-\eta_{0})(\eta_{n}^{*}-\eta_{n})+2(\mathbb{P}_{n}-P_{0})(\eta_{n}^{*}-\eta_{n})}\right|$
$\displaystyle\qquad\leq\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}^{2}+2\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}+2n^{-1/2}\left|{\mathbb{G}_{n}(\eta_{n}^{*}-\eta_{n})}\right|.$

The first two terms are $o_{P_{W}^{*}}(n^{-1/2})$ by assumption. In addition,
$\mathbb{G}_{n}(\eta_{n}^{*}-\eta_{0})=o_{P_{W}^{*}}(1)$ because
$\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(P_{0})}=o_{P_{W}^{*}}(1)$ and
$\eta_{n}^{*}$ is contained in a $P_{0}$-Donsker class with
$P_{W}^{*}$-probability tending to one. Hence, condition (B3) holds for
$\psi_{n,1}^{*}$, and the conditional asymptotic linearity result for
$\psi_{n,1}^{*}$ follows.

For condition (B3) for $\psi_{n,2}^{*}$ and $\psi_{n,3}^{*}$, we have

$\displaystyle
T_{2}(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T_{2}(\eta_{n},\mathbb{P}_{n})-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\phi_{n}^{*}$
$\displaystyle=\int\eta_{n}^{*2}-\int\eta_{n}^{2}-2(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\eta_{n}^{*}$
$\displaystyle=-\int(\eta_{n}^{*}-\eta_{n})^{2}+2\left(\int\eta_{n}^{*2}-\mathbb{P}_{n}^{*}\eta_{n}^{*}\right)-2\left(\int\eta_{n}\eta_{n}^{*}-\mathbb{P}_{n}\eta_{n}^{*}\right)$
$\displaystyle
T_{3}(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T_{3}(\eta_{n},\mathbb{P}_{n})-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\phi_{n}^{*}$
$\displaystyle=\mathbb{P}_{n}^{*}\eta_{n}^{*}-\mathbb{P}_{n}\eta_{n}-2(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\eta_{n}^{*}$
$\displaystyle=-\int(\eta_{n}^{*}-\eta_{n})^{2}+\left(\int\eta_{n}^{*2}-\mathbb{P}_{n}^{*}\eta_{n}^{*}\right)+\left(\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}\right)$
$\displaystyle\qquad-2\left(\int\eta_{n}\eta_{n}^{*}-\mathbb{P}_{n}\eta_{n}^{*}\right).$

Now, $\int(\eta_{n}^{*}-\eta_{n})^{2}$ and
$\int\eta_{n}^{*2}-\mathbb{P}_{n}^{*}\eta_{n}^{*}$ are both
$o_{P_{W}^{*}}(n^{-1/2})$ by assumption, and
$\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}=o_{P_{0}^{*}}(n^{-1/2})$ by
assumption. In addition,

$\displaystyle\left|{\int\eta_{n}\eta_{n}^{*}-\mathbb{P}_{n}\eta_{n}^{*}}\right|=$
$\displaystyle\left|{\int(\eta_{n}-\eta_{0})(\eta_{n}^{*}-\eta_{n})-(\mathbb{P}_{n}-P_{0})(\eta_{n}^{*}-\eta_{n})+\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}}\right|$
$\displaystyle\leq\left\|{\eta_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}+n^{-1/2}\left|{\mathbb{G}_{n}(\eta_{n}^{*}-\eta_{n})}\right|+\left|\int\eta_{n}^{2}-\mathbb{P}_{n}\eta_{n}\right|,$

which is $o_{P_{W}^{*}}(n^{-1/2})$. Therefore, condition (B3) holds for
$\psi_{n,2}^{*}$ and $\psi_{n,3}^{*}$.

We now turn to the setting where $\eta_{n}$ and $\eta_{n}^{*}$ are kernel
density estimators using the same bandwidth $h$. We note that for any $x$,
since $\hat{P}_{n}$ is the empirical bootstrap,

$\displaystyle E_{W}^{*}\left[\eta_{n}^{*}(x)\right]$
$\displaystyle=E_{W}^{*}\left[\mathbb{P}_{n}^{*}h^{-d}K\left(\frac{X_{i}^{*}-x}{h}\right)\right]=\mathbb{P}_{n}h^{-d}K\left(\frac{X_{i}-x}{h}\right)=\eta_{n}(x).$

We therefore have

$\displaystyle
E_{W}^{*}\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}^{2}$
$\displaystyle=\int
E_{W}^{*}[\eta_{n}^{*}(x)-\eta_{n}(x)]^{2}\,\mathrm{d}x=\int\mathrm{Var}_{W}^{*}(\eta_{n}^{*}(x))\,\mathrm{d}x$
$\displaystyle=\int\frac{1}{nh^{2d}}\mathrm{Var}_{W}^{*}\left[K\left(\frac{X_{1}^{*}-x}{h}\right)\right]\,\mathrm{d}x$
$\displaystyle\leq\frac{1}{nh^{2d}}\int
E_{W}^{*}\left[K\left(\frac{X_{1}^{*}-x}{h}\right)\right]^{2}\,\mathrm{d}x$
$\displaystyle=\frac{1}{nh^{2d}}\int\mathbb{P}_{n}\left[K\left(\frac{X_{i}-x}{h}\right)\right]^{2}\,\mathrm{d}x$
$\displaystyle=\frac{1}{nh^{d}}\int K^{2}(t)\,\mathrm{d}t.$

Hence, by Chebyshev’s inequality, for any $\varepsilon>0$, we have

$\displaystyle
P_{W}^{*}\left(n^{1/4}\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}\geq\varepsilon\right)$
$\displaystyle\leq
n^{1/2}E_{W}^{*}\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}^{2}/\varepsilon^{2}$
$\displaystyle\leq\left(nh^{2d}\right)^{-1/2}\varepsilon^{-2}\int
K^{2}(t)\,\mathrm{d}t,$

which goes to zero since $nh^{2d}\longrightarrow\infty$. Therefore
$\left\|{\eta_{n}^{*}-\eta_{n}}\right\|_{L_{2}(\lambda)}=o_{P_{W}^{*}}(n^{-1/4})$.
$\hfill\blacksquare$

The proof of 5.3 relies on technical lemmas that are stated and proved in
Appendix E.

See 5.3

###### Proof of 5.3.

Since $\phi_{P}=2(\eta_{P}-\psi_{P})$, we then have
$S_{n}^{*}=(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})(\phi_{n}^{*}-\phi_{0})=2(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})(\eta_{n}^{*}-\eta_{0})$
and $S_{n}=2(\mathbb{P}_{n}-P_{0})(\eta_{n}-\eta_{0})$. By Lemma E.7, we have
$S_{n}^{*}-S_{n}=o_{P_{W}^{*}}(n^{-1/2})$ under the stated assumptions.

We have
$R_{n}^{*}=\psi_{n}^{*}-\psi_{n}-(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\phi_{n}^{*}=\psi_{n}^{*}-\psi_{n}-2(\mathbb{P}_{n}^{*}-\mathbb{P}_{n})\eta_{n}^{*}$.
For the one-step estimator $\psi_{n,1}^{*}$, we have
$R_{n}^{*}=-\int(\eta_{n}^{*}-\eta_{0})^{2}+\int(\eta_{n}-\eta_{0})^{2}+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})$
and $R_{n}=-\int(\eta_{n}-\eta_{0})^{2}$. Hence

$R_{n}^{*}-R_{n}=-\int(\eta_{n}^{*}-\eta_{0})^{2}+2\int(\eta_{n}-\eta_{0})^{2}+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n}).$

If $nh^{4m}\longrightarrow 0$ and $nh^{d}\longrightarrow\infty$ hold, then
$\int(\eta_{n}^{*}-\eta_{0})^{2}-2\int(\eta_{n}-\eta_{0})^{2}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma E.8, and
$(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})=o_{P_{W}^{*}}(n^{-1/2})$ by
Lemma E.7. This implies bootstrap percentile confidence intervals based on
$\psi_{n,1}^{*}$ are asymptotically valid by 3.9.

For the plug-in estimator $\psi_{n,2}^{*}$, we have
$R_{n}^{*}=-\int(\eta_{n}^{*}-\eta_{0})^{2}+\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}+\mathbb{P}_{n}\phi_{n}+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})$
and $R_{n}=-\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}\phi_{n}$. Hence,

$R_{n}^{*}-R_{n}=-\int(\eta_{n}^{*}-\eta_{0})^{2}+2\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}+2\mathbb{P}_{n}\phi_{n}+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n}).$

Since $nh^{2m}\longrightarrow 0$ and $nh^{d}\longrightarrow\infty$ hold, we
have
$\int(\eta_{n}^{*}-\eta_{0})^{2}-2\int(\eta_{n}-\eta_{0})^{2}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma E.8,
$\mathbb{P}_{n}^{*}\phi_{n}^{*}-2\mathbb{P}_{n}\phi_{n}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma E.9 and
$(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})=o_{P_{W}^{*}}(n^{-1/2})$ by
Lemma E.7. This implies bootstrap percentile confidence intervals based on
$\psi_{n,2}^{*}$ are asymptotically valid by 3.9.

For the empirical mean plug-in estimator $\psi_{n,3}^{*}$, we have
$R_{n}^{*}=-\int(\eta_{n}^{*}-\eta_{0})^{2}+\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}/2+\mathbb{P}_{n}\phi_{n}/2+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})$
and $R_{n}=-\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}\phi_{n}/2$. Hence,

$R_{n}^{*}-R_{n}=-\int(\eta_{n}^{*}-\eta_{0})^{2}+2\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}/2+\mathbb{P}_{n}\phi_{n}+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n}).$

Since $nh^{2m}\longrightarrow 0$ and $nh^{d}\longrightarrow\infty$ hold, we
have
$\int(\eta_{n}^{*}-\eta_{0})^{2}=2\int(\eta_{n}-\eta_{0})^{2}+o_{P_{W}^{*}}(n^{-1/2})$
by Lemma E.8,
$\mathbb{P}_{n}^{*}\phi_{n}^{*}=2\mathbb{P}_{n}\phi_{n}+o_{P_{W}^{*}}(n^{-1/2})$
by Lemma E.9 and
$(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})=o_{P_{W}^{*}}(n^{-1/2})$ by
Lemma E.7. This implies bootstrap percentile confidence intervals based on
$\psi_{n,3}^{*}$ are asymptotically valid by 3.9. $\hfill\blacksquare$

See 5.4

###### Proof of Proposition 5.4.

We use 3.4. Condition (B1) holds because
$P_{W}(\eta_{n}^{*}\in\mathscr{F})\xrightarrow{P_{0}^{*}}1$ and the conditions
of Lemma 3.7 hold by assumption. For any $\varepsilon>0$, by the triangle and
Cauchy–Schwarz inequalities, we have

$\displaystyle
P_{W}^{*}\left(\left\|{\phi_{n}^{*}-\phi_{0}}\right\|_{L_{2}(P_{0})}\geq\varepsilon\right)$
$\displaystyle\qquad\leq
P_{W}^{*}\left(2\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(P_{0})}\geq\varepsilon/2\right)+P_{W}^{*}\left(2\left|{\int\eta_{n}^{*2}-\int\eta_{0}^{2}}\right|\geq\varepsilon/2\right)$
$\displaystyle\qquad\leq
P_{W}^{*}\left(2M\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{W}^{*}\left(2\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\left\|{\eta_{n}^{*}+\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)$
$\displaystyle\qquad\leq
P_{W}^{*}\left(2M\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{W}^{*}\left(4M\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\geq\varepsilon/2\right)+P_{W}^{*}\left(\left\|{\eta_{n}^{*}}\right\|_{L_{2}(\lambda)}\geq
M\right).$

Since
$\left\|{\eta_{n}^{*}-\eta_{0}}\right\|_{L_{2}(\lambda)}\leq\left\|{\eta_{n}^{*}-\hat{\eta}_{n}}\right\|_{L_{2}(\lambda)}+\left\|{\hat{\eta}_{n}-\eta_{0}}\right\|_{L_{2}(\lambda)}$
and
$\left\|{\eta_{n}^{*}}\right\|_{L_{2}(\lambda)}\leq\left\|{\eta_{n}^{*}}\right\|_{\infty}$,
each term on the right-hand side of the previous display is
$o_{P_{0}^{*}}(1)$, which implies condition (B2).

For condition (B3), we have

$\displaystyle
T_{1}(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T_{1}(\eta_{n},\hat{P}_{n})-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*}=$
$\displaystyle-\int(\eta_{n}^{*}-\hat{\eta}_{n})^{2}+\int(\hat{\eta}_{n}-\eta_{n})^{2},$

which is $o_{P_{W}^{*}}(n^{-1/2})$ by assumption. Hence, condition (B3) holds
for $\psi_{n,1}^{*}$. We next have

$\displaystyle
T_{2}(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T_{2}(\eta_{n},\hat{P}_{n})-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*}$
$\displaystyle=\int\eta_{n}^{*2}-\int\eta_{n}^{2}-2(\mathbb{P}_{n}^{*}-\hat{P}_{n})\eta_{n}^{*}$
$\displaystyle=-\int(\eta_{n}^{*}-\hat{\eta}_{n})^{2}+\int(\hat{\eta}_{n}-\eta_{n})^{2}+2\left[\int\eta_{n}^{*2}-\mathbb{P}_{n}^{*}\eta_{n}^{*}\right]$
$\displaystyle\qquad-2\left[\int\eta_{n}^{2}-\hat{P}_{n}\eta_{n}\right],$
$\displaystyle
T_{3}(\eta_{n}^{*},\mathbb{P}_{n}^{*})-T_{3}(\eta_{n},\hat{P}_{n})-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*}$
$\displaystyle=\mathbb{P}_{n}^{*}\eta_{n}^{*}-\hat{P}_{n}\eta_{n}-2(\mathbb{P}_{n}^{*}-\hat{P}_{n})\eta_{n}^{*}$
$\displaystyle=-\int(\eta_{n}^{*}-\hat{\eta}_{n})^{2}+\int(\hat{\eta}_{n}-\eta_{n})^{2}+\left[\int\eta_{n}^{*2}-\mathbb{P}_{n}^{*}\eta_{n}^{*}\right]$
$\displaystyle\qquad-\left[\int\eta_{n}^{2}-\hat{P}_{n}\eta_{n}\right],$

both of which are $o_{P_{W}^{*}}(n^{-1/2})$ by assumption. Hence, condition
(B3) holds for $\psi_{n,2}^{*}$ and $\psi_{n,3}^{*}$.

$\hfill\blacksquare$

The proof of 5.5 relies on technical lemmas that are stated and proved in
Appendix F. See 5.5

###### Proof of 5.5.

Since $\phi_{P}=2(\eta_{P}-\psi_{P})$, we then have
$S_{n}^{*}=(\mathbb{P}_{n}^{*}-\hat{P}_{n})(\phi_{n}^{*}-\phi_{0})=2(\mathbb{P}_{n}^{*}-\hat{P}_{n})(\eta_{n}^{*}-\eta_{0})$
and $S_{n}=2(\mathbb{P}_{n}-P_{0})(\eta_{n}-\eta_{0})$. By Lemma F.8, we have
$S_{n}^{*}-S_{n}=o_{P_{W}^{*}}(n^{-1/2})$ under the stated assumptions.

We have
$R_{n}^{*}=\psi_{n}^{*}-T(\eta_{n},\hat{P}_{n})-(\mathbb{P}_{n}^{*}-\hat{P}_{n})\phi_{n}^{*}=\psi_{n}^{*}-T(\eta_{n},\hat{P}_{n})-2(\mathbb{P}_{n}^{*}-\hat{P}_{n})\eta_{n}^{*}$.
For the one-step estimator $\psi_{n,1}^{*}$, we have
$R_{n}^{*}=-\int(\eta_{n}^{*}-\eta_{n})^{2}$ and
$R_{n}=-\int(\eta_{n}-\eta_{0})^{2}$. Hence

$R_{n}^{*}-R_{n}=-\int(\eta_{n}^{*}-\eta_{n})^{2}+\int(\eta_{n}-\eta_{0})^{2}$

If $nh^{4m}\longrightarrow 0$ and $nh^{2d}\longrightarrow\infty$ hold, then
$\int(\eta_{n}^{*}-\eta_{n})^{2}-\int(\eta_{n}-\eta_{0})^{2}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma F.9. This implies bootstrap percentile confidence intervals based on
$\psi_{n,1}^{*}$ are asymptotically valid by 3.9.

For the plug-in estimator $\psi_{n,2}^{*}$, we have
$R_{n}^{*}=-\int(\eta_{n}^{*}-\eta_{n})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}$
and $R_{n}=-\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}\phi_{n}$. Hence,

$R_{n}^{*}-R_{n}=-\int(\eta_{n}^{*}-\eta_{n})^{2}+\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}+\mathbb{P}_{n}\phi_{n}.$

Since $nh^{4m}\longrightarrow 0$ and $nh^{2d}\longrightarrow\infty$ hold, we
have
$\int(\eta_{n}^{*}-\eta_{n})^{2}-\int(\eta_{n}-\eta_{0})^{2}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma F.9,
$\mathbb{P}_{n}^{*}\phi_{n}^{*}-\mathbb{P}_{n}\phi_{n}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma F.10. This implies bootstrap percentile confidence intervals based on
$\psi_{n,2}^{*}$ are asymptotically valid by 3.9.

For the empirical mean plug-in estimator $\psi_{n,3}^{*}$, we have
$R_{n}^{*}=-\int(\eta_{n}^{*}-\eta_{n})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}/2$
and $R_{n}=-\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}\phi_{n}/2$. Hence,

$R_{n}^{*}-R_{n}=-\int(\eta_{n}^{*}-\eta_{n})^{2}+\int(\eta_{n}-\eta_{0})^{2}-\mathbb{P}_{n}^{*}\phi_{n}^{*}/2+\mathbb{P}_{n}\phi_{n}/2.$

Since $nh^{4m}\longrightarrow 0$ and $nh^{2d}\longrightarrow\infty$ hold, we
have
$\int(\eta_{n}^{*}-\eta_{n})^{2}-\int(\eta_{n}-\eta_{0})^{2}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma F.9,
$\mathbb{P}_{n}^{*}\phi_{n}^{*}-\mathbb{P}_{n}\phi_{n}=o_{P_{W}^{*}}(n^{-1/2})$
by Lemma F.10. This implies bootstrap percentile confidence intervals based on
$\psi_{n,3}^{*}$ are asymptotically valid by 3.9. $\hfill\blacksquare$

See 5.6

###### Proof of 5.6.

Condition (A1) holds by the Donsker assumptions, the boundedness assumptions
on $\mu_{n}$ and $g_{n}$, and $E(Y^{2})<\infty$ together with preservation of
the Donsker condition under Lipschitz transformations (e.g., Theorem 2.10.6 of
van der Vaart & Wellner,, 1996).

We next address condition (A2). By adding and subtracting terms, we have

$\displaystyle\phi_{n}(y,a,z)-\phi_{0}(y,a,z)$
$\displaystyle=(1-a)\left[y-\mu_{0}(z)\right]\frac{g_{n}(z)}{1-g_{n}(z)}\frac{1}{\pi_{n}\pi_{0}}\left[\pi_{0}-\pi_{n}\right]+a\left[\mu_{0}(z)-\psi_{0}\right]\frac{1}{\pi_{n}\pi_{0}}\left[\pi_{0}-\pi_{n}\right]$
$\displaystyle\qquad+\frac{(1-a)\left[y-\mu_{0}(z)\right]}{\pi_{0}\left[1-g_{n}(z)\right]\left[1-g_{0}(z)\right]}\left[g_{n}(z)-g_{0}(z)\right]$
$\displaystyle\qquad+\frac{1}{\pi_{n}}\left[\frac{a}{g_{n}(z)}-1\right]\frac{g_{n}(z)}{1-g_{n}(z)}\left[\mu_{n}(z)-\mu_{0}(z)\right]-\frac{a}{\pi_{n}}\left[\psi_{n}-\psi_{0}\right].$

Therefore, by the assumed bounds and the triangle inequality,
$\left\|{\phi_{n}-\phi_{0}}\right\|_{L_{2}(P_{0})}$ is bounded up to a
constant by

$\displaystyle|\pi_{n}-\pi_{0}|+\left\|{g_{n}-g_{0}}\right\|_{L_{2}(P_{0})}+\left\|{\mu_{n}-\mu_{0}}\right\|_{L_{2}(P_{0})}+|\psi_{n}-\psi_{0}|.$

We note that consistency of $\mu_{n}$ and $g_{n}$ implies consistency of
$\pi_{n}$ and $\psi_{n}$. Therefore, condition (A2) holds.

For condition (A3) for the one-step estimator
$\psi_{n,1}=T_{1}(\eta_{n},\mathbb{P}_{n})$, we have
$T_{1}(\eta_{n},\mathbb{P}_{n})-T_{1}(\eta_{0},P_{0})-(\mathbb{P}_{n}-P_{0})\phi_{n}=\psi_{n}-\psi_{0}+P_{0}\phi_{n}$.
Using the law of total expectation, a straightforward calculation shows that

$\displaystyle P_{0}\phi_{\psi,\eta}$
$\displaystyle=P_{0}\left\\{\frac{(g_{0}-g)(\mu-\mu_{0})}{\pi(1-g)}\right\\}+\frac{\pi_{0}}{\pi}(\psi_{0}-\psi).$

Therefore,

$\displaystyle\psi_{n}-\psi_{0}+P_{0}\phi_{n}$
$\displaystyle=P_{0}\left\\{\frac{(g_{0}-g_{n})(\mu_{n}-\mu_{0})}{\pi_{n}(1-g_{n})}\right\\}+\frac{\pi_{n}-\pi_{0}}{\pi_{n}}(\psi_{n}-\psi_{0}).$

These terms are both $o_{P_{0}^{*}}(n^{-1/2})$ by the assumed rates and
boundedness conditions. The result for the one-step estimator follows.

For the estimating equations-based estimator, we use Lemma 4.2. We have
already established that condition (A1) holds for
$\phi_{n}=\phi_{\psi_{n},\eta_{n}}$. We have that the population estimating
equation is $G_{0,\eta_{0}}(\psi)=\psi-\psi_{0}$, so
$G_{0,\eta_{0}}(\psi_{0})=0$ and
$\inf_{|\psi-\psi_{0}|>\delta}|G_{0,\eta_{0}}(\psi)|=\delta$, and hence
$\psi_{0}$ is a well-separated solution of the population estimating equation.
Furthermore, it is straightforward to see that the boundedness conditions and
the assumption that $E_{0}(Y^{2})<\infty$ imply that
$\psi_{n,2}=O_{P_{0}^{*}}(1)$. Next, defining
$G_{0,\eta}^{\prime}(\psi_{0})=-\pi_{0}\pi^{-1}$, we then have
$G_{0,\eta_{0}}^{\prime}(\psi_{0})=-1$ and
$\lim_{\eta\to\eta_{0}}G_{0,\eta}^{\prime}(\psi_{0})=\lim_{\eta\to\eta_{0}}-\pi_{0}\pi^{-1}=-1$.
In addition,

$\displaystyle\Gamma_{0,\eta}(\psi)$
$\displaystyle=G_{0,\eta}(\psi)-G_{0,\eta}(\psi_{0})-G_{0,\eta}^{\prime}(\psi_{0})(\psi-\psi_{0})$
$\displaystyle=P_{0}\phi_{\eta,\psi}-P_{0}\phi_{\eta,\psi_{0}}+\frac{\pi_{0}}{\pi}(\psi-\psi_{0})$
$\displaystyle=P_{0}\left\\{\frac{g_{0}(Z)-g(Z)}{\pi\left[1-g(Z)\right]}\left[\mu(Z)-\mu_{0}(Z)\right]\right\\}+\frac{\pi_{0}}{\pi}(\psi-\psi_{0})$
$\displaystyle\qquad-
P_{0}\left\\{\frac{g_{0}(Z)-g(Z)}{\pi\left[1-g(Z)\right]}\left[\mu(Z)-\mu_{0}(Z)\right]\right\\}-\frac{\pi_{0}}{\pi}(\psi_{0}-\psi_{0})$
$\displaystyle\qquad+\frac{\pi_{0}}{\pi}(\psi_{0}-\psi)$ $\displaystyle=0.$

Therefore, both conditions about $\Gamma_{0,\eta}$ hold. Finally,

$\displaystyle P_{0}\phi_{\psi_{0},\eta_{n}}$
$\displaystyle=P_{0}\left\\{\frac{g_{0}-g_{n}}{\pi_{n}\left[1-g_{n}\right]}\left[\mu_{n}-\mu_{0}\right]\right\\}-\frac{\pi_{0}}{\pi_{n}}(\psi_{0}-\psi_{0})$
$\displaystyle=P_{0}\left\\{\frac{g_{0}-g_{n}}{\pi_{n}\left[1-g_{n}\right]}\left[\mu_{n}-\mu_{0}\right]\right\\},$

which is $o_{P_{0}^{*}}(n^{-1/2})$ by assumption. Hence, condition (A3) holds
for the estimating equations-based estimator. $\hfill\blacksquare$

See 5.7

###### Proof of 5.7.

By Lemma 3.6, condition (B1) holds by the Donsker assumptions, the boundedness
assumptions on $\mu_{n}^{*}$ and $g_{n}^{*}$, and $E(Y^{2})<\infty$ together
with preservation of the Donsker condition under Lipschitz transformations.

As in the proof of 5.6, we can show that
$\left\|\phi_{n}^{*}-\phi_{0}\right\|_{L_{2}(P_{0})}$ is bounded up to a
constant by

$\left|\pi_{n}^{*}-\pi_{0}\right|+\left\|g_{n}^{*}-g_{0}\right\|_{L_{2}(P_{0})}+\left\|\mu_{n}^{*}-\mu_{0}\right\|_{L_{2}(P_{0})}+\left|\psi_{n}^{*}-\psi_{0}\right|.$

Therefore, condition (B2) holds under the conditional weak $L_{2}(P_{0})$
conditional consistency of $g_{n}^{*}$ and $\mu_{n}^{*}$, which also implies
weak conditional consistency of $\psi_{n}^{*}$ and $\pi_{n}^{*}$.

For condition (B3) for the one-step estimator
$\psi_{n,1}^{*}=T_{1}(\eta_{n}^{*},\mathbb{P}_{n}^{*})$, as in equation (8),
we have

$\displaystyle R_{n}^{*}$
$\displaystyle=\left[\psi_{n}^{*}-\psi_{0}+P_{0}\phi_{n}^{*}\right]-\left[\psi_{n}-\psi_{0}+P_{0}\phi_{n}\right]+(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})$

By the proof of 5.6,
$\psi_{n}-\psi_{0}+P_{0}\phi_{n}=o_{P_{0}^{*}}(n^{-1/2})$, and

$n^{1/2}(\mathbb{P}_{n}-P_{0})(\phi_{n}^{*}-\phi_{n})=\mathbb{G}_{n}\left(\phi_{n}^{*}-\phi_{0}\right)-\mathbb{G}_{n}\left(\phi_{n}-\phi_{0}\right),$

which is $o_{P_{W}^{*}}(n^{-1/2})$ by 5.6 and the above. By the derivations in
5.6, we also have

$\displaystyle\psi_{n}^{*}-\psi_{0}+P_{0}\phi_{n}^{*}$
$\displaystyle=P_{0}\left\\{\frac{\left(g_{0}-g_{n}^{*}\right)\left(\mu_{n}^{*}-\mu_{0}\right)}{\pi_{n}^{*}\left(1-g_{n}^{*}\right)}\right\\}+\frac{\pi_{n}^{*}-\pi_{0}}{\pi_{n}^{*}}\left(\psi_{n}^{*}-\psi_{0}\right).$

This is $o_{P_{W}^{*}}(n^{-1/2})$ by assumption.

For condition (B3) for the estimating equations-based estimator
$\psi_{n,2}^{*}$, we use Lemma 4.3. We note that the conditions of Lemma 4.2
hold by 5.6 and condition (B1) holds by the above. We have
$\psi_{n,2}^{*}=O_{P_{W}^{*}}(1)$ by the boundedness conditions and because
$E_{0}(Y^{2})<\infty$. As in the proof of 5.6, we have
$\Gamma_{0,\eta}(\psi)=0$ for all $\eta\in\mathscr{H}$ and
$\psi\in\mathbb{R}$, so that $\sup_{|\psi|\leq
M}\left|\Gamma_{0,\eta_{n}^{*}}(\psi)-\Gamma_{0,\eta_{0}}(\psi_{0})\right|=0$
for every $M>0$. Finally,

$\displaystyle P_{0}\phi_{\psi_{0},\eta_{n}^{*}}$
$\displaystyle=P_{0}\left\\{\frac{\left(g_{0}-g_{n}^{*}\right)\left(\mu_{n}-\mu_{0}\right)}{\pi_{n}^{*}\left(1-g_{n}^{*}\right)}\right\\}.$

which is $o_{P_{W}^{*}}(n^{-1/2})$ by assumption. $\hfill\blacksquare$

See 5.8

###### Proof of 5.8.

We use Lemma 3.7 to show condition (B1) holds. By the assumed classes of
$\mu_{n}^{*}$ and $g_{n}^{*}$, with conditional probability tending to 1,
$\phi_{n}^{*}$ falls in a class $\mathscr{F}$ with envelope $F(y,a,z):=c|Y|+d$
for some fixed $c,d\in(0,\infty)$. Thus, condition (i) of Lemma 3.7 holds by
the assumption that
$\lim_{M\to\infty}\sup_{P\in\mathscr{P}}E_{P}[Y^{2}I(Y^{2}>M)]=0$. Next, by
preservation of the finite uniform entropy integral condition under Lipschitz
transformations, (see, e.g., Lemma 9.17 of Kosorok,, 2008), $\mathscr{F}$
posesses finite uniform entropy integral. Hence, condition (ii) of Lemma 3.7
is satisfied by Theorem 2.8.3 of van der Vaart & Wellner, (1996). Next, as
noted following Lemma 3.7,
$\sup_{f,g\in\mathscr{F}}\left|(\hat{P}_{n}-P_{0})(fg)\right|=o_{P_{0}^{*}}(1)$
implies (2). Let $\mu,\bar{\mu}\in\mathscr{F}_{\mu}$ and
$g,\bar{g}\in\mathscr{F}_{g}$, and let $\pi,\bar{\pi}$ be implied by
$g,\bar{g}$ and $\psi,\bar{\psi}$ be implied by $\mu,g$ and
$\bar{\mu},\bar{g}$. Using the fact that $a(1-a)=0$ and the tower property, we
then have