distribution function $S:[0,\infty)\to[0,1]$ of $f$ is defined as

$\displaystyle S(y)=\int_{0}^{1}\mathbbm{1}\left(f(u)>y\right)du$

The bounded range of $f$ can be relaxed to $K=\infty$ as long as $f$ vanishes
at infinity, which means that $S(y)<\infty$ for every $y>0$ (and $f$ is
Lebesgue measurable), but we will not need that here. From Leoni [2017,
Proposition 4.1], the distribution function is decreasing and right
continuous, and clearly we have $0\leq S(y)\leq 1$ with $S(y)=0$ for all
$y\geq K$. The decreasing rearrangement of $f$, which we write as
$f_{\dagger}:[0,1]\to[0,K]$, is the left inverse of the distribution function,
that is

$\displaystyle f_{\dagger}(u):=\inf\\{y\in[0,K]:S(y)\leq u\\}.$

The existence of $f_{\dagger}$ follows as $S$ is decreasing and bounded from
below. From Leoni [2017, Proposition 4.3], we have that $f_{\dagger}$ is also
decreasing and right continuous. Another useful property is the equimeasurable
property, that is for all $y\geq 0$, we have

$\displaystyle\int_{0}^{1}\mathbbm{1}\left(f_{\dagger}(u)>y\right)\,du=\int_{0}^{1}\mathbbm{1}\left(f(u)>y\right)\,du.$

In fact, this equimeasurability holds more generally, which will be useful
later on.

######  Lemma A1 ([Leoni, 2017, Theorem 4.16]).

Let $f:[0,1]\to[0,K]$ and let $h:[0,\infty)\to[0,\infty)$ be a Borel
measurable function. We then have

$\displaystyle\int_{0}^{1}h(f(u))\,du=\int_{0}^{1}h(f_{\dagger}(u))\,du.$

###### Proof.

We have the result of Leoni [2017, Theorem 4.16] with equality as $(0,1)$ has
finite Lebesgue measure. ∎

Perhaps the most useful property of decreasing rearrangement for estimation is
the following inequality, of which there are many generalizations.

######  Proposition A3 ([Leoni, 2017, Theorem 4.19]).

Let $f,g:[0,1]\to[0,K]$ with respective decreasing rearrangements
$f^{\dagger},g^{\dagger}$. We then have

$\displaystyle d_{2}(f_{\dagger},g_{\dagger})\leq d_{2}(f,g)$

where $d_{2}(f,g)=\sqrt{\int_{0}^{1}(f(u)-g(u))^{2}\,du}$ is the $L^{2}$ norm.

The above will help us later when considering the convergence of rearranged
quantile estimates, and was used extensively by Chernozhukov et al. [2010].
Essentially, the above states that the decreasing rearrangement is continuous
from $L^{2}$ to itself. Actually, the above proposition can be weakened to the
case where $f,g\in L^{2}((0,1))$ [Kesavan, 2006, Theorem 1.2.3], but we will
not need that here.

As mentioned earlier, rearrangement works nicely with Sobolev spaces, as
rearrangement has a regularization effect on the function. The well-known
result below formalizes this.

######  Theorem A2 ([Leoni, 2017, Theorem 4.22]).

Let $f:[0,1]\to[0,K]$. If $f$ is absolutely continuous on $[0,1]$ with weak
derivative $f^{\prime}$, then $f_{\dagger}$ is also absolutely continuous on
$[0,1]$ with weak derivative $f^{\prime}_{\dagger}$. Furthermore, we have

$\displaystyle\|f_{\dagger}^{\prime}\|_{2}\leq\|f^{\prime}\|_{2}.$

We can apply Lemma A1 with $h(x)=x^{2}$, which gives
$\|f_{\dagger}\|_{2}^{2}=\|f\|_{2}^{2}$, and together with the above gives
$\|f_{\dagger}\|_{1,2}\leq\|f\|_{1,2}$. In other words, decreasing
rearrangement decreases the Sobolev norm, so it has a regularizing effect. In
the univariate case, Coron [1984] showed the stronger result that the
symmetric decreasing rearrangement is also continuous from
$W^{1,p}(\mathbb{R})$ to itself, which hints at an extension of Proposition A3
to the Sobolev norm (with the nonsymmetric decreasing rearrangement), but we
leave that for future work.

####  A.2.2 Increasing rearrangement

Our interest is actually on increasing rearrangement, and on functions with
both positive and negative support. Consider then a function $Q:[0,1]\to C$,
where $C\subset\mathbb{R}$ is a bounded subset of the real line. This is also
assumed in Chernozhukov et al. [2009, 2010]. In the main paper, we introduced
the increasing distribution function $P:C\to[0,1]$ as the familiar cumulative
distribution function,

$\displaystyle P(y)=\int_{0}^{1}\mathbbm{1}\left(Q(u)\leq y\right)\,du,$

with the increasing rearrangement $Q^{\dagger}:[0,1]\to C$ as

$\displaystyle Q^{\dagger}(u)=\inf\\{y\in C:P(y)\geq u\\}.$

Here, we have that $P$ is increasing and right continuous, as $P(y)=1-S(y)$
where $S(y)$ is decreasing and right continuous. This then suggests that
$Q^{\dagger}$ is increasing and left continuous, as expected.

Suppose $C$ is an interval, which is bounded so we can write $C=[-a,b]$ for
positive and finite constants $a,b$. Most results for rearrangement are stated
for non-negative $f$, so it is helpful to carry out a translation.

######  Lemma A2.

For $Q:[0,1]\to[-a,b]$, where $a,b$ are finite and positive constants, let
$Q_{+}:=Q+a$ be the translated non-negative function. We then have
$Q^{\dagger}(u)=Q_{+}^{\dagger}+a$, where $Q^{\dagger}$ and $Q_{+}^{\dagger}$
are the increasing rearrangements of $Q$ and $Q_{+}$ respectively.

###### Proof.

For $y\in[0,a+b]$, we have

$\displaystyle P_{+}(y)$
$\displaystyle=\int_{0}^{1}\mathbbm{1}\left(Q(u)+a\leq y\right)\,du=P(y-a)$

Similarly, for $y^{\prime}=y-a$, we have

$\displaystyle Q_{+}^{\dagger}(u)$
$\displaystyle=\inf\\{y\in[0,a+b]:P(y-a)\geq u\\}$
$\displaystyle=\inf\\{y^{\prime}\in C:P(y^{\prime})\geq
u\\}+a=Q^{\dagger}(u)+a.$

∎

As a result, we can just assume that $Q:[0,1]\to[0,K]$ without loss of
generality for the remainder of this section. To leverage the results on
decreasing rearrangement, we fortunately have a simple relationship between
the increasing and decreasing rearrangement.

######  Lemma A3.

Let $Q:[0,1]\to[0,K]$ for some finite and positive $K$, and let $Q_{\dagger}$
and $Q^{\dagger}$ denote its decreasing and increasing rearrangement
respectively. Then we have

$\displaystyle Q^{\dagger}(u)=Q_{\dagger}(1-u)$

for all $u\in[0,1]$.

###### Proof.

Again, we have $S(y)=1-P(y)$, which for each $u\in[0,1]$ gives

$\displaystyle Q^{\dagger}(u)$ $\displaystyle=\inf\\{y\in[0,K]:1-S(y)\geq
u\\}$ $\displaystyle=\inf\\{y\in[0,K]:S(y)\leq 1-u\\}$
$\displaystyle=Q_{\dagger}(1-u).$

∎

This connection is also commented on Korenovskii [2007, Section 1] and Kesavan
[2006, Exercise 1.4.1]. This allows us then to directly apply all the results
of the previous subsection, which we state formally for completion.

######  Corollary A1.

For any $f,g:[0,1]\to[0,K]$, Lemma A1, Proposition A3 and Theorem A2 all apply
if all instances of $f_{\dagger}$, $f^{\prime}_{\dagger}$ and $g_{\dagger}$
are replaced with $f^{\dagger}$, ${f^{\prime}}^{\dagger}$ and $g^{\dagger}$
respectively.

###### Proof.

For all appropriate integrals involving decreasing rearrangements, substitute
$f_{\dagger}(u)$ with $f^{\dagger}(1-u)$ (and likewise for
$g_{\dagger},f^{\prime}_{\dagger}$) and carry out a change of variables to
$u^{\prime}=1-u$, which has Jacobian determinant $1$ and integration limits
$u^{\prime}\in(0,1)$. ∎

It is perhaps not too surprising as the increasing rearrangement is also
equimeasurable, so it will very similar properties to the decreasing
rearrangement.

###  A.3 Empirical process theory

In this section, we show an auxiliary empirical process result that we require
for showing asymptotic tightness later. The weak $L^{2}$-pseudonorm of a
variable $X\sim\mathbb{P}$ is defined as

$\|X\|_{\mathbb{P},2,\infty}=\sup_{x>0}x\mathbb{P}(|X|>x)^{1/2}.$

Note that it is upper-bounded by the $L^{2}(\mathbb{P})$ norm:

$\|X\|_{\mathbb{P},2,\infty}\leq\|X\|_{\mathbb{P},2}=(\mathbb{E}_{\mathbb{P}}[X^{2}])^{1/2}.$

This is because for any value of $x>0$, we have

$x^{2}\mathbb{E}_{\mathbb{P}}[\mathbbm{1}(|X|>x)]\leq
x^{2}\mathbb{E}_{\mathbb{P}}\left[\frac{X^{2}}{x^{2}}\mathbbm{1}(|X|>x)\right]\leq\|X\|_{\mathbb{P},2}^{2}.$

######  Lemma A4.

For any positive r.v. $X\sim\mathbb{P}$, we have the inequality

$\sup_{x>0}x\mathbb{E}_{\mathbb{P}}[X\mathbbm{1}(X>x)]\leq
2\|X\|_{\mathbb{P},2,\infty}^{2}.$

This is the second inequality of Problem 2.5.5 in Van der Vaart and Wellner
[2023].

###### Proof.

For any value of $x$, the left-hand side of the inequality (without the
supremum) can be written as

$x\int_{t=0}^{\infty}\mathbb{P}(X\mathbbm{1}(X>x)>t)\,dt=x\int_{t=0}^{x}\mathbb{P}(X\mathbbm{1}(X>x)>t)\,dt+x\int_{t=x}^{\infty}\mathbb{P}(X\mathbbm{1}(X>x)>t)\,dt.$
(A1)

For the integrand in the first term on the right-hand side, we have

$\displaystyle\mathbb{P}(X\mathbbm{1}(X>x)>t)$
$\displaystyle=\mathbb{E}_{\mathbb{P}}[\mathbbm{1}(X\mathbbm{1}\\{X>x\\}>t)]$
$\displaystyle=\mathbb{E}_{\mathbb{P}}[\mathbbm{1}\\{X>x\\}]$
$\displaystyle=\mathbb{P}(X>x),$

where the second inequality follows from $t$ being less than or equal to $x$.
Thus, the first term on the right-hand side of (A1) is bounded above by
$\|X\|_{\mathbb{P},2,\infty}^{2}$.

The integrand in the second term can be written as

$\mathbb{P}(X\mathbbm{1}(X>x)>t)=\frac{1}{t^{2}}t^{2}\mathbb{P}(X\mathbbm{1}(X>x)>t)\leq\frac{1}{t^{2}}\|X\|_{\mathbb{P},2,\infty}^{2}.$

The integral of $1/t^{2}$ from $t=x$ to $\infty$ is $1/x$. Putting the two
terms together gives the result. ∎

For each $N\in\mathbb{N}$, let $\\{Z_{Ni}:\,i\geq N\\}$ be a sequence of
independent stochastic processes indexed by a common semimetric space
$(\mathcal{F},d)$. For every $N$, define the bracketing number
$\mathfrak{N}_{[]}(\varepsilon,\mathcal{F},L^{2,N})$ to be the minimal number
of sets $\mathfrak{N}_{\varepsilon}^{N}$ in a partition
$\mathcal{F}=\cup_{j=1}^{\mathfrak{N}_{\varepsilon}^{N}}\mathcal{F}_{\varepsilon
j}^{N}$ of the index set into sets $\mathcal{F}_{\varepsilon j}^{N}$ such
that, for every partitioning set $\mathcal{F}_{\varepsilon j}^{N}$, we have

$\sum_{i=N}^{\infty}\mathbb{E}^{*}\sup_{f,g\in\mathcal{F}_{\varepsilon
j}^{N}}|Z_{Ni}(f)-Z_{Ni}(g)|^{2}\leq\varepsilon^{2}.$

We will ultimately set $Z_{Ni}=\sqrt{N}\alpha_{i}H_{\rho_{i}}(u,V_{i})$. Then

$\sqrt{N}S_{N}=-\sum_{i=N}^{\infty}(Z_{Ni}-\mathbb{E}Z_{Ni}).$

The space $\ell^{\infty}(\mathcal{F})$ is the set of functions
$z:\mathcal{F}\rightarrow\mathbb{R}$ with
$\|z\|_{\mathcal{F}}=\sup_{t\in\mathcal{F}}|z(t)|<\infty$. This is a metric
space with respect to $d(z_{1},z_{2})=\|z_{1}-z_{2}\|_{\mathcal{F}}$. We wish
to show that $\sqrt{N}S_{N}$ is asymptotically tight, which means that for
every $\varepsilon>0$ there exists a compact set $K$ such that
$\liminf_{N\rightarrow\infty}\mathbb{P}(\sqrt{N}S_{N}\in K^{\delta})\geq
1-\varepsilon$, where
$K^{\delta}=\\{y\in\ell^{\infty}((0,1)):\,d(y,K)<\delta\\}$. This can be
achieved by verifying the conditions in the following general result.

######  Theorem A3 (Bracketing CLT with infinite sums).

Suppose that $(\mathcal{F},d)$ is totally bounded and each $Z_{Ni}$ has a
finite second moment. Suppose also that

$\displaystyle\sum_{i=N}^{\infty}\mathbb{E}^{*}\|Z_{Ni}\|_{\mathcal{F}}\mathbbm{1}\\{\|Z_{Ni}\|_{\mathcal{F}}>\eta\\}$
$\displaystyle\rightarrow 0\quad\text{for every $\eta>0$,}$
$\displaystyle\sup_{d(f,g)<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}[(Z_{Ni}(f)-Z_{Ni}(g)^{2}]$
$\displaystyle\rightarrow 0\quad\text{for every $\delta_{N}\downarrow 0$,}$
$\displaystyle\int_{0}^{\delta_{N}}\sqrt{\log\mathfrak{N}_{[]}(\varepsilon,\mathcal{F},L^{2,N}})\,d\varepsilon$
$\displaystyle\rightarrow 0\quad\text{for every $\delta_{N}\downarrow 0$.}$

Then the sequence $\sum_{i=N}^{\infty}(Z_{Ni}-\mathbb{E}Z_{Ni})$ is
asymptotically tight in $\ell^{\infty}(\mathcal{F})$ and converges in
distribution provided it converges marginally. If the partitions can be chosen
independent of $N$, then the middle of the displayed conditions is
unnecessary.

###### Proof.

The proof of this result mostly follows that of Theorem 2.11.9 in Van der
Vaart and Wellner [2023]. The crucial difference lies with the application of
Bernstein’s inequality, which is restricted to finite sums of variables.

Under the conditions of the theorem, there exists for every $N$ a sequence of
nested partitions
$\mathcal{F}=\cup_{j=1}^{\mathfrak{N}_{2^{-q}}^{N}}\mathcal{F}_{qj}^{N}$ such
that for every $j$ and $N$,

$\displaystyle\lim_{q_{0}\rightarrow\infty}\limsup_{N\rightarrow\infty}\sum_{q>q_{0}}2^{-q-1}\sqrt{\log\mathfrak{N}_{2^{-q}}^{N}}$
$\displaystyle=0,$ (A2)
$\displaystyle\sup_{f,g\in\mathcal{F}_{qj}^{N}}\sum_{i=N}^{\infty}\mathbb{E}\\{Z_{Ni}(f)-Z_{Ni}(g)\\}^{2}$
$\displaystyle\leq 2^{-2q},$ (A3)
$\displaystyle\sum_{i=N}^{\infty}\sup_{t_{i}}t_{i}^{2}\mathbb{P}^{*}\left(\sup_{f,g\in\mathcal{F}_{qj}^{N}}|Z_{Ni}(f)-Z_{Ni}(g)|>t_{i}\right)$
$\displaystyle\leq 2^{-2q}.$ (A4)

Equation (A2) above could be viewed as a lower-bound histogram approximation
to the entropy integral

$\int_{\varepsilon=0}^{2^{-q_{0}}}\sqrt{\log\mathfrak{N}_{\varepsilon}^{N}}\,d\varepsilon.$

Equations (A3) and (A4) follow from the same counting argument as the proof of
Theorem 2.5.8 in Van der Vaart and Wellner [2023].

Choose an element $f_{qj}$ from each partitioning set $\mathcal{F}^{N}_{qj}$
and define

$\displaystyle\pi_{q}f$ $\displaystyle=f_{qj},$
$\displaystyle(\Delta_{q}f)_{Ni}$
$\displaystyle=\sup_{g,h\in\mathcal{F}_{qj}^{N}}|Z_{Ni}(g)-Z_{Ni}(h)|,\quad\text{if
}f\in\mathcal{F}^{N}_{qj}$ $\displaystyle a_{q}$
$\displaystyle=2^{-q}\bigg{/}\sqrt{\log\mathfrak{N}_{2^{-(q+1)}}^{N}}.$

We interpret $f_{qj}$ as the “representative” of the partitioning set
$\mathcal{F}^{N}_{qj}$, and $\pi_{q}$ projects $f$ onto the representative
that shares its partitioning set. Also, $(\Delta_{q}f)_{Ni}$ is the maximum
distance between two points on $Z_{Ni}$ evaluated within the partitioning set
that contains $f$. For $q>q_{0}$, define indicator functions

$\displaystyle(A_{q-1}f)_{Ni}$
$\displaystyle=\mathbbm{1}\\{(\Delta_{q_{0}}f)_{Ni}\leq
a_{q_{0}},\ldots,(\Delta_{q-1}f)_{Ni}\leq a_{q-1}\\}$
$\displaystyle(B_{q-1}f)_{Ni}$
$\displaystyle=\mathbbm{1}\\{(\Delta_{q_{0}}f)_{Ni}\leq
a_{q_{0}},\ldots,(\Delta_{q-1}f)_{Ni}\leq a_{q-1},(\Delta_{q}f)_{Ni}>a_{q}\\}$
$\displaystyle(B_{q_{0}}f)_{Ni}$
$\displaystyle=\mathbbm{1}\\{(\Delta_{q_{0}}f)_{Ni}>a_{q_{0}}\\}.$

We wish to show that

$\lim_{q_{0}\rightarrow\infty}\limsup_{n\rightarrow\infty}\mathbb{E}^{*}\left\|\sum_{i=N}^{\infty}(Z_{Ni}^{\circ}(f)-Z_{Ni}^{\circ}(\pi_{q_{0}}f))\right\|_{\mathcal{F}}=0$
(A5)

for the centred processes $Z_{Ni}^{\circ}$, such that Theorem 1.5.6 of Van der
Vaart and Wellner [2023] implies asymptotic tightness in the case where the
partitions do not depend on $N$. To achieve this, we consider the
decomposition

$\displaystyle
Z_{Ni}(f)-Z_{Ni}(\pi_{q_{0}}f)=(Z_{Ni}(f)-Z_{Ni}(\pi_{q_{0}}f))(B_{q_{0}}f)_{Ni}$
$\displaystyle+\sum_{q>q_{0}}(Z_{Ni}(f)-Z_{Ni}(\pi_{q}f)(B_{q}f)_{Ni}$
$\displaystyle+\sum_{q>q_{0}}(Z_{Ni}(\pi_{q}f)-Z_{Ni}(\pi_{q-1}f)(A_{q-1}f)_{Ni}.$

For each of the three terms on the right-hand side separately, we centre at
zero expectation, sum from $i=N$ to $\infty$ and take the supremum over
$\mathcal{F}$. It is sufficient to then show that each of the resulting three
expressions converge to zero in mean as $n\rightarrow\infty$ followed by
$q_{0}\rightarrow\infty$.

As argued by Van der Vaart and Wellner [2023], the Lindeberg condition implies
that there is no loss in generality in assuming that
$\|Z_{Ni}\|_{\mathcal{F}}\leq\eta{N}$ for all $i\geq N$ for some sequence of
numbers $\eta_{N}\downarrow 0$. This implies that $(\Delta_{q}f)_{Ni}\leq
2\eta_{N}$ for all $i\geq N$, and the first expression is zero as soon as
$2\eta_{N}\leq a_{q_{0}}$. Condition (A2) implies that $a_{q_{0}}$ is bounded
away from 0 for any fixed $q_{0}$ as $N\rightarrow\infty$. If this were not
the case, then we must have

$\limsup_{N\rightarrow\infty}\log\mathfrak{N}_{2^{-(q_{0}+1)}}^{N}=\infty.$

And we have

$\log\mathfrak{N}_{2^{-q}}^{N}\geq\log\mathfrak{N}_{2^{-(q_{0}+1)}}^{N}$

for all $q\geq q_{0}+1$. Thus, this would imply that

$\limsup_{N\rightarrow\infty}\sum_{q>q_{0}}2^{-q-1}\sqrt{\log\mathfrak{N}_{2^{-q}}^{N}}=\infty,$

which contradicts the condition. We deduce that $2\eta_{N}\leq a_{q_{0}}$ for
all sufficiently large $n$.

For the second expression, we start by noting that
$(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}\leq(\Delta_{q-1}f)_{Ni}(B_{q}f)_{Ni}\leq
a_{q-1}$ by the nesting of the partitions and the definition of $B_{q}f$. It
follows that

$\displaystyle|Z_{Ni}(f)-Z_{Ni}(\pi_{q}f)|(B_{q}f)_{Ni}$
$\displaystyle\leq(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}\leq a_{q-1}$
$\displaystyle\text{Var}\left[\sum_{i=N}^{\infty}(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}\right]$
$\displaystyle\leq\sum_{i=N}^{\infty}\mathbb{E}\left[(\Delta_{q}f)^{2}_{Ni}(B_{q}f)^{2}_{Ni}\right]$
$\displaystyle\leq
a_{q-1}\sum_{i=N}^{\infty}\mathbb{E}\left[(\Delta_{q}f)_{Ni}\mathbbm{1}\\{(\Delta_{q}f)_{Ni}>a_{q}\\}\right]$
$\displaystyle\leq 2\frac{a_{q-1}}{a_{q}}2^{-2q},$

where the last inequality above follows from Lemma A4 and condition (A4).
Since each summand $(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}$ is independent, the
variance of the infinite sum above is greater than the variance of the partial
sum that replaces $\infty$ with any finite $m>N$. For such a fixed $m$, we can
apply Bernstein’s inequality (e.g. Lemma 2.2.10 of Van der Vaart and Wellner
[2023]) to deduce that for every $x>0$, we have

$\mathbb{P}\left(\left|\sum_{i=N}^{m}(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}-\mathbb{E}[(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}]\right|>x\right)\leq
2\exp\left(-\frac{\frac{1}{2}x^{2}}{2\frac{a_{q-1}}{a_{q}}2^{-2q}+\frac{2}{3}a_{q-1}x}\right).$

Note that the right-hand side does not depend on $m$. We also have that the
left-hand side converges to

$\mathbb{P}\left(\left|\sum_{i=N}^{\infty}(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}-\mathbb{E}[(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}]\right|>x\right)$
(A6)

as $m\rightarrow\infty$ (for fixed $N$ and $q$) at all continuity points $x$,
so the probability (A6) must therefore share the same exponential upper bound.
In fact, the bound also holds at any discontinuity point. To see this, note
that

$\mathbb{P}\left(\left|\sum_{i=N}^{\infty}(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}-\mathbb{E}[(\Delta_{q}f)_{Ni}(B_{q}f)_{Ni}]\right|>x\right)-2\exp\left(-\frac{\frac{1}{2}x^{2}}{2\frac{a_{q-1}}{a_{q}}2^{-2q}+\frac{2}{3}a_{q-1}x}\right)$

is right-continuous. If the above display is greater than 0 at some
discontinuity point $x^{*}$, then it must be greater than zero for all $x$ on
some interval $[x^{*},x^{*}+\delta]$ for $\delta>0$, which leads to a
contradiction because the number of discontinuity points is countable. Thus,
we can now apply Lemma 2.11.17 from Van der Vaart and Wellner [2023], and the
remaining steps for handling the second expression follow the proof of Theorem
2.11.9 from Van der Vaart and Wellner [2023].

The analysis of the third expression proceeds similarly. We have the following
bounds:

$\displaystyle|Z_{Ni}(\pi_{q}f)-Z_{Ni}(\pi_{q-1}f)|(A_{q-1}f)_{Ni}$
$\displaystyle\leq(\Delta_{q-1}f)_{Ni}(A_{q-1}f)_{Ni}$ $\displaystyle\leq
a_{q-1}$
$\displaystyle\text{Var}\left[\sum_{i=N}^{\infty}\\{Z_{Ni}(\pi_{q}f)-Z_{Ni}(\pi_{q-1}f)\\}(A_{q-1}f)_{Ni}\right]$
$\displaystyle\leq\sum_{i=N}^{\infty}\mathbb{E}[\\{Z_{Ni}(\pi_{q}f)-Z_{Ni}(\pi_{q-1}f)\\}^{2}(A_{q-1}f)_{Ni}]$
$\displaystyle\leq 2^{-2(q-1)},$

where the final inequality follows from the nesting of the partitions and
condition (A3). By applying a similar argument to before based on Bernstein’s
inequality, we derive the upper-bound

$\displaystyle\mathbb{P}\left(\left|\sum_{i=N}^{\infty}\\{Z_{Ni}(\pi_{q}f)-Z_{Ni}(\pi_{q-1}f)\\}(A_{q-1}f)_{Ni}-\mathbb{E}[\\{Z_{Ni}(\pi_{q}f)-Z_{Ni}(\pi_{q-1}f)\\}(A_{q-1}f)_{Ni}]\right|>x\right)$
$\displaystyle\leq
2\exp\left(-\frac{\frac{1}{2}x^{2}}{2^{-2(q-1)}+\frac{2}{3}a_{q-1}x}\right).$

Now we can again apply Lemma 2.11.17 to finish handling the third expression.

This concludes our proof of (A5). If the partitions depend on $N$, we require
an additional step. Let $\delta_{N}$ be a sequence tending to zero as
$N\rightarrow\infty$. First we have

$\displaystyle\mathbb{E}\sup_{d(f,g)<\delta_{N}}\left|\sum_{i=N}^{\infty}(Z_{Ni}^{\circ}(f)-Z_{Ni}^{\circ}(g))\right|$
$\displaystyle\leq
2\mathbb{E}^{*}\left\|\sum_{i=N}^{\infty}(Z_{Ni}^{\circ}(f)-Z_{Ni}^{\circ}(\pi_{q_{0}}f))\right\|_{\mathcal{F}}$
$\displaystyle+\mathbb{E}\left[\sup_{d(f,g)<\delta_{N}}\sum_{i=N}^{\infty}\left|Z_{Ni}^{\circ}(\pi_{q_{0}}f)-Z_{Ni}^{\circ}(\pi_{q_{0}}g)\right|\right].$

We have already dealt with the first term on the right-hand side. For the
second term, consider the set

$\mathcal{H}^{N}_{q_{0}}=\\{(\tilde{f},\tilde{g}):\,\text{there exists
}f,g\in\mathcal{F}\text{ with
}d(f,g)<\delta_{N},\pi_{q_{0}}f=\tilde{f},\pi_{q_{0}}g=\tilde{g}\\}.$

The size of $\mathcal{H}^{N}_{q_{0}}$ is at most
$(\mathfrak{N}_{q_{0}}^{N})^{2}$. Define $\zeta_{N}\geq 0$ as

$\zeta^{2}_{n}=\sup_{d(f,g)<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}[(Z_{Ni}^{\circ}(f)-Z_{Ni}^{\circ}(g))^{2}]\lesssim\sup_{d(f,g)<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}[(Z_{Ni}(f)-Z_{Ni}(g))^{2}],$

which tends to zero as $\delta_{N}\downarrow 0$ by assumption. For
$(\tilde{f},\tilde{g})\in\mathcal{H}_{q_{0}}^{N}$, we have the following
bounds:

$\displaystyle\left|Z_{Ni}^{\circ}(\tilde{f})-Z_{Ni}^{\circ}(\tilde{g})\right|$
$\displaystyle\leq 4\eta{N}$
$\displaystyle\text{Var}\left[\sum_{i=N}^{\infty}|Z_{Ni}^{\circ}(\tilde{f})-Z_{Ni}^{\circ}(\tilde{g})|\right]$
$\displaystyle\leq\sum_{i=N}^{\infty}\mathbb{E}[(Z_{Ni}^{\circ}(\tilde{f})-Z_{Ni}^{\circ}(\tilde{g}))^{2}]$
$\displaystyle\leq
3\sum_{i=1}^{\infty}\mathbb{E}[(Z_{Ni}^{\circ}(f)-Z_{Ni}^{\circ}(\tilde{f}))^{2}]+\mathbb{E}[(Z_{Ni}^{\circ}(g)-Z_{Ni}^{\circ}(\tilde{g}))^{2}]$
$\displaystyle+\mathbb{E}[(Z_{Ni}^{\circ}(f)-Z_{Ni}^{\circ}(g))^{2}]$
$\displaystyle\lesssim 2^{-2q_{0}}+\zeta_{N}^{2},$

where $(f,g)$ above are any elements of $\mathcal{F}$ satisfying
$d(f,g)<\delta_{N}$ with $\pi_{q_{0}}f=\tilde{f}$ and
$\pi_{q_{0}}g=\tilde{g}$. Thus, by using a similar Bernstein inequality
argument to before and applying Lemma 2.2.13 of Van der Vaart and Wellner
[2023], we yield

$\displaystyle\mathbb{E}\left[\sup_{d(f,g)<\delta_{N}}\sum_{i=N}^{\infty}\left|Z_{Ni}^{\circ}(\pi_{q_{0}}f)-Z_{Ni}^{\circ}(\pi_{q_{0}}g)\right|\right]$
$\displaystyle=\mathbb{E}\left[\max_{(\tilde{f},\tilde{g})\in\mathcal{H}_{q_{0}}^{N}}\sum_{i=N}^{\infty}\left|Z_{Ni}^{\circ}(\tilde{f})-Z_{Ni}^{\circ}(\tilde{g})\right|\right]$
$\displaystyle\lesssim\log N_{q_{0}}^{N}\eta{N}+\sqrt{\log
N_{q_{0}}^{N}}(2^{-q_{0}}+\zeta_{N}).$

We showed earlier that for fixed $q_{0}$, the limit superior of
$\log\mathfrak{N}_{q_{0}}^{N}$ as $N\rightarrow\infty$ is finite, so the
$\eta_{N}$ and $\zeta_{N}$ terms above go to zero as $N\rightarrow 0$. This
leaves the $\sqrt{\log\mathfrak{N}_{q_{0}}^{N}}2^{-q_{0}}$ term, which goes to
zero as $N\rightarrow\infty$ and then $q_{0}\rightarrow\infty$ by condition
(A2). Finally, we can apply Theorem 1.5.7 of Van der Vaart and Wellner [2023]
to obtain asymptotic tightness. ∎

###  A.4 Almost supermartingales

As our recursive estimates are closely related to stochastic approximation, it
is not surprising that we will borrow some tools from that literature to study
the frequentist asymptotic properties of the QMP. In particular, a very useful
theorem is given by Robbins and Siegmund [1971], which has been used for
proving consistency for other closely connected recursive Bayesian methods
like in Martin and Tokdar [2009], Hahn et al. [2018] and Fong et al. [2023].
We restate the almost supermartingale convergence theorem below.

######  Theorem A4 (Robbins and Siegmund [1971]).

Consider a probability space $(\Omega,\mathcal{F},\mathbb{P})$, and let
$\\{\mathcal{F}_{i}\\}_{i\geq 0}$ denote a filtration. Let $\\{L_{i}\\}_{i\geq
0}$ be a sequence of non-negative r.v.s $L_{i}:\Omega\to\mathbb{R}$ adapted to
the filtration (i.e. $L_{i}$ is $\mathcal{F}_{i}$-measurable for $i\geq 0$).
Suppose that $\\{L_{i}\\}_{i\geq 0}$ is an almost supermartingale, that is it
satisfies for $i\geq 0$

$\displaystyle E[L_{i+1}\mid\mathcal{F}_{i}]\leq(1+B_{i})\,L_{i}+C_{i}-D_{i}$

where $(B_{i},C_{i},D_{i})$ are non-negative adapted r.v.s. If
$\\{\sum_{i=1}^{\infty}B_{i}<\infty,\quad\sum_{i=1}^{\infty}C_{i}<\infty\\}$
hold a.s., then the limit $L_{\infty}:=\lim_{i\to\infty}L_{i}$ exists and is
finite a.s. and $\sum_{n=1}^{\infty}D_{i}<\infty$ a.s.

###  A.5 Bivariate normal copula

Although most of the theory can be extended for general copulas, we specialize
most of the proofs for the case with the bivariate normal copula. As such, we
provide some useful properties here, and refer to Meyer [2013] for more
details.

The bivariate normal copula distribution is the bivariate cumulative
distribution function (CDF) $C_{\rho}(u,v)$ which takes the form

$\displaystyle C_{\rho}(u,v)=\Phi_{2}\left(z_{u},z_{v};\rho\right)$

where $u,v\in(0,1)$, $z_{u}=\Phi^{-1}(u)$ is the normal quantile function at
$u$, and similarly for $v$. The correlation parameter is $\rho\in[-1,1]$ in
general, but we will only consider $\rho\in[0,1]$ for our purposes. Here,
$\Phi_{2}(\mu_{1},\mu_{2};\rho)$ is the standard bivariate normal CDF
evaluated at $\mu_{1},\mu_{2}$ with correlation $\rho$.

The conditional distribution of the bivariate normal copula (conditional on
$v$) takes the form

$\displaystyle H_{\rho}(u,v)$ $\displaystyle=\frac{\partial}{\partial
v}C_{\rho}(u,v)=\Phi\left\\{\frac{\Phi^{-1}\left(u\right)-\rho\Phi^{-1}(v)}{\sqrt{1-\rho^{2}}}\right\\},$

and the density of the bivariate normal copula is

$\displaystyle c_{\rho}(u,v)$ $\displaystyle=\frac{\partial^{2}}{\partial
u\partial v}C_{\rho}(u,v)=\frac{1}{\sqrt{1-\rho^{2}}}\exp\left(\frac{2\rho
z_{u}z_{v}-\rho^{2}(z_{u}^{2}+z_{v}^{2})}{2(1-\rho^{2})}\right).$

Note that $C_{\rho}(u,v)$ and $c_{\rho}(u,v)$ are symmetric in its inputs, but
$H_{\rho}(u,v)$ is not.

As the marginal distribution of $C_{\rho}(u,v)$ is uniform, we have that

$\displaystyle\int_{0}^{1}c_{\rho}(u,v)\,du=\int_{0}^{1}c_{\rho}(u,v)\,dv=1.$

This in turn implies

$\displaystyle\int_{0}^{1}H_{\rho}(u,v)\,dv=\int_{0}^{1}\int_{0}^{u}c_{\rho}(u^{\prime},v)\,du^{\prime}\,dv=u,$

which is crucial for the martingale property.

From Meyer [2013], the bivariate normal copula $C_{\rho}(u,v)$ satisfies the
following ordering property. For any $\rho^{\prime}\leq\rho$, we have

$\displaystyle\max(u+v-1,0)\leq C_{\rho^{\prime}}(u,v)\leq
C_{\rho}(u,v)\leq\min(u,v),$

for all $u,v\in[0,1]$. Furthermore, the lower and upper bounds are attained
with $\rho\to 0$ and $\rho\to 1$ respectively, i.e. $\lim_{\rho\to
1}C_{\rho}(u,v)=\min(u,v)$ and $\lim_{\rho\to 0}C_{\rho}(u,v)=\max(u+v-1,0)$.

###  A.6 Useful identities

A useful integral we will need for the proofs is the following for the
bivariate copula density.

######  Lemma A5.

Let $c_{\rho}(u,v)=\frac{\partial}{\partial u}H_{\rho}(u,v)$ denote the
bivariate normal copula density. We have that

$\displaystyle\int_{0}^{1}\int_{0}^{1}c_{\rho}(u,v)^{2}\,du\,dv=\frac{1}{1-\rho^{2}}\cdot$

###### Proof.

A change of variables from $v\to z_{v}$ gives

$\displaystyle\int_{0}^{1}c_{\rho}(u,v)^{2}\,dv$
$\displaystyle=\frac{1}{{1-\rho^{2}}}\int_{-\infty}^{\infty}\exp\left(\frac{2\rho
z_{u}z_{v}-\rho^{2}(z_{u}^{2}+z_{v}^{2})}{1-\rho^{2}}\right)\,\phi(z_{v})\,dz_{v}$
$\displaystyle=\frac{1}{\sqrt{2\pi}{(1-\rho^{2})}}\int_{-\infty}^{\infty}\exp\left(\frac{4\rho
z_{u}z_{v}-(1+\rho^{2})z_{v}^{2}-2\rho^{2}z_{u}^{2}}{2(1-\rho^{2})}\right)\,dz_{v}$

where $\phi$ is the normal density. Completing the square then gives us

$\displaystyle\int_{0}^{1}c_{\rho}(u,v)^{2}\,dv$
$\displaystyle=\frac{1}{\sqrt{2\pi}(1-\rho^{2})}\exp\left(\frac{\rho^{2}}{1+\rho^{2}}z_{u}^{2}\right)\int_{-\infty}^{\infty}\exp\left(-\frac{(1+\rho^{2})}{2(1-\rho^{2})}\left[\left(z_{v}-\frac{2\rho
z_{u}}{1+\rho^{2}}\right)^{2}\right]\right)\,dz_{v}$
$\displaystyle=\frac{1}{\sqrt{1-\rho^{4}}}\exp\left(\frac{\rho^{2}}{1+\rho^{2}}z_{u}^{2}\right).$

Carrying out another change of variables from $u\to z_{u}$ then gives

$\displaystyle\int_{0}^{1}\int_{0}^{1}c_{\rho}(u,v)^{2}\,du\,dv$
$\displaystyle=\frac{1}{\sqrt{2\pi(1-\rho^{4})}}\int_{-\infty}^{-\infty}\exp\left(-\frac{(1-\rho^{2})}{2(1+\rho^{2})}z_{u}^{2}\right)\,dz_{u}$
$\displaystyle=\sqrt{\frac{1+\rho^{2}}{1-\rho^{2}}}\frac{1}{\sqrt{1-\rho^{4}}}=\frac{1}{1-\rho^{2}}\cdot$

∎

We also have the following useful upper bound on $c_{\rho}(u,v)$.

######  Lemma A6.

The bivariate copula density satisfies

$\displaystyle
c_{\rho}(u,v)\leq\frac{1}{\sqrt{1-\rho^{2}}}\exp\left(\frac{z_{v}^{2}}{2}\right)$

The inequality holds if we replace $z_{v}$ with $z_{u}$.

###### Proof.

For a given $v$, standard calculations give that $z_{u}^{*}=z_{v}/\rho$
maximizes $c_{\rho}(u,v)$, which returns the above expression. ∎

Another very useful lemma which we will use for the covariance function of the
QMP is the following.

######  Lemma A7.

For $u,u^{\prime}\in(0,1)$, the copula update function satisfies

$\displaystyle\int_{0}^{1}\,\left[u-H_{\rho}(u,v)\right]\left[u^{\prime}-H_{\rho}(u^{\prime},v)\right]\,dv=C_{\rho^{2}}(u,u^{\prime})-uu^{\prime}.$

###### Proof.

First, we can easily see that

$\displaystyle\int_{0}^{1}\,\left[u-H_{\rho}(u,v)\right]\left[u^{\prime}-H_{\rho}(u^{\prime},v)\right]\,dv=\int_{0}^{1}\,H_{\rho}(u,v)\,H_{\rho}(u^{\prime},v)\,dv-
uu^{\prime}.$

To compute the integral, we write $H_{\rho}$ in terms of $c_{\rho}$:

$\displaystyle\int_{0}^{1}\,H_{\rho}(u,v)\,H_{\rho}(u^{\prime},v)\,dv=\int_{-\infty}^{u}\int_{-\infty}^{u^{\prime}}\int_{0}^{1}c_{\rho}(w,v)\,c_{\rho}(w^{\prime},v)\,dv\,dw\,dw^{\prime}$

The inner integral can be computed as

$\displaystyle\int_{0}^{1}c_{\rho}(w,v)\,c_{\rho}(w^{\prime},v)\,dv$
$\displaystyle=\frac{1}{1-\rho^{2}}\int_{-\infty}^{\infty}\exp\left(\frac{-\rho^{2}(z_{w}^{2}+z_{w^{\prime}}^{2})+2\rho(z_{w}+z_{w^{\prime}})\,z_{v}-2\rho^{2}z_{v}^{2}}{2(1-\rho^{2})}\right)\,\phi(z_{v})\,dz_{v}$
$\displaystyle=\frac{1}{\sqrt{2\pi}(1-\rho^{2})}\exp\left(\frac{-\rho^{2}(z_{w}^{2}+z_{w^{\prime}}^{2})}{2(1-\rho^{2})}\right)\int_{-\infty}^{\infty}\exp\left(\frac{2\rho(z_{w}+z_{w^{\prime}})\,z_{v}-(1+\rho^{2})z_{v}^{2}}{2(1-\rho^{2})}\right)\,dz_{v}$

where $\phi$ is the standard normal density function. Completing the square
gives

$\displaystyle\int_{-\infty}^{\infty}\exp\left(\frac{2\rho(z_{w}+z_{w^{\prime}})\,z_{v}-(1+\rho^{2})z_{v}^{2}}{2(1-\rho^{2})}\right)\,dz_{v}=\sqrt{2\pi}\sqrt{\frac{1-\rho^{2}}{1+\rho^{2}}}\exp\left(\frac{\rho^{2}\left(z_{w}+z_{w^{\prime}}\right)^{2}}{2(1+\rho^{2})(1-\rho^{2})}\right).$

Combining the above gives

$\displaystyle\int_{0}^{1}c_{\rho}(w,v)\,c_{\rho}(w^{\prime},v)\,dv$
$\displaystyle=\frac{1}{\sqrt{1-\rho^{4}}}\exp\left(\frac{-\rho^{4}\left(z_{w}^{2}+z_{w^{\prime}}^{2}\right)+2\rho^{2}z_{w}z_{w^{\prime}}}{2(1-\rho^{4})}\right)=c_{\rho^{2}}(w,w^{\prime}).$

Finally, this gives

$\displaystyle\int_{0}^{1}\,H_{\rho}(u,v)\,H_{\rho}(u^{\prime},v)\,dv=C_{\rho^{2}}(u,u^{\prime}).$

∎

##  Appendix B Proofs of main results

We now include full proofs of the main results from the paper, leveraging the
prerequisite results.

###  B.1 Proposition 1

For predictive asymptotics, we will treat the first $n$ data points $Y_{1:n}$
as fixed. Note that we start indexing at $N=n$ is so that the extension to the
case where $Y_{1:n}$ is i.i.d. from $P^{*}$ is straightforward. We will apply
Theorem A1 for the space $L^{2}((0,1))$ under Assumptions 1 and 3. First, we
verify that $\\{Q_{N}\\}_{N\geq n}$ is a Banach space valued martingale under
Algorithm 4.

We begin with the simplified case where $n=0$. Let
$(\Omega,\mathcal{F},\mathbb{P})$ denote the probability space and $L^{2}(B)$
the space of Bochner measurable functions with $\|f\|_{L^{2}(B)}<\infty$ as in
Section A.1, with $B=L^{2}((0,1))$. Let
$V_{i}\overset{\mathrm{iid}}{\sim}\mathcal{U}(0,1)$ for $i\geq 1$, i.e. each
$V_{i}:\Omega\to(0,1)$ is an independent uniform r.v. Define the filtration
$\\{\mathcal{F}\\}_{N\geq 1}$ where
$\mathcal{F}_{N}=\sigma(V_{1},\ldots,V_{N})$ and
$\mathcal{F}_{0}=\\{\emptyset,\Omega\\}$. For $N\geq 1$, define the mapping
$S_{N}:(0,1)^{N}\to B$ where

$\displaystyle
S_{N}(v_{1:N})(u)=\sum_{i=1}^{N}\alpha_{i}(u-H_{\rho_{i}}(u,v_{i}))$ (A7)

for each $u\in(0,1)$. For each $v_{1:N}\in(0,1)^{N}$, we clearly have
$S_{N}(v_{1:N})\in B$ as it is bounded by
$|S_{N}(v_{1:N})(u)|\leq\sum_{i=1}^{N}\alpha_{i}$. Here $\rho_{i}\in(0,1)$ and
$\alpha_{i}\in\mathbb{R}^{+}$ are arbitrary sequences where
$\alpha_{i}<\infty$. To begin, we require the following lemma, which is a
technical exercise, but we can fortunately repeat a similar argument for later
proofs. The key is that as we are working in a separable Hilbert space, we can
revert back to checking scalar conditions using inner products (which is
termed ‘scalarization’ for general Banach spaces in Pisier [2016]). This also
formally verifies our intuition that a pointwise martingale condition is
sufficient.

######  Lemma A8.

For each $N$, the random variable $S_{N}\circ V_{1:N}:\Omega\to B$ is Bochner
$\mathcal{F}_{N}$-measurable, lies in $L^{2}(B)$ and satisfies
$\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]=S_{N}(V_{1:N})$ a.s.

###### Proof.

First, we highlight that $B$ is a separable Hilbert space, so we can appeal to
Pettis’ measurability theorem (e.g. Hytönen et al. [2016, Theorem 1.1.6]) to
show Bochner measurability by verifying weak measurability. For each
continuous linear functional $T\in B^{*}$, let $h_{T}$ be the Riesz
representation of $T$. We thus just need to verify Borel measurability of the
scalar function $g_{T}:(0,1)^{N}\to\mathbb{R}$ where

$\displaystyle g(v_{1:N})$ $\displaystyle:=\langle
S_{N}(v_{1:N}),h_{T}\rangle_{L^{2}}=\int_{0}^{1}S_{N}(v_{1:N})(u)\,h_{T}(u)\,du.$

For each $u\in(0,1)$, the function $S_{N}(v_{1:N})(u)\,h_{T}(u)$ is continuous
in $v_{1:N}$, and $|S_{N}(v_{1:N})(u)\,h_{T}(u)|$ is bounded by
$\|h_{T}\|_{\infty}\sum_{i=1}^{N}\alpha_{i}$. Consider an arbitrary vector
sequence $v^{j}_{1:N}\to v^{*}_{1:N}$. Dominated convergence gives
$g(v^{j}_{1:N})\to g(v^{*}_{1:N})$, which implies $g(v_{1:N})$ is a continuous
function on $(0,1)^{N}$, and thus by composition $g\circ V_{1:N}$ is
$\mathcal{F}_{N}$-measurable. We thus have that $S_{N}\circ V_{1:N}$ is
Bochner $\mathcal{F}_{N}$-measurable. To show it lies in $L^{2}(B)$, we have
$\|S_{N}(v_{1:n})\|_{2}\leq\sum_{i=1}^{N}\alpha_{i}$ for all $v_{1:n}$, so
$\|S_{N}(V_{1:N})\|_{L^{2}(B)}\leq\sum_{i=1}^{N}\alpha_{i}$.

For the final part, we leverage the discussion in Section A.1. Consider again
a continuous linear functional $T\in B^{*}$. The conditional expectation
operator satisfies the following (e.g. Pisier [2016, Remark 1.11]):

$\displaystyle
T\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]=\mathbb{E}[TS_{N+1}(V_{1:N+1})\mid\mathcal{F}_{N}]\quad\text{a.s.}$

where we revert to the standard notation $E[\cdot\mid\mathcal{F}]$ when
working with scalar conditional expectations for clarity. Using the Riesz
representation again, we have

$\displaystyle T\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,S_{N+1}(V_{1:N+1})(u)\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,S_{N}(V_{1:N})(u)\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle+\alpha_{N+1}\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,(u-H_{\rho_{N+1}}(u,V_{N+1}))\,du\mid\mathcal{F}_{N}\right].$

We then have

$\displaystyle\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,S_{N}(V_{1:N})(u)\,du\mid\mathcal{F}_{N}\right]=\int_{0}^{1}\,h_{T}(u)\,S_{N}(V_{1:N})(u)\,du=TS_{N}(V_{1:N})\quad\text{a.s.}$

since $\int_{0}^{1}\,h_{T}(u)\,S_{N}(V_{1:N})(u)\,du$ is
$\mathcal{F}_{N}$-measurable. For the second term, we have

$\displaystyle\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,(u-H_{\rho_{N+1}}(u,V_{N+1}))\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,(u-H_{\rho_{N+1}}(u,V_{N+1}))\,du\right]\quad\text{a.s.}$
$\displaystyle=\int_{0}^{1}\int_{0}^{1}\,h_{T}(u)\,(u-H_{\rho_{N+1}}(u,v))\,du\,dv\quad\text{a.s.}$
$\displaystyle=0\quad\text{a.s.}$

where we have used the independence of $V_{N+1}$ from $\mathcal{F}_{N}$ in the
first line, and the last line follows from Fubini’s theorem and the pointwise
martingale property of the bivariate copula update (see Section A.5). As a
result, we have

$\displaystyle
T\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]=TS_{N}(V_{1:N})\quad\text{a.s.}$

for each $T\in B^{*}$. From the Bochner measurability of $S_{N}(V_{1:N})$, we
have from Hytönen et al. [2016, Corollary 1.1.25] that the above is sufficient
for $\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]=S_{N}(V_{1:N})$ a.s.
This follows as testing a.s. equality for all continuous linear functionals is
sufficient under strong measurability. ∎

Although the above is more of a technical exercise, it verifies our intuition
that having a martingale pointwise for a function is sufficient for it to be a
$B$-valued martingale. For the QMP, we can then construct the r.v.
$Q_{N}:\Omega\to B$, where

$\displaystyle
Q_{N}(u)=Q_{n}(u)+\sum_{i=n+1}^{N}\alpha_{i}(u-H_{\rho_{i}}(u,V_{i}))$ (A8)

for $u\in(0,1)$, and $\alpha_{i}$ is now specified as in Assumption 3. With a
relabelling of indices so $n>0$, the term on the right is equivalent to
$S_{N}(V_{1:N})(u)$ as defined above. Under Assumption 1, it is clear from
Lemma A8 that $Q_{N}$ is Bochner $\mathcal{F}_{N}$-measurable, $Q_{N}\in
L^{2}(B)$, and

$\displaystyle\mathbb{E}^{\mathcal{F}_{N}}[Q_{N+1}]=Q_{N}\quad\text{a.s.}$

$\\{Q_{N}\\}_{N\geq n}$ is thus a $B$-valued martingale.

We will now show that $\sup_{N\geq n}\|Q_{N}\|_{L^{2}(B)}<\infty$ in order to
apply Theorem A1. We begin with

$\displaystyle\|Q_{N}\|^{2}_{L^{2}(B)}$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}Q_{N}(u)^{2}\,du\right]=\int_{0}^{1}\mathbb{E}\left[Q_{N}(u)^{2}\right]\,du$

which follows from Tonelli’s theorem. As $\\{Q_{N}(u)\\}_{N\geq n}$ is a
martingale for each $u\in(0,1)$, we have

$\displaystyle\mathbb{E}[Q_{N}(u)^{2}\mid\mathcal{F}_{N-1}]$
$\displaystyle=Q_{N-1}^{2}(u)+\alpha_{N}^{2}\mathbb{E}\left[\left(u-H_{\rho_{N}}\left(u,V_{N}\right)\right)^{2}\right]\leq
Q_{N-1}^{2}(u)+\alpha_{N}^{2}$

which follows as $H_{\rho}(u,v)\leq 1$. Iterated expectation gives us

$\displaystyle\mathbb{E}[Q_{N}(u)^{2}]\leq
Q_{n}^{2}(u)+\sum_{i=n+1}^{N}\alpha_{i}^{2}.$

By Assumption 3, we have that $\sup_{N\geq
n}\sum_{i=n+1}^{N}\alpha_{i}^{2}\leq C<\infty$. As a result, we have

$\displaystyle\sup_{N\geq
n}\|Q_{N}\|^{2}_{L^{2}(B)}\leq\int_{0}^{1}\sup_{N\geq
n}\mathbb{E}\left[Q_{N}(u)^{2}\right]\,du\leq\int_{0}^{1}Q_{n}^{2}(u)\,du+C.$

By Assumption 1, we then have $\sup_{N\geq n}\|Q_{N}\|_{L^{2}(B)}<\infty$. We
can thus apply Theorem A1.

###  B.2 Theorem 1 and Corollary 1

The proof of Theorem 1 is similar to the proof of Proposition 1, but we will
be working in the Sobolev space $B^{\prime}=H^{1}((0,1))$. Following the
discussion in Section A.1, the martingale condition can be checked pointwise
again, as long as $Q_{N}$ takes values in $H^{1}((0,1))$. We extend Lemma A8
below. Consider the same setup as in the proof of Proposition 1, but replace
all mentions of $B$ with $B^{\prime}$. We first verify that $S_{N}(v_{1:N})$
as defined in (A7) is in $H^{1}((0,1))$ for all $v_{1:N}\in(0,1)^{N}$. First,
we note that the update function has partial derivative

$\displaystyle\frac{\partial}{\partial u}(u-H_{\rho}(u,v))=1-c_{\rho}(u,v).$
(A9)

We thus have the following for each $v_{1:N}\in(0,1)^{N}$:

$\displaystyle s_{N}(v_{1:N})(u):=\frac{\partial}{\partial
u}S_{N}(v_{1:N})(u)=\sum_{i=1}^{n}\alpha_{i}(1-c_{\rho_{i}}(u,v_{i})).$

Lemma A6 then gives that $s_{N}(v_{1:n})$ is bounded for each
$v_{1:N}\in(0,1)^{N}$ and each $N$, so $\|s_{N}(v_{1:N})\|_{2}<\infty$ and
$S_{N}:(0,1)^{N}\to H^{1}((0,1))$ for each $N$. We then have the following
lemma.

######  Lemma A9.

For each $N$, the random variable $S_{N}\circ V_{1:N}:\Omega\to B^{\prime}$ is
Bochner $\mathcal{F}_{N}$-measurable, lies in $L^{2}(B^{\prime})$ and
satisfies $\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]=S_{N}(V_{1:N})$
a.s.

###### Proof.

As $H^{1}$ is also a separable Hilbert space, the proof follows very much in
the same way as Lemma A8 with $B$ replaced with $B^{\prime}$, with the only
difference arising from the Sobolev inner product. To check Bochner
measurability, we show Borel measurability of

$\displaystyle g^{\prime}(v_{1:N}):=\langle
S_{N}(v_{1:N}),h_{T}\rangle_{H^{1}}=g(v_{1:N})+\int_{0}^{1}s_{N}(v_{1:N})(u)\,h^{\prime}_{T}(u)\,du$

where $h^{\prime}_{T}\in B$ is the weak derivative of $h_{T}$. The first term
is already shown to be continuous. For the second term, define
$t(v)=\int_{0}^{1}c_{\rho}(u,v)\,h_{T}^{\prime}(u)\,du$, and consider a
sequence $v_{i}\to v^{*}$ for $v^{*}\in(0,1)$. We then have

$\displaystyle|t(v)-t(v^{*})|$
$\displaystyle\leq\int_{0}^{1}|c_{\rho}(u,v_{i})-c_{\rho}(u,v^{*})|\,|h_{T}^{\prime}(u)|\,du$
$\displaystyle\leq\|c_{\rho}(\cdot,v_{i})-c_{\rho}(\cdot,v^{*})\|_{2}\,\|h_{T}^{\prime}\|_{2}$
$\displaystyle\leq K\|c_{\rho}(\cdot,v_{i})-c_{\rho}(\cdot,v^{*})\|_{2}$

As $|v_{i}-v^{*}|\leq\varepsilon$ for sufficiently large $i$, $v_{i}$ is
eventually bounded away from 0 and 1, so $c_{\rho}(u,v_{i})$ is eventually
bounded uniformly over $u$ by Lemma A6. Dominated convergence implies $t(v)$
is continuous on $(0,1)$, so $g^{\prime}(v_{1:N})$ is continuous on
$(0,1)^{N}$ and thus $g^{\prime}\circ V_{1:N}$ is
$\mathcal{F}_{N}$-measurable.

Showing that $S_{N}(V_{1:N})\in L^{2}(B^{\prime})$ requires some more work. As
we already showed that $S_{N}(V_{1:N})$ is in $L^{2}(B)$, we just need to
verify that $\mathbb{E}\left[\|s_{N}(V_{1:N})\|^{2}_{2}\right]<\infty$. The
linearity of expectation gives

$\displaystyle\mathbb{E}\left[\|s_{N}(V_{1:N})\|^{2}_{2}\right]=\int_{0}^{1}\sum_{i=1}^{N}\sum_{j=1}^{N}\alpha_{i}\alpha_{j}\mathbb{E}\left[(1-c_{\rho_{i}}(u,V_{i}))(1-c_{\rho_{j}}(u,V_{j}))\right]\,du.$

As $V_{i}$ is independent of $V_{j}$ for $i\neq j$, and
$\mathbb{E}[(1-c_{\rho}(u,V_{i}))]=0$ from Section A.5, the cross-terms
disappear and we have

$\displaystyle\mathbb{E}\left[\|s_{N}(V_{1:N})\|^{2}_{2}\right]$
$\displaystyle=\sum_{i=1}^{N}\alpha_{i}^{2}\int_{0}^{1}\mathbb{E}\left[(1-c_{\rho_{i}}(u,V_{i}))^{2}\right]\,du$
$\displaystyle=\sum_{i=1}^{N}\alpha_{i}^{2}\left[\int_{0}^{1}\int_{0}^{1}c_{\rho_{i}}(u,v)^{2}\,du\,dv-1\right]$
$\displaystyle=\sum_{i=1}^{N}\alpha_{i}^{2}\frac{\rho_{i}^{2}}{1-\rho_{i}^{2}}$

where the last line follows from Lemma A5. As $\rho_{i}\neq 1$, the above is
bounded for each $N$, so $S_{N}(V_{1:N})\in L^{2}(B^{\prime})$.

For the final part, we again just need to verify a.s. equality for continuous
linear functionals $T\in{B^{\prime}}^{*}$ with Riesz representation $h_{T}\in
B^{\prime}$. This time, we have

$\displaystyle T\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}\,h_{T}(u)\,S_{N+1}(V_{1:N+1})(u)\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle+\mathbb{E}\left[\int_{0}^{1}\,h^{\prime}_{T}(u)\,s_{N+1}(V_{1:N+1})(u)\,du\mid\mathcal{F}_{N}\right].$

From the proof of Lemma A8, we have that the first term is equal to $\langle
h_{T},S_{N}(V_{1:N})\rangle_{L^{2}}$ a.s. For the second term, we can carry
out a similar argument which gives

$\displaystyle\mathbb{E}\left[\int_{0}^{1}\,h^{\prime}_{T}(u)\,s_{N+1}(V_{1:N+1})(u)\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}\,h^{\prime}_{T}(u)\,s_{N}(V_{1:N})(u)\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle+\alpha_{N+1}\mathbb{E}\left[\int_{0}^{1}\,h^{\prime}_{T}(u)\,(1-c_{\rho_{N+1}}(u,V_{N+1}))\,du\mid\mathcal{F}_{N}\right]$

Again, we have the first term as equal to $\langle
h^{\prime}_{T}(u),\,s_{N}(V_{1:N})\rangle_{L^{2}}$ a.s. from
$\mathcal{F}_{N}$-measurability, and

$\displaystyle\mathbb{E}\left[\int_{0}^{1}\,h^{\prime}_{T}(u)\,(1-c_{\rho_{N+1}}(u,V_{N+1}))\,du\mid\mathcal{F}_{N}\right]$
$\displaystyle=\int_{0}^{1}\int_{0}^{1}\,h^{\prime}_{T}(u)\,(1-c_{\rho_{N+1}}(u,v))\,du\,dv\quad\text{a.s.}$
$\displaystyle=0\quad\text{a.s.}$

where we can apply Fubini’s theorem as Cauchy-Schwarz gives

$\displaystyle\int_{0}^{1}\int_{0}^{1}\,\left|h^{\prime}_{T}(u)\,(1-c_{\rho_{N+1}}(u,v))\right|\,du\,dv\leq\|h_{T}^{\prime}\|_{2}\,\sqrt{\int_{0}^{1}\int_{0}^{1}\left(1-c_{\rho_{N+1}}(u,v)\right)^{2}\,du\,dv}<\infty.$

As a result, we have

$\displaystyle T\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]$
$\displaystyle=\langle h_{T}(u),\,S_{N}(V_{1:N})\rangle_{L^{2}}+\langle
h^{\prime}_{T}(u),\,s_{N}(V_{1:N})\rangle_{L^{2}}\quad\text{a.s.}$
$\displaystyle=\langle
h_{T}(u),\,S_{N}(V_{1:N})\rangle_{H^{1}}\quad\text{a.s.}$
$\displaystyle=TS_{N}(V_{1:N})\quad\text{a.s.}$

for each $T\in{B^{\prime}}^{*}$. Again from Bochner measurability and Hytönen
et al. [2016, Corollary 1.1.25], we have
$\mathbb{E}^{\mathcal{F}_{N}}[S_{N+1}(V_{1:N+1})]=S_{N}(V_{1:N})$ a.s. ∎

Once again, pointwise martingales are sufficient. We then define
$Q_{N}:\Omega\to B^{\prime}$ again as (A8), with $\alpha_{i}$ and $\rho_{i}$
from Assumptions 3 and 4 respectively. As $Q_{n}\in B^{\prime}$ by Assumptions
1 and 2, $\\{Q_{N}\\}_{N\geq n+1}$ is a $B^{\prime}$-valued martingale by
Lemma A9.

We now verify that $\sup_{N\geq n}\|Q_{N}\|_{L^{2}(B^{\prime})}<\infty$, where
$Q_{N}$ is weakly differentiable a.s. with a.e. unique weak derivative

$\displaystyle
q_{N}(u)=q_{n}(u)+\sum_{i=n+1}^{N}\alpha_{i}\left(1-c_{\rho_{i}}(u,V_{i})\right)$

for $u\in(0,1)$. We begin with

$\displaystyle\|Q_{N}\|^{2}_{L^{2}(B^{\prime})}$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}Q_{N}(u)^{2}\,du+\int_{0}^{1}q_{N}(u)^{2}\,du\right]=\int_{0}^{1}\left(\mathbb{E}\left[Q_{N}(u)^{2}\right]+\mathbb{E}\left[q_{N}(u)^{2}\right]\right)\,du,$

where we have used Tonelli’s theorem in the second equality. We have already
bounded $\mathbb{E}[Q_{N}(u)^{2}]$ in the proof of Theorem 1, so we focus on
the second term. We first note that $\\{q_{N}(u)\\}_{N\geq n}$ is a martingale
for each $u\in(0,1)$, as $\int_{0}^{1}c_{\rho}(u,v)\,dv=1$. We thus have

$\displaystyle\mathbb{E}\left[q_{N}(u)^{2}\mid\mathcal{F}_{N-1}\right]$
$\displaystyle=q_{N-1}(u)^{2}+\alpha_{N}^{2}\mathbb{E}\left[(1-c_{\rho_{N}}(u,V_{N}))^{2}\mid\mathcal{F}_{N-1}\right]$
$\displaystyle=q_{N-1}(u)^{2}+\alpha_{N}^{2}\left(\mathbb{E}\left[c_{\rho_{N}}(u,V_{N})^{2}\mid\mathcal{F}_{N-1}\right]-1\right)$
$\displaystyle=q_{N-1}(u)^{2}+\alpha_{N}^{2}\frac{\rho_{N}^{2}}{1-\rho_{N}^{2}}$

where we have applied Lemma A5 in the last line. Iterated expectation again
gives us

$\displaystyle\mathbb{E}\left[q_{N}(u)^{2}\right]=q_{n}(u)^{2}+\sum_{i=n+1}^{N}\alpha_{i}^{2}\frac{\rho_{i}^{2}}{1-\rho_{i}^{2}}\cdot$

Combining with $\mathbb{E}[Q_{N}(u)^{2}]$ calculated earlier, this then gives
us

$\displaystyle\|Q_{N}\|^{2}_{L^{2}(B^{\prime})}$
$\displaystyle\leq\|Q_{n}\|^{2}_{1,2}+\sum_{i=n+1}^{N}\alpha_{i}^{2}\left(1+\frac{\rho_{i}^{2}}{1-\rho_{i}^{2}}\right)$
$\displaystyle=\|Q_{n}\|^{2}_{1,2}+\sum_{i=n+1}^{N}\frac{\alpha_{i}^{2}}{1-\rho_{i}^{2}}\cdot$

Now consider the form $\alpha_{N}=a(N+1)^{-1}$ and $\rho_{N}=\sqrt{1-cN^{-k}}$
from Assumptions 3 and 4, where $a,c,k$ are all positive constants (and
$c\in(0,1)$ so $\rho_{N}\in(0,1)$). The term to bound is

$\displaystyle\sup_{N\geq
n}\sum_{i=n+1}^{N}\frac{\alpha_{i}^{2}}{1-\rho_{i}^{2}}\leq
ac^{-1}\sum_{i=n+1}^{\infty}i^{-2+k}.$

The sum on the right is only bounded if $0\leq k<1$, as satisfied by
Assumption 4, and it is clear that $k$ controls the Sobolev norm and hence the
smoothness of the QMP sample paths. We can thus apply Theorem A1 with
$B^{\prime}=H^{1}((0,1))$. Corollary 1 follows directly from Proposition A1,
as $Q_{\infty}$ takes values in $H^{1}((0,1))$ a.s.

###  B.3 Proposition 3

To begin, we note that the $L^{2}$ norm is weaker than the Sobolev norm, that
is $d_{2}(f,g)\leq d_{1,2}(f,g)$. As a result, we have from Theorem 1 that
$d_{2}(Q_{N},Q_{\infty})\to 0$ a.s. As $Q_{N}$ and $Q_{\infty}$ take values in
$H^{1}((0,1))$, Proposition A2 implies that $Q_{N}$ and $Q_{\infty}$ are a.e.
equal to continuous bounded functions on $(0,1)$ a.s., which we can extend to
$[0,1]$. We can then apply Proposition 2, which gives
$d_{2}(Q_{N}^{\dagger},Q_{\infty}^{\dagger})\leq d_{2}(Q_{N},Q_{\infty})$, and
thus $d_{2}(Q_{N}^{\dagger},Q_{\infty}^{\dagger})\to 0$ a.s. Furthermore, the
increasing rearrangement variant of Theorem A2 implies that $Q_{N}^{\dagger}$
and $Q_{\infty}^{\dagger}$ are both in $L^{2}(H^{1}(0,1))$, and thus take
values in $H^{1}((0,1))$ a.s.

###  B.4 Corollary 2

As $P_{N}\to P_{\infty}$ weakly a.s., we can directly apply Aldous et al.
[1985, Lemma 8.2(b)] to show the asymptotic exchangeability of the sequence.
For the second part, we assume $n=0$ so the notation is simpler, but this is
just a matter of relabelling the indices. We begin by writing

$\displaystyle\mathbb{P}_{N}(y):=\frac{1}{N}\sum_{i=1}^{N}\mathbbm{1}\left(Y_{i}\leq
y\right)=\frac{1}{N}\sum_{i=1}^{N}\mathbbm{1}\left(Q_{i-1}(V_{i})\leq
y\right)$

where $V_{i}\overset{\mathrm{iid}}{\sim}\mathcal{U}(0,1)$. We then have that

$\displaystyle\mathbb{E}[\mathbb{P}_{N}(y)\mid\mathcal{F}_{N-1}]=\frac{N-1}{N}\mathbb{P}_{N-1}+\frac{1}{N}P_{N-1}(y)$

which follows as $\int_{0}^{1}\mathbbm{1}\left(Q_{N}(v)\leq y\right)\,dv$ is
precisely $P_{N}(y)$.

Following the proof of [Berti et al., 2004, Theorem 2.2], for any continuous
and bounded $f\in C_{b}(\mathbb{R})$, we define $U_{N}=f(Y_{N+1})$. We then
define the martingale

$Z_{N}=\sum_{i=0}^{N-1}\frac{U_{i}-\mathbb{E}[U_{i}\mid\mathcal{F}_{i-1}]}{i+1},$

which is bounded in $L^{2}$, so $(Z_{N})_{N\geq 1}$ converges a.s. Kronecker’s
lemma then gives

$\displaystyle\frac{1}{N}\sum_{i=0}^{N-1}\left(U_{i}-\mathbb{E}[U_{i}\mid\mathcal{F}_{i-1}]\right)\to
0\quad\text{a.s.}$

As a result, we have

$\displaystyle\lim_{N\to\infty}\frac{1}{N}\sum_{i=0}^{N-1}U_{i}=\lim_{N\to\infty}\frac{1}{N}\sum_{i=0}^{N-1}\mathbb{E}[U_{i}\mid\mathcal{F}_{i-1}]\quad\text{a.s.}$

Note that the left term is exactly $\mathbb{P}_{N}[f]$, where $P[f]=\int fdP$.
We also have

$\displaystyle\mathbb{E}[U_{N}\mid\mathcal{F}_{N-1}]=P_{N}[f]\to
P_{\infty}[f]\quad\text{a.s.}$

as $P_{N}$ converges weakly to $P_{\infty}$ a.s., so $\mathbb{P}_{N}[f]\to
P_{\infty}[f]$ a.s. through a Césaro means argument. The above holds a.s. for
the function $f=e^{ity}$ for each $t\in\mathbb{R}$, so Berti et al. [2006,
Theorem 2.6] gives us $\mathbb{P}_{N}\to P_{\infty}$ weakly a.s.

###  B.5 Theorem 2

The proof of this theorem consists of two parts. First we verify the
conditions of Theorem A3 to obtain asymptotic tightness of $\sqrt{N}S_{N}$.
Then it suffices to check marginal convergence on $\mathcal{F}$ using the
Lindeberg-Feller CLT; that is, we show that there is weak convergence to the
requisite multivariate normal distribution for any finite collection of
points.

######  Theorem A5.

The sequence of functions $\sqrt{N}S_{N}$ is asymptotically tight in
$\ell^{\infty}((0,1))$.

###### Proof.

We will verify the conditions of Theorem A3. Our semimetric space is
$(\mathcal{F}=(0,1),d)$, where $d(u_{1},u_{2})=|u_{1}-u_{2}|^{1/2}$. Clearly,
this semimetric space is totally bounded.

First define $Z_{Ni}(u)=\sqrt{N}\alpha_{i}H_{\rho_{i}}(u,V_{i})$ for all $N$
and $i\geq N$. A trivial envelope function for $Z_{Ni}$ is
$F_{Ni}=\sqrt{N}\alpha_{i}$. We need to verify the Lindeberg condition

$\sum_{i=N}^{\infty}F_{Ni}\mathbbm{1}\\{F_{Ni}>\eta\\}\rightarrow 0$

for every $\eta>0$. Since $F_{Ni}<c/\sqrt{N}$, we will have $F_{Ni}<\eta$ for
all sufficiently large $n$. Thus, the Lindeberg condition holds.

Next we need

$\sup_{d(u_{1},u_{2})<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}\left\\{Z_{Ni}(u_{1})-Z_{Ni}(u_{2})\right\\}^{2}\rightarrow
0$

for every $\delta_{N}\downarrow 0$. Let $u_{1}>u_{2}$ with
$d(u_{1},u_{2})<\delta_{N}$. Note that $Z_{Ni}(u)$ is non-decreasing with
$\lim_{u\downarrow 0}Z_{Ni}(u)=0$ and $\lim_{u\uparrow
1}Z_{Ni}(u)=\sqrt{N}\alpha_{i}$. So

$\displaystyle\mathbb{E}\left\\{Z_{Ni}(u_{1})-Z_{Ni}(u_{2})\right\\}^{2}$
$\displaystyle\leq\sqrt{N}\alpha_{i}\mathbb{E}\left\\{Z_{Ni}(u_{1})-Z_{Ni}(u_{2})\right\\}$
$\displaystyle\leq N\alpha_{i}^{2}(u_{1}-u_{2})$
$\displaystyle<N\alpha_{i}^{2}\delta_{N}^{2}.$

So

$\sup_{d(u_{1},u_{2})<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}\left\\{Z_{Ni}(u_{1})-Z_{Ni}(u_{2})\right\\}^{2}<N\delta_{N}^{2}\sum_{i=N}^{\infty}\alpha_{i}^{2}.$

Since $\limsup_{N\rightarrow\infty}N\sum_{i=N}^{\infty}\alpha_{i}^{2}\leq
c^{2}$, the right-hand side of the above display tends to zero for any
$\delta_{N}\downarrow 0$.

Finally, we need to verify the bracketing entropy integral condition. For all
sufficiently large $N$, we will have $N\sum_{i=N}^{\infty}\alpha_{i}^{2}\leq
2c^{2}$. Given $\varepsilon>0$, choose a partition
$0=u_{0}<u_{1}<\ldots<u_{M}=1$ such that
$u_{j}-u_{j-1}<\varepsilon^{2}/(2c^{2})$ for every $j$. The number of points
in the partition can be chosen to be smaller than a constant times
$1/\varepsilon^{2}$. Then

$\mathbb{E}\sup_{u_{j-1}\leq
s,t<u_{j}}|Z_{Ni}(s)-Z_{Ni}(t)|^{2}<\sqrt{N}\alpha_{i}\mathbb{E}[Z_{Ni}(u_{j})-Z_{Ni}(u_{j-1})]<\frac{N\alpha_{i}^{2}\varepsilon^{2}}{2c^{2}},$

where we have taken $Z_{Ni}(0)=0$ and $Z_{Ni}(1)=\sqrt{N}\alpha_{i}$ for
notational convenience. We deduce that for all sufficiently large $N$,

$\sum_{i=N}^{\infty}\mathbb{E}\sup_{u_{j-1}\leq
s,t<u_{j}}|Z_{Ni}(s)-Z_{Ni}(t)|^{2}<\varepsilon^{2}.$

In other words,
$\mathfrak{N}_{[]}(\varepsilon,\mathcal{F},L^{2,N})\lesssim{\varepsilon^{-2}}$,
which verifies the entropy condition. Thus, Theorem A3 implies that

$-\sum_{i=N}^{\infty}(Z_{Ni}(u)-\mathbb{E}Z_{Ni}(u))=\sqrt{N}\sum_{i=N}^{\infty}\alpha_{i}(u-H_{\rho_{i}}(u,V_{i}))$

is asymptotically tight in $\ell^{\infty}(\mathcal{F})$. ∎

######  Proposition A4.

Consider a vector
$\mathbf{S}_{N}(\mathbf{u})=[S_{N}(u_{1}),\ldots,S_{N}(u_{d})]^{T}$ where each
$u_{j}\in(0,1)$. We then have

$\displaystyle\sqrt{N}\mathbf{S}_{N}(\mathbf{u})\overset{d}{\to}\mathcal{N}(\mathbf{0},a^{2}\Sigma)$

where $\Sigma_{j,k}=\min\\{u_{j},u_{k}\\}-u_{j}u_{k}$.

###### Proof.

We will use the Lindeberg-Feller CLT [Van der Vaart, 2000, Proposition 2.27],
which we state below for convenience. For each $N$, let
$\\{Z_{N,N},\ldots,Z_{N,m_{N}}\\}$ be independent r.v.s with finite variances
that satisfy

$\displaystyle\sum_{i=N}^{m_{N}}\left(\mathbb{E}[Z_{Ni}^{2}]-\mathbb{E}[Z_{Ni}]^{2}\right)\to\sigma^{2}$

and the Lindeberg condition

$\displaystyle\sum_{i=N}^{m_{N}}\mathbb{E}[Z_{Ni}^{2}\mathbbm{1}\left(|Z_{Ni}|>\varepsilon\right)]\to
0$

for every $\varepsilon>0$. Then we have

$\displaystyle\sum_{i=N}^{m_{N}}(Z_{Ni}-\mathbb{E}[Z_{Ni}])\overset{d}{\to}\mathcal{N}(0,\sigma^{2}).$

We take $m_{N}$ to be any sequence such that $N/m_{N}=o(1)$. Consider an
arbitrary vector $\mathbf{t}\in\mathbb{R}^{d}$, and we study the convergence
of

$\displaystyle\sqrt{N}\,\mathbf{t}^{T}\mathbf{S}_{N}(\mathbf{u})=\sqrt{N}\sum_{j=1}^{d}\,t_{j}\,S_{N}(u_{j})=\sum_{i=N}^{\infty}\sqrt{N}\sum_{j=1}^{d}\,t_{j}\,\alpha_{i}(u_{j}-H_{\rho_{i}}(u_{j},V_{i})).$

For $i\geq N$, set

$Z_{Ni}=\sqrt{N}\sum_{j=1}^{d}\,t_{j}\,\alpha_{i}H_{\rho_{i}}(u_{j},V_{i}),$

which has expectation $\sqrt{N}\alpha_{i}\sum_{j=1}^{d}t_{j}u_{j}$. We have

$|Z_{Ni}|\leq
a\frac{\sqrt{N}}{i}\sum_{j=1}^{d}|t_{j}|\leq\frac{a}{\sqrt{N}}\sum_{j=1}^{d}|t_{j}|.$

So for every $\varepsilon>0$, $|Z_{Ni}|<\varepsilon$ eventually, which
establishes the Lindeberg condition. It remains to study the limiting
variance. Using the integral test, we obtain

$\displaystyle\mathbb{E}[Z_{Ni}^{2}]-\mathbb{E}[Z_{Ni}]^{2}$
$\displaystyle=a^{2}\frac{N}{i^{2}}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{\mathbb{E}\left[H_{\rho_{i}}(u_{j},V_{i})\,H_{\rho_{i}}(u_{k},V_{i})\right]-u_{j}u_{k}\\}$
$\displaystyle=a^{2}\frac{N}{i^{2}}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{C_{\rho_{i}^{2}}(u_{j},u_{k})-u_{j}u_{k}\\}.$

Taking $\rho^{2}\to 1$ gives
$C_{\rho^{2}}(u_{j},u_{k})\to\min\\{u_{j},u_{k}\\}$ as discussed in Section
A.5. Then

$\displaystyle\left|\sum_{i=N}^{m_{N}}\left(\mathbb{E}[Z^{2}_{Ni}]-\mathbb{E}[Z_{Ni}]^{2}\right)-a^{2}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{\min\\{u_{j},u_{k}\\}-u_{j}u_{k}\\}\right|$
$\displaystyle\leq\sum_{j=1}^{d}\sum_{k=1}^{d}\left|\sum_{i=N}^{m_{N}}\frac{a^{2}N}{i^{2}}\left[C_{\rho_{i}^{2}}(u_{j},u_{k})-\min\\{u_{j},u_{k}\\}\right]\right|+a^{2}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{\min\\{u_{j},u_{k}\\}-u_{j}u_{k}\\}\left|\sum_{i=N}^{m_{N}}\frac{N}{i^{2}}-1\right|$
$\displaystyle\leq\sum_{j=1}^{d}\sum_{k=1}^{d}\left|C_{\rho_{N}^{2}}(u_{j},u_{k})-\min\\{u_{j},u_{k}\\}\right|\left|\sum_{i=N}^{m_{N}}\frac{a^{2}N}{i^{2}}\right|+a^{2}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{\min\\{u_{j},u_{k}\\}-u_{j}u_{k}\\}\left|\sum_{i=N}^{\infty}\frac{N}{i^{2}}-1\right|$
$\displaystyle+a^{2}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{\min\\{u_{j},u_{k}\\}-u_{j}u_{k}\\}\left|\sum_{i=m_{N}+1}^{\infty}\frac{N}{i^{2}}\right|.$

The first inequality uses the triangle inequality. The first and second terms
in the last expression are $o(1)$. The third term is $O(N/m_{N})$, which is
$o(1)$ by construction. Thus, we have

$\sum_{i=N}^{m_{N}}\left(\mathbb{E}[Z^{2}_{Ni}]-\mathbb{E}[Z_{Ni}]^{2}\right)\to\sigma^{2}=a^{2}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{\min\\{u_{j},u_{k}\\}-u_{j}u_{k}\\}$

and we can apply the Lindeberg-Feller CLT to obtain

$\sum_{i=N}^{m_{N}}\sqrt{N}\sum_{j=1}^{d}\,t_{j}\,\alpha_{i}(u_{j}-H_{\rho_{i}}(u_{j},V_{i}))\overset{d}{\to}\mathcal{N}(0,\sigma^{2}).$

The final step is to show that the tail sums from $m_{N}+1$ onwards become
asymptotically negligible. We do this by checking that
$\sum_{i=m_{N}+1}^{\infty}\text{Var}[Z_{Ni}]\to 0$:

$\displaystyle\sum_{i=m_{N}+1}^{\infty}\text{Var}[Z_{Ni}]$
$\displaystyle=\sum_{i=m_{N}+1}^{\infty}a^{2}\frac{N}{i^{2}}\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\\{C_{\rho_{i}^{2}}(u_{j},u_{k})-u_{j}u_{k}\\}$
$\displaystyle\leq\sum_{i=m_{N}+1}^{\infty}a^{2}\frac{N}{i^{2}}\left|\sum_{j=1}^{d}\sum_{k=1}^{d}t_{j}t_{k}\right|$
$\displaystyle\to 0.$

Now the Cramér-Wold device completes the proof. ∎

###  B.6 Theorem 3

To begin, we assume that $Y_{1:n}\overset{\mathrm{iid}}{\sim}P^{*}$ which has
quantile function $Q^{*}$ satisfying Assumption 5. Let
$\\{\mathcal{F}_{i}\\}_{i\geq 1}$ denote the filtration where
$\mathcal{F}_{i}:=\sigma\left(Y_{1},\ldots,Y_{i}\right)$. For now, we do not
need to consider $Y_{n+1:\infty}$ arising from predictive resampling. To
start, we construct an almost supermartingale as in Theorem A4 arising from
Algorithm 3:

$\displaystyle\left(Q^{*}(u)-Q_{n}(u)\right)^{2}$
$\displaystyle=(Q^{*}(u)-Q^{\dagger}_{n-1}(u))^{2}+2\alpha_{n}(Q^{*}(u)-Q^{\dagger}_{n-1}(u))\left(H_{\rho_{n}}\left(u,V_{n}\right)-u\right)$
$\displaystyle+\alpha_{n}^{2}\left(H_{\rho_{n}}\left(u,V_{n})\right)-u\right)^{2}.$

where for shorthand we write $V_{n}=P_{n-1}(Y_{n})$. We note that
$|H_{\rho}(u,v)-u|\leq 1$. Integrating with respect to $u$, we have

$\displaystyle d_{2}^{2}\left(Q^{*},Q_{n}\right)$ $\displaystyle\leq
d_{2}^{2}(Q^{\dagger}_{n-1},Q^{*})+2\alpha_{n}\int\,(Q^{*}(u)-Q^{\dagger}_{n-1}(u))\left(H_{\rho_{n}}\left(u,V_{n}\right)-u\right)du+\alpha_{n}^{2}.$

Now comes the key step due to the rearrangement. By Assumption 5, both $Q^{*}$
and $Q_{0}$ are Lipschitz continuous and thus bounded on $[0,1]$. Since
$H_{\rho}(u,v)-u$ is bounded on $u\in[0,1]$, $Q_{n}^{\dagger}$ is bounded on
$[0,1]$. We can thus apply Proposition A3, which gives

$d_{2}^{2}(Q_{n}^{\dagger},Q^{*})\leq d_{2}^{2}(Q_{n},Q^{*}),$

so we have

$\displaystyle d_{2}^{2}\left(Q^{\dagger}_{n},Q^{*}\right)$ $\displaystyle\leq
d_{2}^{2}(Q^{\dagger}_{n-1},Q^{*})+2\alpha_{n}\int\,(Q^{*}(u)-Q^{\dagger}_{n-1}(u))\left(H_{\rho_{n}}\left(u,V_{n}\right)-u\right)du+\alpha_{n}^{2}.$

Taking the conditional expectation gives us

$\displaystyle\mathbb{E}\left[L_{n}\mid\mathcal{F}_{n-1}\right]\leq
L_{n-1}+2\alpha_{n}\int\,(Q^{*}(u)-Q^{\dagger}_{n-1}(u))\left(K_{n}\left(u\right)-u\right)du+\alpha_{n}^{2},$

where we write $L_{n}=d_{2}^{2}(Q^{\dagger}_{n},Q^{*})$ and

$K_{n}(u)=\int H_{\rho_{n}}(u,P_{n-1}(y))\,p^{*}(y)\,dy.$

We now subtract and add terms to get

$\displaystyle\mathbb{E}\left[L_{n}\mid\mathcal{F}_{n-1}\right]\leq
L_{n-1}-2\alpha_{n}\int\,(Q^{*}(u)-Q^{\dagger}_{n-1}(u))(u-P^{*}(Q^{\dagger}_{n-1}(u)))\,du+\alpha_{n}^{2}+2\alpha_{n}\zeta_{n}$

where

$\displaystyle\zeta_{n}=\int(Q^{*}(u)-Q^{\dagger}_{n-1}(u))(K_{n}\left(u\right)-P^{*}(Q^{\dagger}_{n-1}(u)))\,du.$
(A10)

It is not too hard to see that

$\displaystyle
T(Q_{n-1}^{\dagger}):=\int\,(Q^{*}(u)-Q^{\dagger}_{n-1}(u))(u-P^{*}(Q^{\dagger}_{n-1}(u)))du\geq
0,$

where the positivity can be seen by applying $Q^{*}$, which is monotonically
increasing, to each term in $u-P^{*}(Q^{\dagger}_{n-1}(u))$, giving

$\displaystyle u-P^{*}(Q^{\dagger}_{n-1}(u))\geq 0\implies
Q^{*}(u)-Q^{\dagger}_{n-1}(u)\geq 0.$

To get a handle on $\zeta_{n}$, Cauchy-Schwartz gives us

$|\zeta_{n}|\leq\sqrt{L_{n-1}}\sqrt{\int(K_{n}\left(u\right)-P^{*}(Q^{\dagger}_{n-1}(u)))^{2}\,du}$

and applying $\sqrt{x}\leq x+1$ gives

$|\zeta_{n}|\leq(L_{n-1}+1)\sqrt{\kappa_{n}}$

where we write

$\displaystyle\kappa_{n}={\int(K_{n}\left(u\right)-P^{*}(Q^{\dagger}_{n-1}(u)))^{2}\,du}$
(A11)

This gives the original inequality as

$\displaystyle\mathbb{E}[L_{n}\mid\mathcal{F}_{n-1}]\leq\left(1+2\alpha_{n}\sqrt{\kappa_{n}}\right)L_{n}-2\alpha_{n}T(Q_{n-1}^{\dagger})+\alpha_{n}^{2}+2\alpha_{n}\sqrt{\kappa_{n}}.$

We now seek to apply Theorem A4 with $B_{i}=2\alpha_{i}\sqrt{\kappa_{i}}$,
$C_{i}=\alpha_{i}^{2}+2\alpha_{i}\sqrt{\kappa_{i}}$ and
$D_{i}=2\alpha_{i}T(Q_{i-1}^{\dagger})$, where all terms are positive. A
sufficient condition for $\sum_{i=1}^{\infty}B_{i}<\infty$ and
$\sum_{i=1}^{\infty}C_{i}<\infty$ a.s. is if

$\sum_{i=1}^{\infty}\alpha_{i}\sqrt{\kappa_{i}}<\infty\quad\text{a.s. }$

which we now prove.

####  B.6.1 Controlling $\kappa_{n}$

We begin by upper bounding $\kappa_{i}$, which intuitively measures how far
the copula kernel $H_{\rho}(u,v)$ is from the indicator function
$\mathbbm{1}\left(v\leq u\right)$ (averaged over $u,v$). The result is simple
to state, but the proof is surprisingly quite involved, and requires specific
properties of the bivariate Gaussian copula. The difficulty of the proof
arises from the copula’s dependence on $P_{n}$, as we require bounds
independent of $P_{n}$ and $P^{*}$. This highlights that while the bivariate
copula is very useful for attaining the martingale and coherence required for
Bayesian inference, its adaptivity makes it harder to study its properties for
estimation.

######  Lemma A10.

Let $\kappa_{n}$ be defined as in (A11), and suppose $\rho_{n}\to 1$. For
sufficiently large $n$, we have that $\kappa_{n}$ satisfies

$\displaystyle\sqrt{\kappa_{n}}\leq K(1-\rho_{n}^{2})^{1/4}$

for some positive finite constant $K$.

###### Proof.

To start, we can write

$\displaystyle P^{*}(Q_{n-1}^{\dagger}(u))$
$\displaystyle=\int\mathbbm{1}(y\leq
Q_{n-1}^{\dagger}(u))\,dP^{*}(y)=\int\mathbbm{1}(P_{n-1}(y)\leq
u)\,dP^{*}(y).$

This gives

$\displaystyle\kappa_{n}$
$\displaystyle=\int\left[\int(H_{\rho_{n}}(u,P_{n-1}(y))-\mathbbm{1}(P_{n-1}(y)\leq
u))\,dP^{*}(y)\right]^{2}\,du$
$\displaystyle\leq\int\int(H_{\rho_{n}}(u,P_{n-1}(y))-\mathbbm{1}(P_{n-1}(y)\leq
u))^{2}\,du\,dP^{*}(y).$

Let us write the inner integral as a function of a general $\rho,v\in(0,1)$:

$\displaystyle U(v;\rho):=\int\int(H_{\rho}(u,v)-\mathbbm{1}(v\leq
u))^{2}\,du.$

Fortunately, we can control $U(v;\rho)$ by taking $\rho\to 1$. To see this,
note that

$\displaystyle\int(H_{\rho}(u,v)-\mathbbm{1}(v\leq u))^{2}\,du=\int
H^{2}_{\rho}(u,v)\,du-2\int_{v}^{1}H_{\rho}(u,v)\,du+\int_{v}^{1}\,du.$ (A12)

A change of variables with $z_{u}=\Phi^{-1}(u)$ gives

$H_{\rho}(u,v)=\Phi(a+bz_{u})$

where

$\displaystyle a=-\frac{\rho}{\sqrt{1-\rho^{2}}}z_{v},\quad
b=\frac{1}{\sqrt{1-\rho^{2}}}$

where $z_{v}=\Phi^{-1}(v)$. We now require some integrals of Gaussian CDF and
density terms, which can be found in Owen [1980]. For the first term of (A12),
we have

$\displaystyle\int_{-\infty}^{\infty}\Phi(a+bz_{u})^{2}\phi(z_{u})\,dz_{u}$
$\displaystyle=\Phi_{2}\left(\frac{a}{\sqrt{1+b^{2}}},\frac{a}{\sqrt{1+b^{2}}};\rho=\frac{b^{2}}{1+b^{2}}\right)$

where $\Phi_{2}(\mu_{1},\mu_{2};\rho)$ is the standard bivariate normal CDF
evaluated at $\mu_{1},\mu_{2}$ with correlation $\rho$. This follows from
codes (20,010.4) and (3.5) from Owen [1980].

For the second term of (A12), we have

$\displaystyle\int_{z_{v}}^{\infty}\Phi(a+bz_{u})\,\phi(z_{u})\,dz_{u}$
$\displaystyle=\int_{-\infty}^{\infty}\Phi(a+bz_{u})\,\phi(z_{u})\,dz_{u}-\int_{-\infty}^{z_{v}}\Phi(a+bz_{u})\,\phi(z_{u})\,dz_{u}$
$\displaystyle=\Phi\left(\frac{a}{\sqrt{1+b^{2}}}\right)-\Phi_{2}\left(\frac{a}{\sqrt{1+b^{2}}},z_{v};\rho=\frac{-b}{\sqrt{1+b^{2}}}\right)$
$\displaystyle=\Phi_{2}\left(\frac{a}{\sqrt{1+b^{2}}},-z_{v};\rho=\frac{b}{\sqrt{1+b^{2}}}\right)$

where the second line comes from codes (10,010.1) and (10,010.1) from Owen
[1980], and the third line comes from the identity
$\Phi(a)-\Phi_{2}(a,b;\rho)=\Phi_{2}(a,-b;-\rho)$. The final term of (A12) is
simply $1-v$. Putting this together, we have that

$\displaystyle U(v;\rho)$
$\displaystyle\leq\Phi_{2}\left(\frac{a}{\sqrt{1+b^{2}}},\frac{a}{\sqrt{1+b^{2}}};\rho=\frac{b^{2}}{1+b^{2}}\right)-2\Phi_{2}\left(\frac{a}{\sqrt{1+b^{2}}},-z_{v};\rho=\frac{b}{\sqrt{1+b^{2}}}\right)+(1-v)$
$\displaystyle=\Phi_{2}\left(\frac{-\rho}{\sqrt{2-\rho^{2}}}z_{v},\frac{-\rho}{\sqrt{2-\rho^{2}}}z_{v};\rho=\frac{1}{2-\rho^{2}}\right)-2\Phi_{2}\left(\frac{-\rho}{\sqrt{2-\rho^{2}}}z_{v},-z_{v};\rho=\frac{1}{\sqrt{2-\rho^{2}}}\right)+(1-v)$

We will first write the above in terms of the tail probability of a bivariate
normal distribution, $L(h,k;\rho)=P(x>h,y>k;\rho)$, which is easier to bound
and satisfies

$\displaystyle L(h,k;\rho)$ $\displaystyle=\Phi_{2}(-h,-k;\rho)$

This gives us

$\displaystyle U(v;\rho)$ $\displaystyle\leq
L\left(\frac{\rho}{\sqrt{2-\rho^{2}}}z_{v},\frac{\rho}{\sqrt{2-\rho^{2}}}z_{v};\frac{1}{2-\rho^{2}}\right)-2L\left(\frac{\rho}{\sqrt{2-\rho^{2}}}z_{v},z_{v};\frac{1}{\sqrt{2-\rho^{2}}}\right)+(1-v)$
$\displaystyle=L(h,h;\bar{\rho}^{2})-2L(h,z_{v};\bar{\rho})+Q(z_{v})$

where we define $h=\frac{\rho}{\sqrt{2-\rho^{2}}}z_{v}$,
$\bar{\rho}=\frac{1}{\sqrt{2-\rho^{2}}}$, and $Q(z)=1-\Phi(z)$ is the Gaussian
tail probability. We first show that this upper bound is symmetric around
$z_{v}=0$. Using the identity

$L(h,k;\rho)=1-\Phi(h)-\Phi(k)+L(-h,-k;\rho)$

from the above, we can see that $U(v;\rho)=U(1-v;\rho)$ noting that
$z_{1-v}=-z_{v}$. As a result, we just need to bound $U(v;\rho)$ for
$z_{v}\geq 0$ (i.e. $v\geq 0.5$).

We note that $L(h,h;\bar{\rho}^{2})\leq L(h,h;\bar{\rho})$ as $L$ is
increasing with $\bar{\rho}$, and furthermore $L(h,z_{v};\bar{\rho})\geq
L(z_{v},z_{v};\bar{\rho})$ as $z_{v}\geq h$. This then gives us

$\displaystyle U(v;\rho)$ $\displaystyle\leq
L(h,h;\bar{\rho})-2L(z_{v},z_{v};\bar{\rho})+Q(z_{v}).$

One can show that

$\frac{\partial L(h,h;\bar{\rho})}{\partial h}=-2\Phi(-\theta h)\phi(h)$

where $\theta=\sqrt{\frac{1-\bar{\rho}}{1+\bar{\rho}}}$. A Taylor expansion
gives

$\displaystyle L(z_{v},z_{v};\bar{\rho})$
$\displaystyle=L(h,h;\bar{\rho})-2\Phi(-\theta\tilde{h})\,\phi(\tilde{h})\,(z_{v}-h)$
$\displaystyle\geq L(h,h;\bar{\rho})-2\Phi(-\theta{h})\,\phi({h})\,(z_{v}-h),$

where $h\leq\tilde{h}\leq z_{v}$ . The second line follows as the function
$\Phi(-\theta h)\,\phi(h)$ is monotonically decreasing with $h$ for $h\geq 0$.
This then implies

$\displaystyle U(v;\rho)\leq
2\Phi(-\theta{h})\,\phi({h})(z_{v}-h)+Q(z_{v})-L(z_{v},z_{v};\bar{\rho}).$

We can upper bound the first term as

$\displaystyle\phi(h)(z_{v}-h)$
$\displaystyle=\left(\frac{\sqrt{2-\rho^{2}}}{\rho}-1\right)\,\phi(h)\,h\leq\phi(1)\left(\frac{\sqrt{2-\rho^{2}}}{\rho}-1\right)$

and $\Phi(-\theta h)\leq 0.5$. For the second term, we can compute

$\displaystyle\frac{\partial}{\partial
z_{v}}(Q(z_{v})-L(z_{v},z_{v};\bar{\rho}))=[2\Phi(-\theta
z_{v})-1]\,\phi(z_{v}).$

The derivative is always non-positive for $z_{v}\geq 0$ and is equal to 0 at
$z_{v}=0$, so the maximum value must be

$\displaystyle
Q(0)-L(0,0;\bar{\rho})=\frac{1}{2}-\left(\frac{1}{4}+\frac{1}{2\pi}\arcsin(\bar{\rho})\right).$

Together, this implies

$\displaystyle U(v;\rho)$
$\displaystyle\leq\phi(1)\left(\frac{\sqrt{2-\rho^{2}}}{\rho}-1\right)+\frac{1}{4}-\frac{1}{2\pi}\arcsin\left(\frac{1}{\sqrt{2-\rho^{2}}}\right)$
$\displaystyle=\phi(1)\left(\frac{\sqrt{2-\rho^{2}}}{\rho}-1\right)+\frac{1}{2\pi}\arctan\sqrt{1-\rho^{2}}$

For $x\geq 0$, we have $\arctan x\leq x$ as

$\displaystyle\frac{d}{dx}\arctan x=\frac{1}{1+x^{2}}\leq\frac{d}{dx}x=1$

and $\arctan x^{\prime}=x^{\prime}$ at $x^{\prime}=0$. We thus have

$\displaystyle U(v;\rho)$
$\displaystyle\leq\phi(1)\left(\frac{\sqrt{2-\rho^{2}}}{\rho}-1\right)+\frac{1}{2\pi}\sqrt{1-\rho^{2}}$

Finally, for sufficiently large $\rho$ we have that

$\frac{\sqrt{2-\rho^{2}}}{\rho}-1\leq\sqrt{1-\rho^{2}}.$

This follows because the roots of
$g(\rho)={\sqrt{2-\rho^{2}}}/{\rho}-1-\sqrt{1-\rho^{2}}$ occur at $\rho\approx
0.7184$ and $\rho=1$, $g(\rho)$ is continuous on $\rho\in(0,1]$ and $g(\rho)$
is negative for some point in between the two roots. For $\rho$ sufficiently
close to $1$, we thus have

$\displaystyle U(v;\rho)\leq K^{2}\sqrt{1-\rho^{2}}$ (A13)

for some finite and positive $K$. If $\rho_{n}\to 1$, this thus gives

$\sqrt{\kappa_{n}}\leq K(1-\rho_{n}^{2})^{1/4}$

for sufficiently large $n$. ∎

####  B.6.2 Almost supermartingale

If $\alpha_{n}=a(n+1)^{-1}$, Lemma A10 implies that setting

$1-\rho_{n}^{2}=O(n^{-k})$

for some $k>0$ is sufficient for
$\sum_{i=1}^{\infty}\alpha_{i}\sqrt{\kappa_{i}}<\infty$. Given Assumptions 3
and 4, we have that $L_{n}$ is an almost supermartingale, so we can apply
Theorem A4. This implies $L_{n}\to L_{\infty}<\infty$ a.s. and more
importantly,

$\sum_{i=1}^{\infty}\alpha_{i}T(Q_{i-1}^{\dagger})<\infty\quad\text{a.s.}$

As $\sum_{i=1}^{\infty}\alpha_{i}=\infty$, one can verify that the above
implies

$\liminf_{n}T(Q_{n}^{\dagger})=0\quad\text{a.s.}$

which implies there is a subsequence $n_{j}$ on which
$\lim_{n_{j}\to\infty}T(Q_{n_{j}}^{\dagger})=0$. We will now use the fact that
$Q^{*}$ is $M$-Lipschitz on $[0,1]$ from Assumption 5, which gives

$\displaystyle|Q^{*}(u)-Q(u)|\leq M|u-P^{*}(Q(u))|$

for all $u\in[0,1]$. We thus have

$\displaystyle T(Q_{n}^{\dagger})$
$\displaystyle=\int\,(Q^{*}(u)-Q^{\dagger}_{n}(u))(u-P^{*}(Q^{\dagger}_{n}(u)))\,du$
$\displaystyle=\int\,|Q^{*}(u)-Q^{\dagger}_{n}(u)|\,|u-P^{*}(Q^{\dagger}_{n}(u))|\,du$
$\displaystyle\geq
M^{-1}\int\,(Q^{*}(u)-Q^{\dagger}_{n}(u))^{2}\,du\,=M^{-1}L_{n}$

where the second equality follows from the positivity of $T(Q)$. Applying this
to the subsequence $n_{j}$ gives $L_{n_{j}}\leq MT(Q_{n_{j}}^{\dagger})$, so
$L_{n_{j}}\to 0$ along this subsequence. Since $L_{n}\to L_{\infty}$ a.s., we
have $L_{\infty}=0$ a.s.

###  B.7 Theorem 4

Consider now the same setting as the proof of Theorem 3. To begin, we first
show that $Q_{n}^{\dagger}$ satisfies Assumptions 1 and 2 for each $n$. We
note that the update function $(u-H_{\rho}(u,v))$ has continuous partial
derivative in $u$ for all $v\in[0,1]$ (as shown in (A9)). As a result, $Q_{1}$
is a sum of a Lipschitz function $Q_{0}$ and a continuously differentiable
function, so $Q_{1}\in H^{1}((0,1))$, and $Q_{1}^{\dagger}\in H^{1}((0,1))$ by
Theorem A2. Repeating the argument gives $Q_{n}^{\dagger}\in H^{1}((0,1))$,
thus satisfying Assumptions 1 and 2.

We now extend the probability space. For each $n$, define
$Q_{n\infty}^{\dagger}$ as the random function with realizations in
$H^{1}((0,1))$ arising from Algorithm 4 starting from $Q_{n}^{\dagger}$. The
existence of $Q_{n\infty}^{\dagger}\in H^{1}((0,1))$ is guaranteed by Theorem
1. There are a few possible constructions of this space, for example we can
let $Y_{1},Y_{2},\ldots$ be i.i.d. r.v.s from $P^{*}$, and independently let
$V_{1},V_{2},\ldots$ be i.i.d. r.v.s from $\mathcal{U}(0,1)$. We can then
define the following:

$\displaystyle
S_{n}(u)=\sum_{i=n+1}^{\infty}\alpha_{i}[u-H_{\rho_{i}}(u,V_{i})],\quad
Q_{n\infty}(u)=Q_{n}^{\dagger}(u)+S_{n}(u).$

Another option is to let $V_{n,n+1},V_{n,n+2},\ldots$ be distinct independent
sequences of uniform r.v.s for each $n$. Either way, this does not affect the
next step, as the distribution of $Q_{n\infty}^{\dagger}$ for each $n$ is
unchanged.

To show posterior consistency, we apply Markov’s inequality which gives

$\displaystyle\Pi_{n}\left(Q^{\dagger}_{n\infty}:d_{2}\left(Q^{\dagger}_{n\infty},Q^{*}\right)\geq\varepsilon\mid
Y_{1:n}\right)\leq\frac{1}{\varepsilon^{2}}\mathbb{E}\left[d^{2}_{2}\left(Q^{\dagger}_{n\infty},Q^{*}\right)\mid
Y_{1:n}\right].$

As $Q_{n}^{\dagger}$ and $Q_{n\infty}^{\dagger}$ are essentially bounded by
Proposition A2, we can apply Proposition 2 which gives
$d^{2}_{2}(Q^{\dagger}_{n\infty},Q^{*})\leq
d^{2}_{2}\left(Q_{n\infty},Q^{*}\right)$, and thus

$\displaystyle\Pi_{n}\left(Q^{\dagger}_{n\infty}:d_{2}\left(Q^{\dagger}_{n\infty},Q^{*}\right)\geq\varepsilon\mid
Y_{1:n}\right)\leq\frac{1}{\varepsilon^{2}}\mathbb{E}\left[d^{2}_{2}\left(Q_{n\infty},Q^{*}\right)\mid
Y_{1:n}\right].$

Applying the triangle inequality gives

$\displaystyle E\left[d^{2}_{2}\left(Q_{n\infty},Q^{*}\right)\mid
Y_{1:n}\right]\leq\mathbb{E}\left[\left(d_{2}\left(Q_{n\infty},Q_{n}^{\dagger}\right)+d_{2}\left(Q_{n}^{\dagger},Q^{*}\right)\right)^{2}\mid
Y_{1:n}\right]$
$\displaystyle=\mathbb{E}\left[d^{2}_{2}\left(Q_{n\infty},Q_{n}^{\dagger}\right)\mid
Y_{1:n}\right]+2d_{2}\left(Q_{n}^{\dagger},Q^{*}\right)\mathbb{E}\left[d_{2}\left(Q_{n\infty},Q_{n}^{\dagger}\right)\mid
Y_{1:n}\right]+d^{2}_{2}\left(Q_{n}^{\dagger},Q^{*}\right).$

To compute the first term, we have

$\displaystyle
d^{2}_{2}\left(Q_{N},Q_{n}^{\dagger}\right)=d^{2}_{2}(Q_{N-1},Q_{n}^{\dagger})+d^{2}_{2}\left(Q_{N},Q_{N-1}\right)-2\int\left(Q_{N}(u)-Q_{N-1}(u)\right)\left(Q_{N-1}(u)-Q_{n}^{\dagger}(u)\right)\,du.$

As $Q_{N}(u)$ is a martingale, we have that

$\displaystyle\mathbb{E}\left[d^{2}_{2}\left(Q_{N},Q_{n}^{\dagger}\right)\mid
Y_{1:N-1}\right]$
$\displaystyle=d^{2}_{2}(Q_{N-1},Q_{n}^{\dagger})+\alpha_{N}^{2}\int\,\int\left(u-H_{\rho_{N}}(u,v)\right)^{2}\,du\,dv$
$\displaystyle\leq d^{2}_{2}(Q_{N-1},Q_{n}^{\dagger})+\alpha_{N}^{2}.$

Iterating further, we have that

$\displaystyle\mathbb{E}\left[d^{2}_{2}\left(Q_{n\infty},Q_{n}^{\dagger}\right)\mid
Y_{1:n}\right]$
$\displaystyle\leq\sum_{i=n+1}^{\infty}\alpha_{i}^{2}=O(n^{-1}).$

Let us consider the second term. We have

$\displaystyle\mathbb{E}\left[d_{2}\left(Q_{n\infty},Q_{n}^{\dagger}\right)\mid
Y_{1:n}\right]$
$\displaystyle\leq\sqrt{E\left[d^{2}_{2}(Q_{n\infty},Q_{n}^{\dagger})\mid
Y_{1:n}\right]}$ $\displaystyle=O(n^{-1/2}).$

Putting this together, we have

$\displaystyle\Pi_{n}\left(Q^{\dagger}_{n\infty}:d_{2}\left(Q_{n\infty}^{\dagger},Q^{*}\right)\geq\varepsilon\mid
Y_{1:n}\right)\leq\frac{1}{\varepsilon^{2}}\left[O(n^{-1})+2O(n^{-1/2})\,d_{2}(Q_{n}^{\dagger},Q^{*})+d^{2}_{2}(Q_{n}^{\dagger},Q^{*})\right].$
(A14)

We thus have the above going to 0 as long as
$d^{2}_{2}(Q_{n}^{\dagger},Q^{*})\to 0$ $P^{*}$-a.s., which follows from
Theorem 3.

###  B.8 Theorem 5

This proof fortunately recycles many steps from the proof of Theorem 3, with
additional steps inspired by Aboubacar and Thiam [2014]. Let us begin again
with the almost supermartingale construction:

$\displaystyle\mathbb{E}[L_{n}\mid\mathcal{F}_{n-1}]\leq\left(1+2\alpha_{n}\sqrt{\kappa_{n}}\right)L_{n-1}-2\alpha_{n}T(Q_{n-1}^{\dagger})+\alpha_{n}^{2}+2\alpha_{n}\sqrt{\kappa_{n}},$

where $\sqrt{\kappa_{n}}=O(n^{-k/4})$, which is the error term controlled by
the bandwidth with $k>0$. From the Lipschitz condition in Assumption 5, we
again have

$\displaystyle T(Q_{n-1}^{\dagger})\geq M^{-1}L_{n-1}.$

Putting this together, we get

$\displaystyle\mathbb{E}[L_{n}\mid\mathcal{F}_{n-1}]$
$\displaystyle\leq\left[1-2\alpha_{n}\left(M^{-1}-\sqrt{\kappa_{n}}\right)\right]L_{n-1}+\alpha_{n}^{2}+2\alpha_{n}\sqrt{\kappa_{n}}$
$\displaystyle=\left[1-2\alpha_{n}(M^{-1}+O(n^{-k/4}))\right]L_{n-1}+\alpha_{n}^{2}+2\alpha_{n}\sqrt{\kappa_{n}}.$

Premultiplying by $n^{\delta}$ for $\delta<1$, we have

$\displaystyle\mathbb{E}[n^{\delta}L_{n}\mid\mathcal{F}_{n-1}]$
$\displaystyle\leq\left[1-2an^{-1}(M^{-1}+O(n^{-k/4}))\right][1+\delta
n^{-1}+O(n^{-2})](n-1)^{\delta}L_{n-1}$
$\displaystyle+O\left(n^{-(2-\delta)}\right)+O(n^{\delta-1-k/4})$

where we have used the fact that

$\displaystyle\left(\frac{n}{n-1}\right)^{\delta}=1+\frac{\delta}{n}+O(n^{-2})$

which is also used in Aboubacar and Thiam [2014]. Simplifying, we get

$\displaystyle\mathbb{E}[n^{\delta}L_{n}\mid\mathcal{F}_{n-1}]$
$\displaystyle\leq\left[1-n^{-1}(2aM^{-1}-\delta+{o}(1))\right](n-1)^{\delta}L_{n-1}+O\left(n^{-(2-\delta)}\right)+O(n^{\delta-1-k/4})$

If we choose $a$ such that $a>\frac{M\delta}{2}$, then
$2aM^{-1}-\delta+o(1)>0$ eventually for sufficiently large $n$. If we further
assume $k>4$, then the last terms is $O(n^{-(2-\delta)})$, so we just need
$\sum_{n=1}^{\infty}n^{-(2-\delta)}<\infty$ for the bounded variance
condition, which holds if $\delta<1$. Summarizing, we have that under
Assumption 6, for sufficiently large $n$, there exists some positive constant
$0<K<\infty$ such that:

$\displaystyle\mathbb{E}[n^{\delta}L_{n}\mid\mathcal{F}_{n-1}]$
$\displaystyle\leq(n-1)^{\delta}L_{n-1}+\underbrace{O\left(n^{-(2-\delta)}\right)}_{C_{n}}-\underbrace{n^{-1}K\,(n-1)^{\delta}L_{n-1}}_{D_{n}}$

where $B_{n}=0$ and $n^{\delta}L_{n}$ is an almost supermartingale, as
$C_{n},D_{n}\geq 0$ and $\sum_{i=1}^{\infty}C_{i}<\infty$ a.s. As a result,
Theorem A4 gives us $n^{\delta}L_{n}\to X_{\infty}$ a.s. under Assumption 6.
Finally, we have that $X_{\infty}=0$ a.s. which follows from
$\sum_{i=1}^{\infty}i^{-(1-\delta)}L_{i}<\infty$ a.s.

###  B.9 Theorem 6

We start again have from (A14)

$\displaystyle\Pi_{n}\left(Q^{\dagger}_{n\infty}:d_{2}\left(Q^{\dagger}_{n\infty},Q^{*}\right)\geq\varepsilon\mid
Y_{1:n}\right)$
$\displaystyle\leq\frac{1}{\varepsilon^{2}}\left[O(n^{-1})+2O(n^{-1/2})\,d_{2}(Q^{\dagger}_{n},Q^{*})+d_{2}^{2}(Q^{\dagger}_{n},Q^{*})\right].$

From Theorem 5, for each $0<\delta<1$, we have that

$\displaystyle\Pi_{n}\left(Q^{\dagger}_{n\infty}:d_{2}\left(Q^{\dagger}_{n\infty},Q^{*}\right)\geq\varepsilon\mid
Y_{1:n}\right)$
$\displaystyle\leq\frac{1}{\varepsilon^{2}}\left[O(n^{-1})+O(n^{-(\delta+1)/2})+O(n^{-\delta})\right]\quad\text{a.s.}[P^{*}]$
$\displaystyle\leq\frac{1}{\varepsilon^{2}}O(n^{-\delta})\quad\text{a.s.}[P^{*}]$

The above means that for all $\varepsilon>0$, for any $0<\delta<1$, there
exists some constant $B_{\delta}<\infty$ such that we have

$\displaystyle\Pi_{n}\left(Q^{\dagger}_{n\infty}:d_{2}(Q_{n\infty}^{\dagger},Q^{*})\geq\varepsilon\mid
Y_{1:n}\right)\leq\frac{B_{\delta}n^{-\delta}}{\varepsilon^{2}}\quad\text{a.s.}[P^{*}]$

for sufficiently large $n$. Now choose an arbitrary $0<\delta<1$, and also
choose $\delta<\delta^{\prime}<1$ with corresponding $B_{\delta^{\prime}}$. If
we plug-in $\varepsilon_{n}=Kn^{-\delta/2}$ as in Theorem 6 for an arbitrary
finite positive constant $K$, then we have

$\displaystyle\Pi_{n}(Q^{\dagger}_{n\infty}:d_{2}(Q_{n\infty}^{\dagger},Q^{*})\geq
K\varepsilon_{n}\mid Y_{1:n})$
$\displaystyle\leq\frac{B_{\delta^{\prime}}n^{-\delta^{\prime}}}{\varepsilon_{n}^{2}}\quad\text{a.s.}[P^{*}]$
$\displaystyle=\frac{B_{\delta^{\prime}}}{K^{2}}n^{-\left(\delta^{\prime}-\delta\right)}\quad\text{a.s.}[P^{*}]$

for sufficiently large $n$. Since $\delta^{\prime}>\delta$ can always be
chosen, we have that the above goes to 0 with $n$ for any $K>0$.

###  B.10 Proposition 5

We showed in the proof of Theorem 4 that $Q_{n}^{\dagger}$ satisfies
Assumptions 1 and 2, so we can apply Proposition 1 or Theorem 1. The
probability space can be constructed in the same way as Theorem 4.

Fix $n\geq 1$ and define the filtration $\\{\mathcal{F}_{N}\\}_{N\geq n}$ with
$\mathcal{F}_{N}=\sigma(Y_{1},\ldots,Y_{n},V_{n+1},\ldots,V_{N})$ and
$\mathcal{F}_{n}=\sigma(Y_{1},\ldots,Y_{n})$. We will write
$\mathbb{E}[\cdot\mid\mathcal{F}_{n}]$ as $\mathbb{E}[\cdot\mid Y_{1:n}]$ to
make it clear that it is conditioned on ‘real’ data. To begin, we highlight
the very useful property that

$\displaystyle\mu_{nN}:=\int_{0}^{1}Q_{nN}^{\dagger}(u)\,du=\int_{0}^{1}Q_{nN}(u)\,du$

which follows directly from Lemma A1 with $h(x)=x$. The above also holds for
$N=\infty$, where the existence of $Q_{n\infty}$ is guaranteed by Proposition
1.

This is particularly convenient as we do not need to consider the
rearrangement procedure to study the distribution of $\mu_{nN}$. Note that
this property is not unique to the mean functional. Another subtle but
important point is that $\\{\mu_{nN}\\}_{N\geq n}$ is a martingale, even if
$Q_{nN}^{\dagger}$ is not, due to the above property, which follows from

$\displaystyle\mathbb{E}\left[\mu_{nN}\mid\mathcal{F}_{N-1}\right]$
$\displaystyle=\int_{0}^{1}\mathbb{E}\left[Q_{nN}(u)\mid\mathcal{F}_{N-1}\right]\,du$
$\displaystyle=\int_{0}^{1}Q_{n,N-1}(u)du$ $\displaystyle=\mu_{n,N-1}.$

This arises from the linearity of the mean, which is unique to the mean and
does not apply for other functionals. We can then directly show that $\mu_{N}$
is bounded in $L^{2}$, as $E[X]^{2}\leq\mathbb{E}[X^{2}]$ gives

$\displaystyle\sup_{N\geq n}\mathbb{E}\left[\mu_{nN}^{2}\mid
Y_{1:n}\right]\leq\sup_{N\geq n}\mathbb{E}\left[\|Q_{nN}\|^{2}_{2}\mid
Y_{1:n}\right]<\infty,$

where the boundedness was shown in Proposition 1. As a result, $\mu_{nN}$ is a
martingale bounded in $L^{2}$, so there exists a finite
$\tilde{\mu}_{n\infty}$ such that $\mu_{nN}\to\tilde{\mu}_{n\infty}$ a.s. and
$E[\tilde{\mu}_{n\infty}\mid Y_{1:n}]=\mu_{n}$. Finally, we have

$\displaystyle(\mu_{n\infty}-\mu_{nN})^{2}\leq
d_{2}^{2}(Q_{n\infty},Q_{nN})\to 0\quad\textnormal{a.s.}$

from Proposition 1, so $\tilde{\mu}_{n\infty}=\mu_{n\infty}$ a.s. We thus have
the first part of Proposition 5, that is $E[\mu_{n\infty}\mid
Y_{1:n}]=\mu_{n}$ a.s. for each $n$.

For the posterior variance, we note that

$\displaystyle\mathbb{E}[(\mu_{n\infty}-\mu_{n})^{2}\mid
Y_{1:n}]=\sum_{i=n+1}^{\infty}\sum_{j=n+1}^{\infty}\alpha_{i}\alpha_{j}\mathbb{E}[Z_{i}Z_{j}]$

where $Z_{i}=\int_{0}^{1}(u-H_{\rho_{i}}(u,V_{i}))\,du$. As $Z_{i}$ is
independent of $Z_{j}$ for $i\neq j$ and are zero-mean, the cross-terms are
zero, so we just have

$\displaystyle\mathbb{E}[(\mu_{n\infty}-\mu_{n})^{2}\mid
Y_{1:n}]=\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\,\mathbb{E}[Z_{i}^{2}].$

We can then show from Lemma A7 that

$\displaystyle\mathbb{E}[Z_{i}^{2}]=\int_{0}^{1}\int_{0}^{1}C_{\rho_{i}^{2}}(u,v)\,du\,dv-\frac{1}{4}\cdot$

As $C_{\rho}(u,v)\leq 1$ and $\lim_{\rho\to 1}C_{\rho^{2}}(u,v)=\min(u,v)$,
dominated convergence gives

$\displaystyle\lim_{i\to\infty}\mathbb{E}[Z_{i}^{2}]$
$\displaystyle=\int_{0}^{1}\int_{0}^{1}\min(u,v)\,du\,dv-\frac{1}{4}$
$\displaystyle=\frac{1}{12}$

Scaling by ${n}$ gives us

$\displaystyle{n}\,\mathbb{E}[(\mu_{n\infty}-\mu_{n})^{2}\mid Y_{1:n}]$
$\displaystyle=a^{2}\sum_{i=n+1}^{\infty}\frac{{n}}{(i+1)^{2}}\,\mathbb{E}[Z_{i}^{2}].$

Now consider

$\displaystyle\left|\sum_{i=n+1}^{\infty}\frac{{n}}{(i+1)^{2}}\,\mathbb{E}[Z_{i}^{2}]-\frac{1}{12}\right|\leq\left|\sum_{i=n+1}^{\infty}\frac{{n}}{(i+1)^{2}}\,\left(\mathbb{E}[Z_{i}^{2}]-\frac{1}{12}\right)+\frac{1}{12}\left(\sum_{i=n+1}^{\infty}\frac{{n}}{(i+1)^{2}}-1\right)\right|$
$\displaystyle\leq\sum_{i=n+1}^{\infty}\frac{{n}}{(i+1)^{2}}\,\left|\mathbb{E}[Z_{i}^{2}]-\frac{1}{12}\right|+\frac{1}{12}\left|\sum_{i=n+1}^{\infty}\frac{{n}}{(i+1)^{2}}-1\right|.$

For each $\varepsilon>0$, for sufficiently large $n$, we have $\sup_{i\geq
n+1}\left|\mathbb{E}[Z_{i}^{2}]-\frac{1}{12}\right|<\varepsilon$. Furthermore,
we have

$\displaystyle\left|n\sum_{i=n+1}^{\infty}\frac{1}{(i+1)^{2}}-1\right|\to 0,$

which follows as $(n+2)^{-1}\leq\sum_{i=n+1}^{\infty}(i+1)^{-2}\leq(n+1)^{-1}$
from the integral test. We thus have

$\displaystyle{n}\,\mathbb{E}[(\mu_{n\infty}-\mu_{n})^{2}\mid
Y_{1:n}]\to\frac{a^{2}}{12}\quad\textnormal{a.s.}[P^{*}]$

Note that the result does not depend on the convergence of $\mu_{n}$, as this
will always act as the center of the posterior.

###  B.11 Theorem 7

We begin with showing the inequality. Once again, we have

$\displaystyle
S_{n}(u)=\sum_{i=n+1}^{\infty}\alpha_{i}(u-H_{\rho_{i}}(u,V_{i}))$

where $V_{i}\overset{\mathrm{iid}}{\sim}\mathcal{U}(0,1)$ and the existence of
the random function $S_{n}$ with realizations in $H^{1}((0,1))$ is guaranteed
by Theorem 1 as $\alpha_{i}$ and $\rho_{i}$ satisfy Assumptions 3 and 4
respectively. It is clear that this function has mean 0. To begin, we have the
following lemma.

######  Lemma A11.

The covariance function
$k_{n}(u,u^{\prime}):=\mathbb{E}[S_{n}(u)\,S_{n}(u^{\prime})]$ takes the form

$\displaystyle
k_{n}(u,u^{\prime})=\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\,[C_{\rho_{i}^{2}}(u,u^{\prime})-uu^{\prime}].$

###### Proof.

A direct calculation gives

$\displaystyle k_{n}(u,u^{\prime})$
$\displaystyle=\sum_{i=n+1}^{\infty}\alpha_{i}\alpha_{j}\mathbb{E}\left[\left(u-H_{\rho_{i}}(u,V_{i})\right)\left(u^{\prime}-H_{\rho_{j}}(u^{\prime},V_{j})\right)\right]$
$\displaystyle=\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\left[C_{\rho_{i}^{2}}(u,u^{\prime})-uu^{\prime}\right].$

The cross-terms are zero, so we can invoke Lemma A7 which gives the last line.
∎

To get the inequalities, we note that $\rho_{i}^{2}\to 1$ and
$\rho_{i}^{2}\leq\rho_{j}^{2}$ for $i\leq j$, so we can apply the ordering
property of the bivariate Gaussian copula from Section A.5, where the ordering
holds uniformly over $u,u^{\prime}\in(0,1)$. Note that $\lim_{\rho\to
1}C_{\rho^{2}}(u,u^{\prime})=\min\\{u,u^{\prime}\\}$, so the squeeze theorem
gives the convergence to the Brownian bridge covariance function as
$\rho_{n}s\to 1$ for both $k_{\rho_{n}}$ and $r_{n}^{-1}k_{n}$.

We now turn our focus to the Gaussian process
$\mathbb{G}_{\rho}\sim\mathcal{GP}(0,C_{\rho^{2}}(u,u^{\prime})-uu^{\prime})$
for some $\rho\in(0,1)$ as in the approximate sampling scheme for the QMP. We
will show that sample paths of $\mathbb{G}_{\rho}$ are in $H^{1}((0,1))$ using
Scheuerer [2010, Theorem 1]. This depends on properties of the partial
derivatives of the kernel function, which exists and is equal to

$\displaystyle\frac{\partial^{2}}{\partial u\partial
v}[C_{\rho}(u,v)-uv]=c_{\rho}(u,v)-1.$

We then have the following lemma.

######  Lemma A12.

The bivariate copula density $c_{\rho}(u,v)$ satisfies

$\displaystyle\int_{0}^{1}[c_{\rho}(u,u)-1]\,du<\infty.$

###### Proof.

We have the following from a change of variables $u\to z_{u}=\Phi^{-1}(u)$:

$\displaystyle\int_{0}^{1}[c_{\rho}(u,u)-1]\,du$
$\displaystyle=\frac{1}{\sqrt{1-\rho^{2}}}\int_{-\infty}^{\infty}\exp\left(\frac{2\rho
z_{u}^{2}-2\rho^{2}z_{u}^{2}}{2(1-\rho^{2})}\right)\phi(z_{u})\,dz_{u}-1$
$\displaystyle=\frac{1}{\sqrt{2\pi(1-\rho^{2})}}\int_{-\infty}^{\infty}\exp\left(\frac{z_{u}^{2}\left(2\rho-1-\rho^{2}\right)}{2(1-\rho^{2})}\right)\,dz_{u}-1$
$\displaystyle=\frac{1}{\sqrt{2\pi(1-\rho^{2})}}\int_{-\infty}^{\infty}\exp\left(-\frac{z_{u}^{2}(1-\rho)}{2(1+\rho)}\right)\,dz_{u}-1<\infty$

where the finiteness follows as $(1-\rho)/(1+\rho)>0$. ∎

Continuity of $c_{\rho}(u,u)$ for all $u\in(0,1)$ and the above lemma means
that the covariance function $k_{\rho}$ satisfies the conditions of Scheuerer
[2010, Theorem 1], so sample paths of $\mathbb{G}_{\rho}$ are in
$H^{1}((0,1))=W^{1,2}((0,1))$ a.s.

Finally, we show the weak convergence to the Brownian motion. Marginal
convergence is quite obvious as the covariance function approaches
$\min\\{u,u^{\prime}\\}-uu^{\prime}$ with $n\to\infty$. However, showing
tightness of the sequence of GPs requires a bit more work.

######  Lemma A13.

The sequence $\mathbb{G}_{\rho_{n}}$ is asymptotically tight in
$\ell^{\infty}((0,1))$.

###### Proof.

We begin by computing the standard deviation semimetric of
$\mathbb{G}_{\rho_{n}}$:

$\displaystyle\mathbb{E}\left[\left(\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)\right)^{2}\right]$
$\displaystyle=\mathbb{E}\left[\mathbb{G}_{\rho_{n}}(u)^{2}\right]+\mathbb{E}\left[\mathbb{G}_{\rho_{n}}(v)^{2}\right]-2\mathbb{E}\left[\mathbb{G}_{\rho_{n}}(u)\,\mathbb{G}_{\rho_{n}}(v)\right]$
$\displaystyle=C_{\rho_{n}^{2}}(u,u)-u^{2}+C_{\rho_{n}^{2}}(v,v)-{v}^{2}-2\left(C_{\rho_{n}^{2}}(u,v)-uv\right)$

From Meyer [2013], we have the following property:

$\displaystyle\min\\{u,v\\}-C_{\rho}(u,v)=\int_{\rho}^{1}\phi_{2}\left(z_{u},z_{v};\rho\right)\,dr$

where

$\displaystyle\phi_{2}(z_{u},z_{v})$
$\displaystyle=\frac{1}{2\pi\sqrt{1-\rho^{2}}}\exp\left(-\frac{z_{u}^{2}+z_{v}^{2}-2\rho
z_{u}z_{v}}{2(1-\rho^{2})}\right).$

For $\mathbb{G}\sim\mathcal{GP}(0,\min\\{u,u^{\prime}\\}-uu^{\prime})$ as the
Brownian bridge, we have

$\displaystyle\mathbb{E}\left[\left(\mathbb{G}(u)-\mathbb{G}(v)\right)^{2}\right]-\mathbb{E}\left[\left(\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)\right)^{2}\right]=\frac{1}{2\pi}\int_{\rho_{n}^{2}}^{1}\frac{g_{r}(z_{u},z_{v})}{\sqrt{1-r^{2}}}\,dr$

where

$\displaystyle
g_{r}(z_{u},z_{v})=\exp\left(-\frac{z_{u}^{2}}{1+r}\right)+\exp\left(-\frac{z_{v}^{2}}{1+r}\right)-2\exp\left(-\frac{z_{u}^{2}+z_{v}^{2}-2rz_{u}z_{v}}{2(1-r^{2})}\right).$

Completing the square gives

$\displaystyle g_{r}(z_{u},z_{v})$
$\displaystyle=\left[\exp\left(-\frac{z_{u}^{2}}{2(1+r)}\right)-\exp\left(-\frac{z_{v}^{2}}{2(1+r)}\right)\right]^{2}+2\left[\exp\left(-\frac{z_{u}^{2}+z_{v}^{2}}{2(1+r)}\right)-\exp\left(-\frac{z_{u}^{2}+z_{v}^{2}-2rz_{u}z_{v}}{2(1-r^{2})}\right)\right]$

The second term can be written as

$\displaystyle\exp\left(-\frac{(z_{u}^{2}+z_{v}^{2})(1-r)}{2(1-r^{2})}\right)-\exp\left(-\frac{z_{u}^{2}+z_{v}^{2}-2rz_{u}z_{v}}{2(1-r^{2})}\right)$
$\displaystyle=\exp\left(-\frac{(z_{u}^{2}+z_{v}^{2})(1-r)}{2(1-r^{2})}\right)\left[1-\exp\left(-\frac{r(z_{u}^{2}+z_{v}^{2}-2z_{u}z_{v})}{2(1-r^{2})}\right)\right]$
$\displaystyle=\exp\left(-\frac{(z_{u}^{2}+z_{v}^{2})(1-r)}{2(1-r^{2})}\right)\left[1-\exp\left(-\frac{r(z_{u}-z_{v})^{2}}{2(1-r^{2})}\right)\right]$

Since $(z_{u}-z_{v})^{2}\geq 0$, we have that the above is non-negative, so
$g_{r}(z_{u},z_{v})\geq 0$ for all $z_{u},z_{v}\in\mathbb{R}$. This thus gives

$\displaystyle\mathbb{E}\left[\left(\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)\right)^{2}\right]\leq\mathbb{E}\left[\left(\mathbb{G}(u)-\mathbb{G}(v)\right)^{2}\right]$
(A15)

for all $n$. Consider the semimetric space $(\mathcal{F}=(0,1),d)$ where
$d(u,v)=|u-v|^{1/2}$. It is clear that $\mathcal{F}$ is totally bounded under
this semimetric. Let $d_{n}$ and $d_{\infty}$ denote the standard deviation
semimetrics of $\mathbb{G}_{\rho_{n}}$ and $\mathbb{G}$ respectively, which
are

$\displaystyle d_{n}^{2}(u,v)$
$\displaystyle=\mathbb{E}\left[\left(\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)\right)^{2}\right]$
$\displaystyle d_{\infty}^{2}(u,v)$
$\displaystyle=\mathbb{E}\left[\left(\mathbb{G}(u)-\mathbb{G}(v)\right)^{2}\right]=|u-v|(1-|u-v|).$

As $|u-v|<1$, we have $d_{\infty}(u,v)\leq d(u,v)$, which combined with (A15)
gives

$\displaystyle d_{n}(u,v)\leq d_{\infty}(u,v)\leq d(u,v)$

for all $u,v\in(0,1)$ and $n$.

Let $D(\varepsilon,d)$ denote the packing number of the space
$(\mathcal{F},d)$. Van der Vaart and Wellner [2023, Corollary 2.2.9] states
that for $X$ as a separable Gaussian process with $d_{X}$ as its standard
deviation semimetric, we have for every $\delta>0$:

$\displaystyle\mathbb{E}\left[\sup_{d_{X}(u,v)\leq\delta}|X(u)-X(v)|\right]\leq
K\int_{0}^{\delta}\sqrt{\log D(\varepsilon,d_{X})}\,d\varepsilon$

for a universal constant $K$.

In particular, as we have $d_{n}(u,v)\leq d(u,v)$, this implies that
$\\{u,v:d(u,v)\leq\delta\\}\subseteq\\{u,v:d_{n}(u,v)\leq\delta\\}$, which
gives

$\displaystyle\mathbb{E}\left[\sup_{d(u,v)\leq\delta}|\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)|\right]\leq\mathbb{E}\left[\sup_{d_{n}(u,v)\leq\delta}|\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)|\right]\leq
K\int_{0}^{\delta}\sqrt{\log D(\varepsilon,\rho_{n})}\,d\varepsilon.$

Furthermore, since $d_{n}(u,v)\leq d(u,v)$ where $d(u,v)=|u-v|^{1/2}$, the
packing numbers similarly satisfy

$\displaystyle D(\varepsilon,d_{n})\leq D(\varepsilon,d).$

Under the semimetric $d$, the packing number for any $\varepsilon>0$ satisfies

$\displaystyle D(\varepsilon,d)\leq\frac{C}{\varepsilon^{2}}$

for a universal constant $C$. We thus have

$\displaystyle\int_{0}^{\delta}\sqrt{\log D(\varepsilon,\rho)}\,d\varepsilon$
$\displaystyle=\int_{0}^{\delta}\sqrt{\log
C+2\log\varepsilon^{-1}}\,d\varepsilon$
$\displaystyle\leq\sqrt{\int_{0}^{\delta}(\log
C+2\log\varepsilon^{-1})\,d\varepsilon}$ $\displaystyle=\sqrt{\delta\left[\log
C+2(1+\log\delta^{-1})\right]}$

where the second line follows from Jensen’s inequality. The above can be made
arbitrarily small by decreasing $\delta$. Finally, Markov’s inequality gives

$\displaystyle\limsup_{n}\mathbb{P}\left(\sup_{d(u,v)\leq\delta}|\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)|>\varepsilon\right)$
$\displaystyle\leq\varepsilon^{-1}\limsup_{n}\mathbb{E}\left[\sup_{d(u,v)\leq\delta}|\mathbb{G}_{\rho_{n}}(u)-\mathbb{G}_{\rho_{n}}(v)|\right]$
$\displaystyle\leq\varepsilon^{-1}\sqrt{\delta\left[\log
C+2(1+\log\delta^{-1})\right]}.$

We can make the right hand side less than any $\eta>0$ by sufficiently
decreasing $\delta$, so $\mathbb{G}_{\rho_{n}}$ is asymptotically uniformly
$d$-equicontinuous in probability.

For each $u\in(0,1)$, uniform tightness of the sequence
$\mathbb{G}_{\rho_{n}}(u)\sim\mathcal{N}(0,C_{\rho_{n}^{2}}(u,u)-u^{2})$ can
be verified with

$\displaystyle\sup_{n}\,\mathbb{P}\left(|\mathbb{G}_{\rho_{n}}(u)|>M\right)\leq\sup_{n}\frac{\mathbb{E}\left[\mathbb{G}_{\rho_{n}}(u)^{2}\right]}{M^{2}}\leq\frac{u(1-u)}{M^{2}}\cdot$

For any $\varepsilon>0$, we can choose $M^{2}>u(1-u)/\varepsilon$ which gives
$\sup_{n}\,\mathbb{P}\left(|\mathbb{G}_{\rho_{n}}(u)|>M\right)<\varepsilon$.
From Van der Vaart and Wellner [2023, Theorem 1.5.7], the sequence
$\mathbb{G}_{\rho_{n}}$ is asymptotically tight in $\ell^{\infty}((0,1))$. ∎

Finally, the marginals of $\mathbb{G}_{\rho_{n}}$ are simply zero-mean
Gaussian vectors with covariance matrix with entries
$C_{\rho_{n}^{2}}(u_{i},u_{j})-u_{i}u_{j}$. Each entry converges pointwise
from below to $\min\\{u_{i},u_{j}\\}-u_{i}u_{j}$ as $\rho_{n}\to 1$, so from
Lévy’s continuity theorem, the marginals converge to a zero-mean Gaussian
vector with the appropriate covariance matrix. From Van der Vaart and Wellner
[2023, Theorem 1.5.4], $\mathbb{G}_{\rho_{n}}$ converges weakly to
$\mathbb{G}$ in $\ell^{\infty}((0,1))$.

###  B.12 Theorem 8

The additional required assumptions of Theorem 8 are as follows, which is
analogous to Assumptions 1 and 2 from the unconditional version.

######  Assumption A1 (Bounded in $L^{2}$).

For each $j\in\\{1,\ldots,p\\}$, $\beta_{nj}$ satisfies
$\|\beta_{nj}\|_{2}<\infty$.

######  Assumption A2 (Weak derivatives bounded in $L^{2}$).

For each $j\in\\{1,\ldots,p\\}$, $\beta_{nj}$ is weakly differentiable with
weak derivative $\beta^{\prime}_{nj}$ which satisfies
$\|\beta^{\prime}_{nj}\|_{2}<\infty$, so $\|\beta_{nj}\|_{1,2}<\infty$.

The proof is an extension of Theorem 1, with the additional complication of
random covariates $X_{n+1:\infty}$ arising from the Bayesian bootstrap. Let
$B^{\prime}=H^{1}((0,1))$, and consider a single component
$j\in\\{1\ldots,p\\}$. We then have the update

$\displaystyle\beta_{N+1,j}(u)=\beta_{Nj}(u)+\alpha_{N+1}\left[u-H_{\rho_{N+1}}(u,V_{N+1})\right]X_{N+1,j}$

for each $u\in(0,1)$ and $N\geq n$, where
$V_{N+1}\overset{\mathrm{iid}}{\sim}\mathcal{U}(0,1)$ and $X_{N+1}\mid
X_{1:N}\sim\frac{1}{N}\sum_{i=1}^{N}\delta_{X_{i}}$. Our filtration now
consists of
$\mathcal{F}_{N}=\sigma\left(X_{n+1},V_{n+1},\ldots,X_{N},V_{N}\right)$ for
$N\geq n+1$, with $\mathcal{F}_{n}=\\{\emptyset,\Omega\\}$ again.

The above is again a pointwise martingale, as we have
$\int_{0}^{1}H_{\rho}(u,v)\,dv=u$ which gives

$\displaystyle\mathbb{E}\left[\beta_{N+1,j}(u)\mid\mathcal{F}_{N}\right]$
$\displaystyle=\mathbb{E}\left[\mathbb{E}\left[\beta_{N+1,j}(u)\mid\mathcal{F}_{N},X_{N+1}\right]\mid\mathcal{F}_{N}\right]$
$\displaystyle=\mathbb{E}\left[\beta_{Nj}(u)\mid\mathcal{F}_{N}\right]$
$\displaystyle=\beta_{Nj}(u).$

Note that the (conditional) distribution of $X_{N+1}$ does not affect the
martingale. The argument using continuous bounded functionals in Lemma A9 can
be repeated here to show that $\\{\beta_{Nj}\\}_{N\geq n+1}$ is a
$B^{\prime}$-valued martingale if $\beta_{Nj}\in B^{\prime}$. We can then
upper bound the $L^{2}(B^{\prime})$ norm as in Theorem 1 with

$\displaystyle\|\beta_{Nj}\|_{L^{2}(B^{\prime})}\leq\|\beta_{nj}\|_{1,2}+\sup_{k\in\\{1,\ldots,n\\}}X_{kj}^{2}\sum_{i=n+1}^{N}\frac{\alpha_{i}^{2}}{1-\rho_{i}^{2}},$

where we have used the fact that $X_{n+1:N}$ will be repeats of $X_{1:n}$ and
$\sup_{k\in\\{1,\ldots,n\\}}X_{kj}^{2}$ is finite as we only have finitely
many (i.e. $n$) covariate observations. Under Assumptions 3, 4, A1 and A2, we
thus have $\beta_{Nj}\in B^{\prime}$ for each $N$ and $\sup_{N\geq
n}\|\beta_{Nj}\|_{L^{2}(B^{\prime})}<\infty$, so we can apply Theorem A1.

We can repeat the above for all components $j\in(1,\ldots,p)$, and as $p$ is
finite, the union of the null sets on which convergence does not occur for
each component has measure 0, so the vector $\beta_{N}$ converges to
$\beta_{\infty}$ component-wise a.s.

We now describe the space of vector functions $\beta_{N}$ and
$\beta_{\infty}$, which we will need for a later proof. Consider the finite
product of Banach spaces

$\displaystyle H^{1}((0,1))^{p}:=H^{1}((0,1))\times\ldots\times H^{1}((0,1)).$

For a vector $f\in H^{1}((0,1))^{p}$, we define the norm of this Banach space
as

$\displaystyle\|f\|_{1,2,p}:=\sum_{j=1}^{p}\|f_{j}\|_{1,2}.$

It is clear that $\beta_{N},\beta_{\infty}\in H^{1}((0,1))^{p}$, and as each
component converges a.s., we have

$\displaystyle\|\beta_{N}-\beta_{\infty}\|_{1,2,p}\to 0\quad\text{a.s.}$

###  B.13 Proposition 6

For an arbitrary $x\in\mathcal{X}$, consider the mapping
$h_{x}:H^{1}((0,1))^{p}\to H^{1}((0,1))$ defined by

$\displaystyle h_{x}(\beta)=\sum_{j=1}^{p}\beta_{j}x_{j}$

where $x_{j}\in\mathbb{R}$ is the $j$-th component of $x$. This mapping can be
shown to be continuous as follows. Consider a sequence
$\beta_{N}\to\beta_{\infty}$ in $H^{1}((0,1))^{p}$, then we have

$\displaystyle\|h_{x}(\beta_{N})-h_{x}(\beta_{\infty})\|_{1,2}$
$\displaystyle\leq\sum_{j=1}^{p}\|(\beta_{Nj}-\beta_{\infty
j})\,x_{j}\|_{1,2}$
$\displaystyle=\sum_{j=1}^{p}|x_{j}|\,\|\beta_{Nj}-\beta_{\infty j}\,\|_{1,2}$
$\displaystyle\leq\sup_{j}|x_{j}|\,\|\beta_{N}-\beta_{\infty}\|_{1,2,p}\to 0,$

where we have applied the triangle inequality in the first step. For
$Q_{N}(\cdot\mid x):=h_{x}(\beta_{N})$ and $Q_{\infty}(\cdot\mid
x):=h_{x}(\beta_{\infty})$ from Theorem 8, the continuous mapping theorem then
gives

$\displaystyle d_{1,2}\left(Q_{N}(\cdot\mid x),Q_{\infty}(\cdot\mid
x)\right)\to 0\quad\text{a.s.}$

From Theorem A2, we have that $Q_{N}^{\dagger}(\cdot\mid x)$ and
$Q_{\infty}(\cdot\mid x)$ are in $H^{1}((0,1))$. We can thus apply Proposition
A3 to give

$\displaystyle d_{2}(Q^{\dagger}_{N}(\cdot\mid
x),Q^{\dagger}_{\infty}(\cdot\mid x))\to 0\quad\text{a.s.}$

###  B.14 Proposition 7

For each $x\in\mathcal{X}$, we can once again apply Lemma A1 with $h(x)=x$,
which gives

$\displaystyle\int_{0}^{1}Q^{\dagger}_{\infty}(u\mid x)\,du$
$\displaystyle=\int_{0}^{1}Q_{\infty}(u\mid x)\,du$
$\displaystyle=\int_{0}^{1}\beta_{\infty}(u)^{T}x\,du\quad\textnormal{a.s.}$

Linearity of expectation then gives $\mathbb{E}_{\infty}\left[Y\mid
x\right]=\left[\int_{0}^{1}\beta_{\infty}(u)\,du\right]^{T}x$, which is a
(random) linear function in $x$ a.s.

###  B.15 Theorem 9

We follow the same strategy as the proof for Theorem 2. We will use Theorem A3
to verify asymptotic tightness, and then we establish marginal convergence. To
reduce clutter, we will suppress the conditioning on the weights in the
notation. Define

$C_{X}=\max_{\begin{subarray}{c}i\in\\{1,\ldots,n\\}\\\
j\in\\{1,\ldots,p\\}\end{subarray}}|X_{ij}|.$

######  Theorem A6.

The sequence of functions $\sqrt{N}S_{N}$ is asymptotically tight in
$\ell^{\infty}(\mathcal{F})$ with probability 1.

###### Proof.

We verify the assumptions in Theorem A3. Our semimetric space is
$(\mathcal{F}=(0,1)\times\\{1,\ldots,p\\},d)$, where
$d((u_{1},j_{1}),(u_{2},j_{2}))=|u_{1}-u_{2}|^{1/2}+\mathbbm{1}(j_{1}\neq
j_{2})$. We have used the discrete metric on $\\{1,\ldots,p\\}$ and then
specified the sum of the two semimetrics to define the semimetric product
space. Clearly, this semimetric space is totally bounded.

First define $Z_{Ni}(u,j)=\sqrt{N}\alpha_{i}(H_{\rho_{i}}(u,V_{i})-u)X_{ij}$
for all $N$ and $i\geq N$. Note that this definition differs in nature to the
non-regression case (where the $-u$ term can be omitted) because the
randomness in the covariates must be accounted for. A trivial envelope
function for $Z_{Ni}$ is $F_{Ni}=2\sqrt{N}\alpha_{i}C_{X}$. We need to verify
the Lindeberg condition

$\sum_{i=N}^{\infty}F_{Ni}\mathbbm{1}\\{F_{Ni}>\eta\\}\rightarrow 0$

for every $\eta>0$. Since $F_{Ni}<2cC_{X}/\sqrt{N}$, we will have
$F_{Ni}<\eta$ for all sufficiently large $N$. Thus, the Lindeberg condition
holds.

Next we need

$\sup_{d((u_{1},j_{1}),(u_{2},j_{2}))<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}\left\\{Z_{Ni}(u_{1})-Z_{Ni}(u_{2})\right\\}^{2}\rightarrow
0$

for every $\delta_{N}\downarrow 0$. For all sufficiently large $N$, we must
have $\delta_{N}<1$, in which case

$d((u_{1},j_{1}),(u_{2},j_{2}))<\delta_{N}\implies
j_{1}=j_{2}\quad\text{and}\quad|u_{1}-u_{2}|^{1/2}<\delta_{N}.$

Let $u_{1}>u_{2}$ with $|u_{1}-u_{2}|^{1/2}<\delta_{N}$. We have

$\displaystyle\mathbb{E}\left\\{Z_{Ni}(u_{1},j)-Z_{Ni}(u_{2},j)\right\\}^{2}$
$\displaystyle\leq
2\mathbb{E}\left\\{\sqrt{N}\alpha_{i}X_{ij}H_{\rho_{i}}(u_{1},V_{i})-\sqrt{N}\alpha_{i}X_{ij}H_{\rho_{i}}(u_{2},V_{i})\right\\}^{2}$
$\displaystyle+2\mathbb{E}\left\\{\sqrt{N}\alpha_{i}X_{ij}u_{1}-\sqrt{N}\alpha_{i}X_{ij}u_{2}\right\\}^{2}$
$\displaystyle\leq
2N\alpha_{i}^{2}C_{X}^{2}\mathbb{E}\left\\{H_{\rho_{i}}(u_{1},V_{i})-H_{\rho_{i}}(u_{2},V_{i})\right\\}^{2}$
$\displaystyle+2N\alpha_{i}^{2}C_{X}^{2}\delta_{N}^{2}$

Note that $H_{\rho_{i}}(u,V_{i})$ is non-decreasing in $u$ with
$H_{\rho_{i}}(0,V_{i})=0$ and $H_{\rho_{i}}(1,V_{i})=1$. So

$\displaystyle\mathbb{E}\left\\{H_{\rho_{i}}(u_{1},V_{i})-H_{\rho_{i}}(u_{2},V_{i})\right\\}^{2}$
$\displaystyle\leq\mathbb{E}\left\\{H_{\rho_{i}}(u_{1},V_{i})-H_{\rho_{i}}(u_{2},V_{i})\right\\}$
$\displaystyle=(u_{1}-u_{2})$ $\displaystyle<\delta_{N}^{2}.$

Thus, for all sufficiently large $N$ such that $\delta_{N}<1$, we have

$\sup_{d((u_{1},j_{1}),(u_{2},j_{2}))<\delta_{N}}\sum_{i=N}^{\infty}\mathbb{E}\left\\{Z_{Ni}(u_{1},j_{1})-Z_{Ni}(u_{2},j_{2})\right\\}^{2}<4N\delta_{N}^{2}C_{X}^{2}\sum_{i=N}^{\infty}\alpha_{i}^{2}.$

Since $\limsup_{N\rightarrow\infty}N\sum_{i=N}^{\infty}\alpha_{i}^{2}\leq
c^{2}$, the right-hand side of the above display tends to zero for any
$\delta_{N}\downarrow 0$.

Finally, we need to verify the bracketing entropy integral condition. Fix
$j\in\\{1,\ldots,p\\}$ for the time being. For all sufficiently large $n$, we
will have $N\sum_{i=N}^{\infty}\alpha_{i}^{2}\leq 2c^{2}$. Given
$\varepsilon>0$, choose a partition $0=u_{0}<u_{1}<\ldots<u_{M}=1$ such that
$u_{k}-u_{k-1}<\varepsilon^{2}/(8C_{X}^{2}c^{2})$ for every $k$. The number of
points in the partition can be chosen to be smaller than a constant times
$1/\varepsilon^{2}$. Then

$\displaystyle\mathbb{E}\sup_{u_{k-1}\leq
s,t<u_{k}}|Z_{Ni}(s,j)-Z_{Ni}(t,j)|^{2}$
$\displaystyle<2N\alpha^{2}_{i}C_{X}^{2}\mathbb{E}\left\\{H_{\rho_{i}}(u_{k},V_{i})-H_{\rho_{i}}(u_{k-1},V_{i})\right\\}^{2}$
$\displaystyle+2N\alpha^{2}_{i}C_{X}^{2}\mathbb{E}\sup_{u_{k-1}\leq
s,t<u_{k}}|s-t|^{2}$
$\displaystyle<2N\alpha^{2}_{i}C_{X}^{2}\mathbb{E}\left\\{H_{\rho_{i}}(u_{k},V_{i})-H_{\rho_{i}}(u_{k-1},V_{i})\right\\}$
$\displaystyle+2N\alpha^{2}_{i}C_{X}^{2}(u_{k}-u_{k-1})$
$\displaystyle<\frac{N\alpha_{i}^{2}\varepsilon^{2}}{2c^{2}}.$

We deduce that for all sufficiently large $N$,

$\sum_{i=N}^{\infty}\mathbb{E}\sup_{u_{j-1}\leq
s,t<u_{j}}|Z_{Ni}(s)-Z_{Ni}(t)|^{2}<\varepsilon^{2}.$

In other words,
$\mathfrak{N}_{[]}(\varepsilon,\mathcal{F},L_{2,n})\lesssim\frac{1}{\varepsilon^{2}}$,
which verifies the entropy condition.

Thus, Theorem A3 implies that

$-\sum_{i=N}^{\infty}Z_{Ni}(u,j)=\sqrt{N}\sum_{i=N}^{\infty}\alpha_{i}(u-H_{\rho_{i}}(u,V_{i}))X_{ij}$

is asymptotically tight in $\ell^{\infty}(\mathcal{F})$. ∎

######  Proposition A5.

If $(u_{1},\ldots,u_{d})\in(0,1)^{d}$ and
$(j_{1},\ldots,j_{d})\in\\{1,\ldots,p\\}^{d}$, then

$\sqrt{N}[S_{N}(u_{1},j_{1}),\ldots,S_{N}(u_{d},j_{d})]^{T}\mid
w_{1:n}\xrightarrow[]{d}\mathcal{N}(0,a^{2}\Sigma)$

as $N\rightarrow\infty$, where
$\Sigma_{l,m}=\left[\sum_{k=1}^{n}w_{k}X_{kj_{l}}X_{kj_{m}}\right](\min\\{u_{l},u_{m}\\}-u_{l}u_{m})$.

###### Proof.

Fix an arbitrary vector $\mathbf{t}=(t_{1},\ldots,t_{d})\in\mathbb{R}^{d}$ and
we study the convergence of

$\displaystyle\sqrt{N}\sum_{l=1}^{d}\,t_{l}\,S_{N}(u_{l},j_{l}).$

Consider

$\displaystyle
Z_{Ni}[\mathbf{t}]=\sqrt{N}\,\alpha_{i}\sum_{l=1}^{d}\,t_{l}\,(H_{\rho_{i}}(u_{l},V_{i})-u_{l})X_{ij}$

which has expectation $0$. We have that

$\displaystyle|Z_{Ni}[\mathbf{t}]|\leq
2aC_{X}\frac{\sqrt{N}}{i}\sum_{l=1}^{d}|t_{j}|,$

so for every $\varepsilon>0$, $|Z_{Ni}[\mathbf{t}]|<\varepsilon$ eventually,
which verifies the Lindeberg condition for the Lindeberg-Feller CLT. For the
limiting variance, we first have

$\displaystyle\mathbb{E}[Z_{Ni}[\mathbf{t}]^{2}]$
$\displaystyle=a^{2}\frac{N}{i^{2}}\sum_{l=1}^{d}\sum_{m=1}^{d}t_{l}t_{m}\left[\sum_{k=1}^{n}w_{k}X_{kj_{l}}X_{kj_{m}}\right]$
$\displaystyle\quad\left(\mathbb{E}\left[H_{\rho_{i}}(u_{j},V_{i})\,H_{\rho_{i}}(u_{k},V_{i})\right]-u_{l}u_{m}\right)$

If we take $N/m_{N}=o(1)$, then

$\displaystyle\sum_{i=N}^{N+m_{N}}\mathbb{E}[Z_{Ni}[\mathbf{t}^{2}]\to\sigma^{2}=a^{2}\sum_{l=1}^{d}\sum_{m=1}^{d}t_{l}t_{m}\left[\sum_{k=1}^{n}w_{k}X_{kj_{l}}X_{kj_{m}}\right]\left(\min\\{u_{l},u_{m}\\}-u_{l}u_{m}\right)$

via similar computations to the non-regression case. So we can apply the
Lindeberg-Feller CLT to the sequence of sums up to $m_{N}$. We check that the
tail sums from $m_{N}+1$ onwards are asymptotically negligible:

$\displaystyle\sum_{i=m_{N}+1}^{\infty}\text{Var}[Z_{Ni}[\mathbf{t}]]$
$\displaystyle=N\sum_{i=m_{N}+1}^{\infty}\alpha_{i}^{2}\sum_{l=1}^{d}\sum_{m=1}^{d}t_{l}t_{m}\left[\sum_{k=1}^{n}w_{k}X_{kj_{l}}X_{kj_{m}}\right]\left(C_{\rho_{i}^{2}}(u_{l},u_{m})-u_{l}u_{m}\right)$
$\displaystyle\leq\left|\sum_{l=1}^{d}\sum_{m=1}^{d}t_{l}t_{m}C_{X}^{2}\right|N\sum_{i=m_{N}+1}^{\infty}\alpha_{i}^{2}\to
0.$

Finally, the Cramér-Wold device gives us

$\displaystyle\sqrt{N}[S_{N}(u_{1},j_{1}),\ldots,S_{N}(u_{d},j_{d})]^{T}\overset{d}{\to}\mathcal{N}(\mathbf{0},a^{2}\Sigma)$

where
$\Sigma_{l,m}=\left[\sum_{k=1}^{n}w_{k}X_{kj_{l}}X_{kj_{m}}\right](\min\\{u_{l},u_{m}\\}-u_{l}u_{m})$.
∎

###  B.16 Proposition 8

We require the following assumption on the covariance matrix of the
covariates.

######  Assumption A3 (Covariance matrix of covariates).

The covariance matrix of the covariate distribution
$\Sigma_{x}=\int_{\mathcal{X}}xx^{T}\,dP^{*}(x)$ is positive definite, and all
elements are finite.

We follow a similar approach to Proposition 5. Let
$Y_{i},X_{i}\overset{\mathrm{iid}}{\sim}P^{*}(y,x)$, and $\beta_{n}$ is
computed by Algorithm 6. Following the same argument as in Theorem 4 for each
component of $\beta_{n}$, one can see that $\beta_{n}\in H^{1}((0,1))^{p}$ for
each $n$, thus satisfying Assumptions A1 and A2. This allows us to apply
Theorem 8 giving the existence of $\beta_{n\infty}$. To construct the
probability space, we can again consider a single sequence
$V_{1},V_{2},\ldots$ of uniform r.v.s. For each $n$, consider the Bayesian
bootstrap starting with $X_{1:n}$, i.e. we have

$\displaystyle w_{1:n}$ $\displaystyle\sim\text{Dir}(1,\ldots,1),\quad
X_{nN}\mid
w_{1:n},X_{1:n}\overset{\mathrm{iid}}{\sim}\sum_{i=1}^{n}w_{i}\delta_{X_{i}}$

for $N\geq n+1$, where the additional subscript $n$ on $X$ indicates how many
‘real’ observations we start predictive resampling from. We can then define
for each $n\geq 1$

$\displaystyle\beta_{n\infty}(u)=\beta_{n}(u)+\sum_{i=n+1}^{\infty}\alpha_{i}\left(u-H_{\rho_{i}}(u,V_{i})\right)\,X_{ni}.$

We now consider the posterior distribution of the mean functional
$\bar{\beta}_{n\infty}$ where

$\displaystyle\bar{\beta}_{n\infty},=\int_{0}^{1}\beta_{n\infty}(u)\,du.$

Fubini’s theorem gives

$\displaystyle\mathbb{E}[\bar{\beta}_{n\infty}\mid
Y_{1:n},X_{1:n}]=\bar{\beta}_{n}.$

The posterior covariance matrix is thus

$\displaystyle\mathbb{E}[b_{n}b_{n}^{T}\mid
Y_{1:n},X_{1:n}]=\mathbb{E}\left[\left(\sum_{i=n+1}^{\infty}\alpha_{i}Z_{i}\,X_{ni}\right)\left(\sum_{i=n+1}^{\infty}\alpha_{i}Z_{i}\,X_{ni}\right)^{T}\Big{|}Y_{1:n},X_{1:n}\right]$

where we write $b_{n}=\bar{\beta}_{n\infty}-\bar{\beta}_{n}$ and
$Z_{i}=\int_{0}^{1}\left(u-H_{\rho_{i}}(u,V_{i})\right)\,du$ for shorthand.
Let us first condition on $w_{1:n}$, which gives

$\displaystyle\mathbb{E}[b_{n}b_{n}^{T}\mid
w_{1:n},Y_{1:n},X_{1:n}]=\sum_{i=n+1}^{\infty}\sum_{j=n+1}^{\infty}\alpha_{i}\alpha_{j}\mathbb{E}\left[Z_{i}Z_{j}\,X_{ni}X_{nj}^{T}\mid
w_{1:n},Y_{1:n},X_{1:n}\right].$

As $Z_{i}$ and $Z_{j}$ are independent for $i\neq j$, and the covariates are
independent from the uniform r.v.s, the cross-terms are all 0, so the above
simplifies to

$\displaystyle\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\,\mathbb{E}[Z_{i}^{2}]\,\mathbb{E}[X_{ni}X_{ni}^{T}\mid
w_{1:n},Y_{1:n},X_{1:n}].$

As before, we have

$\displaystyle\mathbb{E}[Z_{i}^{2}]=\int_{0}^{1}\int_{0}^{1}\,C_{\rho_{i}^{2}}(u,v)\,du\,dv-\frac{1}{4}$

and now we have the additional term

$\displaystyle\mathbb{E}[X_{ni}X_{ni}^{T}\mid
w_{1:n},X_{1:n}]=\sum_{i=1}^{n}w_{i}X_{i}X_{i}^{T}.$

This gives

$\displaystyle\mathbb{E}[b_{n}b_{n}^{T}\mid
w_{1:n},Y_{1:n},X_{1:n}]=\left[\sum_{i=1}^{n}w_{i}X_{i}X_{i}^{T}\right]\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\,\mathbb{E}[Z_{i}^{2}].$

The tower property then gives

$\displaystyle\mathbb{E}[b_{n}b_{n}^{T}\mid
Y_{1:n},X_{1:n}]=\left[\frac{1}{n}\sum_{i=1}^{n}X_{i}X_{i}^{T}\right]\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\,\mathbb{E}[Z_{i}^{2}].$

We can then scale this by ${n}$ and take the limit, giving us

$\displaystyle\lim_{n\to\infty}{n}\,\mathbb{E}[b_{n}b_{n}^{T}\mid
Y_{1:n},X_{1:n}]$
$\displaystyle=\lim_{n\to\infty}\left[\frac{1}{n}\sum_{i=1}^{n}X_{i}X_{i}^{T}\right]\lim_{n\to\infty}\left[n\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\,\mathbb{E}[Z_{i}^{2}]\right]$
$\displaystyle=\frac{a^{2}}{12}\Sigma_{x}\quad P^{*}\text{-a.s.}$

##  Appendix C Additional results

###  C.1 Asymptotic distribution of mean functional

For the mean functional, we only computed its posterior mean and asymptotic
variance in Proposition 5. We can actually extend this in the unconditional
case and quantify the asymptotic distribution of $\mu_{n\infty}-\mu_{n}$ due
to it being a sum of independent terms. This could potentially lead the way to
future Bernstein-von Mises for functionals of the QMP. However, as mentioned
in the main paper, quantifying the distribution of $\mu_{n}$ is more
challenging.

######  Proposition A6.

Let $\mu_{n}=\int Q^{\dagger}_{n}(u)\,du$ for $\\{Q^{\dagger}_{n}\\}_{n\geq
1}$ from Algorithm 3, and
$\mu_{n\infty}=\int_{0}^{1}Q_{n\infty}^{\dagger}(u)\,du$ where
$Q_{n\infty}^{\dagger}$ arises from Algorithm 4 starting from
$Q^{\dagger}_{n}$. Under Assumptions 3 and 4, we have

$\displaystyle\sqrt{n}\left(\mu_{n\infty}-\mu_{n}\right)\overset{d}{\to}\mathcal{N}(0,a^{2}/12)$

###### Proof.

We will extend the proof of Proposition 5 by applying Theorem 2, although we
highlight that one can also prove the above using the standard Lindeberg-
Feller CLT for scalar r.v.s. In order to apply Theorem 2, we will leverage the
specific construction for $Q_{n\infty}^{\dagger}$ as in the proof of Theorem
4. Let us define

$\displaystyle
S_{N}(u)=\sum_{i=N+1}^{\infty}\alpha_{i}\left[u-H_{\rho_{i}}\left(u,V_{i}\right)\right]$

for $n\geq 1$, where $V_{i}\overset{\mathrm{iid}}{\sim}\mathcal{U}(0,1)$. We
then define

$\displaystyle Q_{n\infty}=Q_{n}^{\dagger}+S_{n},$

for $n\geq 1$. For each $n$, $Q_{n\infty}$ has the same distribution as that
induced by Algorithm 4, but is not independent across $n$. However, this is
inconsequential as we are studying a weak limit. This gives a corresponding
sequence of random means:

$\displaystyle\mu_{n\infty}=\int_{0}^{1}Q_{n\infty}(u)\,du.$

We then clearly have

$\displaystyle\sqrt{n}(\mu_{n\infty}-\mu_{n})$
$\displaystyle=\int_{0}^{1}\sqrt{n}\,S_{n}(u)\,du.$

Since the weak limit of $\sqrt{N}S_{N}$ is independent of the initial estimate
$Q_{n}^{\dagger}$, we can apply Theorem 2 directly. This gives
$\sqrt{n}S_{n}\to\mathbb{G}_{a}$ weakly in $\ell^{\infty}((0,1))$, where
$\mathbb{G}_{a}$ is a zero-mean GP with covariance function
$\mathbb{E}\left[\mathbb{G}_{a}(u)\,\mathbb{G}_{a}(u^{\prime})\right]=a^{2}(\min\\{u,u^{\prime}\\}-uu^{\prime})$.

Consider now the integral operator $h:\ell^{\infty}((0,1))\to\mathbb{R}$ where
$h(f)=\int_{0}^{1}\,f(u)\,du$. It is not too hard to see that this is a
continuous function, as for any sequence $f_{n}\to f_{\infty}$ for
$f_{n},f_{\infty}\in\ell^{\infty}((0,1))$, we have

$\displaystyle|h(f_{n})-h(f_{\infty})|\leq\int_{0}^{1}|f(u)-f_{\infty}(u)|\,du\leq\|f_{n}-f_{\infty}\|_{\infty}\to
0.$

The continuous mapping theorem then gives us

$\displaystyle\sqrt{n}(\mu_{n\infty}-\mu_{n})\overset{d}{\to}h(\mathbb{G}_{a}).$

We now show that $h(\mathbb{G}_{a})\sim\mathcal{N}(0,a^{2}/12)$. It is clear
that $h(\mathbb{G}_{a})$ has mean zero, and its variance is

$\displaystyle\mathbb{E}\left[h(\mathbb{G}_{a})^{2}\right]$
$\displaystyle=\mathbb{E}\left[\int_{0}^{1}\int_{0}^{1}\mathbb{G}_{a}(u)\,\mathbb{G}_{a}(u^{\prime})\,du\,du^{\prime}\right]$
$\displaystyle=\int_{0}^{1}\int_{0}^{1}\mathbb{E}\left[\mathbb{G}_{a}(u)\,\mathbb{G}_{a}(u^{\prime})\right]\,du\,du^{\prime}$
$\displaystyle=a^{2}\int_{0}^{1}\int_{0}^{1}\left[\min\\{u,u^{\prime}\\}-uu^{\prime}\right]du\,du^{\prime}$
$\displaystyle=\frac{a^{2}}{12}\cdot$

The normality of $h(\mathbb{G}_{a})$ then follows from an approximating
Riemann sum argument (e.g. [Hassler, 2016, Chapter 8.3]) as sample paths of
the Brownian motion are continuous a.s. ∎

###  C.2 Frequentist consistency for quantile regression

In this section, we outline a posterior consistency result for the QMP for
quantile regression with $\rho=1$. This setting lends itself more easily to a
consistent initial estimate. Consider the QMP with the updates

$\displaystyle\beta_{N+1}(u)$
$\displaystyle=\beta_{N}(u)+\alpha_{N+1}\left[u-G_{N+1}(\beta_{N},Y_{N+1},X_{N+1})\right]\,X_{N+1}$
(A16) $\displaystyle G_{N}(\beta,Y,X)$
$\displaystyle=\begin{cases}\mathbbm{1}\left(Y\leq\beta(u)^{T}X\right)\quad&\text{for
}N\leq n\\\ \mathbbm{1}\left(Y\leq Q^{\dagger}(u\mid X)\right)\quad&\text{for
}N\geq n+1\end{cases}$ (A17)

where $Q^{\dagger}(u\mid X)$ is the increasing rearrangement of
$\beta(u)^{T}X$. The difference between the update for the initial estimate
and predictive resampling is subtle but important for both consistency and the
martingale. To derive the latter form, note that

$\displaystyle\lim_{\rho\to 1}H_{\rho}(u,V_{N+1})=\mathbbm{1}\left(V_{N+1}\leq
u\right)=\mathbbm{1}\left(Y_{N+1}\leq Q_{N}^{\dagger}(u\mid X_{N+1})\right)$

which is obtained by applying the proper quantile function
$Q_{N}^{\dagger}(\cdot\mid X_{N+1})$ to both sides of the inequality. The
martingale under predictive resampling is thus preserved in this case. For the
‘real data’ update however, we opt to use the standard stochastic
approximation estimate of $\beta_{n}$, as it is non-trivial to derive a
copula-smoothed version of the above initial estimate (i.e. an equivalent
version of (9) for $\beta_{n}^{\dagger}(u)$). Consider now the following
assumptions on the data generating distribution.

######  Assumption A4 (Covariate distribution).

$P^{*}(x)$ has compact support and the covariance matrix
$\Sigma_{x}=\int_{\mathcal{X}}xx^{T}\,dP^{*}(x)$ is positive definite.

######  Assumption A5 (Linear quantiles).

There exists some true function $\beta^{*}(u)$ such that the quantile function
corresponding to $P^{*}(\cdot\mid x)$ takes the form $Q^{*}(u\mid
x)=\beta^{*}(u)^{T}x$ for all $x$ in the support of $P^{*}(x)$.

######  Assumption A6 (Lipschitz continuity).

There exists a finite $L$ such that

$\displaystyle\sup_{j\in\\{1,\ldots,p\\}}|\beta^{*}_{j}(u)-\beta^{*}_{j}(u^{\prime})|\leq
L|u-u^{\prime}|$

where $\beta^{*}_{j}(u)$ is the $j$-th component of the vector $\beta^{*}(u)$.
Assume that the initial vector function $\beta_{0}$ also satisfies the above
Lipschitz condition.

We now define the norms to study the conditional quantile function

$\displaystyle{d}^{2}_{2,x}(Q^{*}(\cdot\mid x),Q(\cdot\mid
x))=\int_{\mathcal{X}}d^{2}_{2}(Q^{*}(\cdot\mid x),Q(\cdot\mid x))\,dP^{*}(x)$

which is the covariate average $L^{2}$ distance between the conditional
quantiles, and also

$\displaystyle{d}^{2}_{2,p}(\beta^{*},\beta)=\int_{0}^{1}\left(\beta^{*}(u)-\beta(u)\right)^{T}\left(\beta^{*}(u)-\beta(u)\right)\,du.$

We have a standard result from stochastic approximation arguments:

######  Proposition A7.

Under Assumptions A4, A5 and A6, we have that
${d}_{2,p}(\beta^{*},\beta_{n})\to 0$ a.s.$[P^{*}]$ as $n\to\infty$ under
(A16) and (A17) with the $N\leq n$ form of $G_{N}$.

###### Proof.

The $L^{2}$ distance can be expanded recursively:

$\displaystyle d^{2}_{2,p}\left(\beta^{*},\beta_{n+1}\right)$
$\displaystyle=d^{2}_{2,p}\left(\beta^{*},\beta_{n}\right)-2\alpha_{n+1}\int_{0}^{1}\left(\beta^{*}(u)-\beta_{n}(u)\right)^{T}X_{n+1}\left(u-\mathbbm{1}\left(Y_{n+1}\leq\beta_{n}(u)^{T}X_{n+1}\right)\,\right)\,du$
$\displaystyle+\alpha_{n+1}^{2}\int_{0}^{1}\left(u-\mathbbm{1}\left(Y_{n+1}\leq\beta_{n}(u)^{T}X_{n+1}\right)\right)du\,X_{n+1}^{T}X_{n+1}\
$

Taking the conditional expectation of the above given
$\mathcal{F}_{n}=\sigma(Z_{1},\ldots,Z_{n})$ for $Z_{i}=(Y_{i},X_{i})$ gives

$\displaystyle\mathbb{E}\left[d^{2}_{2,p}\left(\beta^{*},\beta_{n+1}\right)\mid\mathcal{F}_{n}\right]$
$\displaystyle\leq
d^{2}_{2,p}\left(\beta^{*},\beta_{n}\right)-2\alpha_{n+1}g(\beta^{*},\beta_{n})+\alpha_{n+1}^{2}V^{2}$

where $V^{2}=\mathbb{E}[X^{T}X]$ for $X\sim P^{*}(x)$ which is finite by
Assumption A4, and

$\displaystyle g(\beta^{*},\beta)$
$\displaystyle=\int_{\mathcal{X}}\,\int_{0}^{1}\left(\beta^{*}(u)-\beta(u)\right)^{T}x\left(u-P^{*}\left(\beta(u)^{T}x\mid
x\right)\right)\,du\,dP^{*}(x)$
$\displaystyle=\int_{\mathcal{X}}\int_{0}^{1}\left(Q^{*}(u\mid x)-Q(u\mid
x)\right)\left(u-P^{*}\left(Q(u\mid x)\mid x\right)\right)\,du\,dP^{*}(x).$

For each value of $(u,x)$, we have that

$\displaystyle\left(Q^{*}(u\mid x)-Q(u\mid x)\right)\left(u-P^{*}\left(Q(u\mid
x)\mid x\right)\right)\geq 0.$

which follows as $P^{*}(\cdot\mid x)$ is monotonic, so we have

$\displaystyle\left(Q^{*}(u\mid x)-Q(u\mid x)\right)\geq
0\implies\left(u-P^{*}\left(Q(u\mid x)\mid x\right)\right)\geq 0.$

We thus have $g(\beta^{*},\beta)\geq 0$ and
$V^{2}\sum_{n=1}^{\infty}{\alpha_{n+1}^{2}}<\infty$ which gives us the almost
supermartingale from Theorem A4. We thus have

$\displaystyle d^{2}_{2,p}\left(\beta^{*},\beta_{n}\right)\to
d_{\infty}\quad\text{a.s.},\quad\sum_{n=1}^{\infty}\alpha_{n}g\left(\beta^{*},\beta_{n}\right)<\infty\quad\text{a.s
}$

We now seek to show $d_{\infty}=0$ a.s. Let $C_{X}$ denote the magnitude of
the maximum value of $\mathcal{X}$ in all dimensions which is finite by
Assumption A4. Note that we have

$\displaystyle\left|Q^{*}(u\mid x)-Q^{*}(u^{\prime}\mid x)\right|$
$\displaystyle=\left|\left(\beta^{*}(u)-\beta^{*}(u^{\prime})\right)^{T}x\right|$
$\displaystyle\leq
C_{X}\sum_{j=1}^{p}\left|\beta^{*}_{j}(u)-\beta^{*}_{j}(u^{\prime})\right|$
$\displaystyle\leq C_{X}\,p\,L\,|u-u^{\prime}|$

where $L$ is the Lipschitz constant from Assumption A6. We thus have
$M=C_{X}pL<\infty$ such that

$\displaystyle|Q^{*}(u\mid x)-Q(u\mid x)|\leq M\left|u-P^{*}\left(Q(u\mid
x)\mid x\right)\right|$

where we have plugged in $u^{\prime}=P^{*}\left(Q(u\mid x)\mid x\right)$, and
$M$ is chosen uniformly over $x$. Then this gives

$\displaystyle g(\beta^{*},\beta)\geq
M^{-1}\int_{\mathcal{X}}\int_{0}^{1}\left(Q^{*}(u\mid x)-Q(u\mid
x)\right)^{2}\,\,du\,dP^{*}(x)$

With the above, we have

$\displaystyle g(\beta^{*},\beta)$ $\displaystyle\geq
M^{-1}\int_{\mathcal{X}}\int_{0}^{1}\left(\beta^{*}(u)-\beta(u)\right)^{T}xx^{T}\left(\beta^{*}(u)-\beta(u)\right)\,du\,dP^{*}(x)$
$\displaystyle=M^{-1}\int_{0}^{1}\left(\beta^{*}(u)-\beta(u)\right)^{T}\Sigma_{x}\left(\beta^{*}(u)-\beta(u)\right)\,du$

where $\Sigma=\int xx^{T}dP^{*}(x)$ and we have used Tonelli’s theorem. As
$\Sigma_{x}$ is positive definite by Assumption A4, we have that

$\displaystyle\frac{x^{T}\Sigma_{x}x}{x^{T}x}\geq\lambda_{\text{min}}>0$

where $\lambda_{\text{min}}$ is the minimum eigenvalue of $\Sigma_{x}$.
Therefore, we have that there exists
$\varepsilon=\lambda_{\text{min}}M^{-1}>0$ such that

$\displaystyle
g(\beta^{*},\beta_{n})\geq\varepsilon\,d^{2}_{2,p}\left(\beta^{*},\beta_{n}\right).$

As $\sum_{n=1}^{\infty}\alpha_{n}g(\beta^{*},\beta_{n})<\infty$ a.s., this
then ensures that $d_{\infty}=0$ a.s. by the usual argument. ∎

We can show that consistency of $\beta_{n}$ implies consistency of the
conditional quantiles.

######  Corollary A2.

Under Assumptions A4, A5 and A6, we have that ${d}_{2,x}(Q^{*}(\cdot\mid
x),Q_{n}(\cdot\mid x))\to 0$ a.s.$[P^{*}]$ as $n\to\infty$ under (A16) and
(A17) with the $N\leq n$ form of $G_{N}$.

###### Proof.

First, we write

$\displaystyle{d}^{2}_{2,x}\left(Q^{*}(\cdot\mid x),Q_{n}(\cdot\mid x)\right)$
$\displaystyle=\int_{\mathcal{X}}\int_{0}^{1}\left(Q^{*}(u\mid x)-Q_{n}(u\mid
x)\right)^{2}\,du\,dP^{*}(x)$
$\displaystyle=\int_{\mathcal{X}}\int_{0}^{1}\left(\beta^{*}(u)-\beta_{n}(u)\right)^{T}xx^{T}\left(\beta^{*}(u)-\beta_{n}(u)\right)\,du\,dP^{*}(x)$
$\displaystyle=\int_{0}^{1}\left(\beta^{*}(u)-\beta_{n}(u)\right)^{T}\Sigma_{x}\left(\beta^{*}(u)-\beta_{n}(u)\right)\,du$

which looks familiar. Using the other side of the inequality for Rayleigh’s
quotient, we have that

$\displaystyle\int_{0}^{1}\left(\beta^{*}(u)-\beta_{n}(u)\right)^{T}\Sigma_{x}\left(\beta^{*}(u)-\beta_{n}(u)\right)\,du\leq\lambda_{\text{max}}\int_{0}^{1}\left(\beta^{*}(u)-\beta_{n}(u)\right)^{T}\left(\beta^{*}(u)-\beta_{n}(u)\right)\,du$

where $\lambda_{\text{max}}$ is the maximum eigenvalue of the covariance
matrix, which is bounded due to compact support from Assumption A4. We then
have the desired result as ${d}^{2}_{2,x}\left(Q^{*}(\cdot\mid
x),Q_{n}(\cdot\mid
x)\right)\leq\lambda_{\text{max}}\,d_{2,p}^{2}(\beta^{*},\beta_{n})\to 0$ a.s.
∎

Posterior consistency can then be showed as follows, where we work directly
with $Q^{\dagger}(u\mid x)$ instead of $\beta_{\infty}(u)$ due to need to take
into account the rearrangement operator. The setup is the same as Theorem 4,
where we extend the probability space as before, with the additional
ingredients of a vector of weights $w_{1:n}$ and random covariates
$\\{X_{n,n+1},X_{n,n+1},\ldots\\}$ for each $n$ arising from the Bayesian
bootstrap.

######  Theorem A7.

Under Assumptions A4, A5 and A6, the QMP with $\rho=1$ as in (A16) and (A17)
is consistent, that is for any $\varepsilon>0$, we have that

$\displaystyle\Pi\left({d}^{2}_{2,x}(Q^{*}(\cdot\mid
x),Q_{n\infty}^{\dagger}(\cdot\mid x))\geq\varepsilon\mid
Y_{1:n},X_{1:n}\right)\to 0\quad\textnormal{a.s.}[P^{*}]$

###### Proof.

Once again, Markov’s inequality gives us

$\displaystyle\Pi\left({d}^{2}_{2,x}(Q^{*}(\cdot\mid
x),Q_{n\infty}^{\dagger}(\cdot\mid x))\geq\varepsilon\mid
Y_{1:n},X_{1:n}\right)\leq\frac{1}{\varepsilon^{2}}\mathbb{E}\left[{d}^{2}_{2,x}(Q^{*}(\cdot\mid
x),Q_{n\infty}^{\dagger}(\cdot\mid x))\mid Y_{1:n},X_{1:n}\right].$

Expanding out the triangle inequality, we have

$\displaystyle E\left[{d}^{2}_{2,x}(Q^{*}(\cdot\mid
x),Q_{n\infty}^{\dagger}(\cdot\mid x))\mid Y_{1:n},X_{1:n}\right]$ (A18)
$\displaystyle\leq\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}^{\dagger}(\cdot\mid
x),Q^{\dagger}_{n}(\cdot\mid x))\mid Y_{1:n},X_{1:n}\right]$
$\displaystyle+2E\left[{d}_{2,x}(Q_{n\infty}^{\dagger}(\cdot\mid
x),Q^{\dagger}_{n}(\cdot\mid x))\mid
Y_{1:n},X_{1:n}\right]{d}_{2,x}(Q^{*}(\cdot\mid x),Q^{\dagger}_{n}(\cdot\mid
x))$ $\displaystyle+{d}^{2}_{2,x}(Q^{*}(\cdot\mid x),Q^{\dagger}_{n}(\cdot\mid
x)).$

For each $x\in\mathcal{X}$, $Q^{*}(\cdot\mid x)$ and $Q_{n}(\cdot\mid x)$ has
compact range from Assumption A6. We can thus apply Proposition 2 to show
${d}^{2}_{2}(Q^{*}(\cdot\mid x),Q^{\dagger}(\cdot\mid
x))\leq{d}^{2}_{2}(Q^{*}(\cdot\mid x),Q(\cdot\mid x))$ for each
$x\in\mathcal{X}$, which gives

$\displaystyle{d}^{2}_{2,x}(Q^{*}(\cdot\mid x),Q^{\dagger}(\cdot\mid
x))\leq{d}^{2}_{2,x}(Q^{*}(\cdot\mid x),Q(\cdot\mid x)).$

For the final term then, we have

$\displaystyle{d}^{2}_{2,x}(Q^{*}(\cdot\mid x),Q^{\dagger}_{n}(\cdot\mid
x))\leq{d}^{2}_{2,x}(Q^{*}(\cdot\mid x),Q_{n}(\cdot\mid x))\to
0\quad\textnormal{a.s.}[P^{*}]$

For the first term , we also apply the rearrangement inequality to get

$\displaystyle\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}^{\dagger}(\cdot\mid
x),Q^{\dagger}_{n}(\cdot\mid x))\mid
Y_{1:n},X_{1:n}\right]\leq\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}(\cdot\mid
x),Q_{n}(\cdot\mid x))\mid Y_{1:n},X_{1:n}\right]$

which we now bound. First, we look at the inner term

$\displaystyle{d}^{2}_{2,x}(Q_{n\infty}(\cdot\mid x),Q_{n}(\cdot\mid
x))=\int_{\mathcal{X}}\int_{0}^{1}\left[\left(\beta_{n\infty}(u)-\beta_{n}(u)\right)^{T}x\right]^{2}\,du\,dP^{*}(x)$
$\displaystyle=\sum_{i=n+1}^{\infty}\sum_{j=n+1}^{\infty}\alpha_{i}\,\alpha_{j}\int_{0}^{1}\left(u-\mathbbm{1}\left(V_{i}\leq
u\right)\right)\,\left(u-\mathbbm{1}\left(V_{j}\leq
u\right)\right)\,du\,X_{i}^{T}\int_{\mathcal{X}}xx^{T}dP^{*}(x)\,X_{j}$

where we have applied Tonelli’s theorem. Taking the expectation conditional on
$\\{w_{1:n},Y_{1:n},X_{1:n}\\}$ gives

$\displaystyle\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}(\cdot\mid
x),Q_{n}(\cdot\mid x))\mid w_{1:n},Y_{1:n},X_{1:n}\right]$
$\displaystyle=\sum_{i=n+1}^{\infty}\alpha_{i}^{2}\int_{0}^{1}(u-\mathbbm{1}\left(V_{i}\leq
u\right))^{2}\,du\,E[X_{i}^{T}\Sigma_{x}X_{i}\mid w_{1:n}]\,$

where the cross-terms disappear as $V_{i}$ is independent of $V_{j}$ (and both
are independent of $X_{i},X_{j}$) for $i\neq j$, and the terms have mean 0. We
can upper bound the above term by

$\displaystyle E\left[X_{n+1}^{T}\Sigma_{x}X_{n+1}\mid
w_{1:n}\right]\sum_{i=n+1}^{\infty}\alpha_{i}^{2}=O(n^{-1})\left[\sum_{i=1}^{n}w_{i}X_{i}^{T}\Sigma_{x}X_{i}\right]$

where we have used the fact that $X_{n+1:\infty}$ are i.i.d. conditional on
$w_{1:n}$. Taking the expectation over the weights then gives

$\displaystyle\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}(\cdot\mid
x),Q_{n}(\cdot\mid x))\mid Y_{1:n},X_{1:n}\right]\leq
O(n^{-1})\left[\frac{1}{n}\sum_{i=1}^{n}X_{i}^{T}\Sigma_{x}X_{i}\right]$

As $\Sigma_{x}$ has finite eigenvalues from Assumption A4, we have that

$\displaystyle\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}(\cdot\mid
x),Q_{n}(\cdot\mid x))\mid Y_{1:n},X_{1:\infty}\right]\leq
O(n^{-1})\left(\frac{1}{n}\sum_{i=1}^{n}X_{i}^{T}X_{i}\right)$

We then have $\frac{1}{n}\sum_{i=1}^{n}X_{i}^{T}X_{i}\to V^{2}<\infty$ a.s.,
which gives

$\displaystyle\mathbb{E}\left[{d}^{2}_{2,x}(Q_{n\infty}(\cdot\mid
x),Q_{n}(\cdot\mid x))\mid Y_{1:n},X_{1:\infty}\right]\leq
O(n^{-1})\quad\text{a.s.}[P^{*}]$

A similar argument as in the proof of Theorem 4 can be used to handle the
second cross-term in (LABEL:app:eq_triangle), so we have the result. ∎

We thus have posterior consistency of the QMP for linear regression for
$\rho=1$. A similar result can likely be derived for the posterior contraction
rate. Like in the quantile estimation case, we suspect that the QMP with the
smoothed update (11) satisfies a similar result on posterior consistency and
contraction, but it is not immediately obvious due to the non-linearity of the
increasing rearrangement operator. In practice, we see that the rearrangement
is negligible for the initial estimate for a sequence $\rho_{N}$ which
approaches $1$ sufficiently slowly. We leave a thorough investigation of this
for future work.

##  Appendix D Practical details