Paper Title: Varextropy of doubly truncated random variable.



Varextropy  of doubly truncated random variable] Varextropy  of doubly truncated random variable 



 [
    Simon Valentin^1[1]
        Jinmiao Fu^2[1]
        Gianluca Detommaso^3These authors contributed equally to this research. 
        Shaoyuan Xu^2
        Giovanni Zappella^1
        Bryan Wang^2
    
    ^1Amazon Web Services, Berlin, Germany 

    

    ^2Amazon, Seattle, USA 

    

    ^2Helsing, Berlin, Germany 

    


=======================================================================================================================================================================================================












§ ABSTRACT 
Recently, there has been growing attention to study uncertainty measures for doubly truncated random variables.
In this paper, the concept of varextropy for doubly truncated random variables is introduced. The changes of this measure under linear transforms are investigated.
Also, lower and upper bounds for the interval varextropy under some conditions are obtained and some other properties for this measure are proved.
In the following, three estimators for the interval varextropy are proposed. A simulation study has been carried out to investigate the behavior of estimators. At the end, an application for these estimators is proposed.




Keywords:
Doubly Truncated Data, Interval Varextropy, Kernel Density.



MSC2010: 94A17, 62B10.


§ INTRODUCTION AND PRELIMINARIES

Uncertainty or randomness is associated with any event or phenomenon. A basic measure of uncertainty for a random variable was given by <cit.>.
Let X be a non-negative random variable with the density function f, the Shannon entropy is defined as 

    H(x)=-∫_0^∞  f(x)log(f(x)) dx.

<cit.> introduced the varentropy function. This   measure  is the expectation of the squared deviation of the information content -log(f(x)) from entropy:

    VJ(X)=Var(-log(f(X))=∫_0^∞  f(x)(log(f(x)))^2 dx-[∫_0^∞  f(x) log(f(x))]^2 dx.

VJ(X) measures how much variability we may expect in the information  content of X.  

<cit.> defined the varentropy for the residual lifetime X_t=(X-t|X>t), called the residual varentropy at time t and defined as 

    VJ(X_t)=∫_t^∞f(x)/F(t)(log(f(x)/F(t)))^2 dx-[∫_t^∞f(x)/F(t)log(f(x)/F(t)) dx]^2,

where F(t)=1-F(t).

<cit.> studied the dual measure of residual varentropy for  the past lifetime X_(t)=(t-X|X ≤ t), called past varentropy and defined as follows:

    VJ(X_(t))=∫_0^tf(x)/F(t)(log(f(x)/F(t)))^2 dx-[∫_0^tf(x)/F(t)log(f(x)/F(t)) dx]^2.

The dual of entropy is called the extropy and is defined as:

    J(X)=-1/2∫_0^∞ (f(x))^2 dx.

A detailed motivation for J(X) is explained in <cit.>. Among other authors who have investigated the properties of the extropy and its applications, we can mention <cit.>, <cit.>, <cit.>, <cit.>, <cit.> and <cit.>.

<cit.> defined the concept of measuring uncertainty in residual lifetimes called the residual extropy:

    J(X_t)=-1/2(F(t))^2∫_t^∞ (f(x))^2 dx.

<cit.> and <cit.> introduced the notion of the past extropy as 

    J(X_(t))=-1/2(F(t))^2∫_0^t (f(x))^2 dx,

which measures the uncertainty in the time X_(t). An alternative measure to (<ref>) is introduced by <cit.>. This measure is known as varextropy and defined as 

    V(X)=Var(-1/2f(X))=1/4∫   (f(x))^3  dx-(1/2∫ (f(x))^2 dx)^2.

This measure indicates how the information content is scattered around the extropy. <cit.> proposed some estimators for the varextropy of a continuous random variable.

<cit.> proposed non-parametric estimators for varextropy under length-biased sampling. They investigated some asymptotic properties of the estimators such as a.s. consistency and asymptotic normality. <cit.> showed that, if extropy and varextropy series and parallel systems are equal then distribution function F(x) is a symmetric function.

For measuring the dispersion of the residual and past lifetimes, <cit.> introduced V(X_t) and V(X_[t]) as follows:

    V(X_t)=1/4(F(t))^3∫_t^∞  (f(x))^3  dx-(1/2(F(t))^2∫_t^∞ (f(x))^2 dx)^2,


    V(X_[t])=1/4(F(t))^3∫_0^t  (f(x))^3  dx-(1/2(F(t))^2∫_0^t (f(x))^2 dx)^2.

Doubly truncated data play an important role in the many fields such as reliability theory, economy and astronomy. Recently, uncertainty measures for doubly truncated random variable are studied.
Let X_t_1,t_2=(X|t_1<X<t_2) be the lifetime of an item falls in the interval (t_1,t_2). <cit.>  introduced the concept of varentropy for X_t_1,t_2. They  applied this concept to some known distributions. They also studied  the behavior of the interval varentropy in terms of varied transformations. The interval varentropy is introduced as 

    VJ(X_t_1,t_2)   =   ∫_t_1^t_2f(x)/F(t_2)-F(t_1)(log(f(x)/F(t_2)-F(t_1)))^2 dx
          -[∫_t_1^t_2f(x)/F(t_2)-F(t_1)log(f(x)/F(t_2)-F(t_1)) dx]^2.

(<ref>) takes the form of the residual varentropy (<ref>) when  t_2 →∞ and of the past varentropy  (<ref>) when t_1 → 0.

<cit.> proposed the dynamic version of extropy for doubly truncated random variables as: 

    IJ(t_1,t_2)=J(X_t_1,t_2)=-1/2(F(t_2)-F(t_1))^2∫_t_1^t_2 (f(x))^2 dx.

They evaluated some characterizations of J(X_t_1,t_2) and presented some bounds for it. They also evaluated this measure under the effect of linear transformations.

In analogy with the interval varentropy and interval extropy, we introduce the concept of interval varextropy for doubly truncated random variables. This measure indicates how the information content is scattered around the interval extropy for a doubly truncated random variable.
Similar to varextropy, interval varextropy is used to determine which interval extropy is more appropriate for measuring uncertainty when the interval extropies of two doubly truncated variables are equal.
On the other hand, the interval varextropy is free of the model parameters in some distributions and therefore is more flexible than the interval varentropy.

In the rest of the paper, we introduce the interval varextropy IV(t_1, t_2) for doubly truncated random variable.
In the following, several examples are shown to illustrate the interval extropy and the interval varextropy for some reliability distributions. Also, we investigate some properties
 of the IV(t_1,t_2) and  give some bounds for the interval varextropy. In Section <ref>, we propose some estimators for the interval varextropy and investigate their some asymptotic properties.
In  Section <ref>, we investigate the behavior of proposed estimators by a Monte Carlo study. We give an application for proposed estimators in Section <ref>. We illustrate the use  of proposed estimators in the real case  in Section <ref>.


§ INTERVAL VAREXTROPY MEASURE
  
Let X_t_1,t_2=(X|t_1<X<t_2) be the lifetime of a component in a system. That is, the lifetime of the component falls in the interval (t_1,t_2). 
In the present paper, an alternative measure analogous to the interval varentropy for X_t_1,t_2 is proposed where indicates how the information content is scattered around the doubly truncated extropy defined in (<ref>).

Let D={(u,v)∈ _+^2: F(u)<F(v)}, the interval varextropy for (t_1, t_2) ϵ D is defined as:

    IV(t_1,t_2)   =   var(-1/2f(X_t_1,t_2))=1/41/(F(t_2)-F(t_1))^3∫_ t_1^t_2 (f(x))^3 dx
          -1/41/(F(t_2)-F(t_1))^4(∫_ t_1^t_2 f^2(x) dx)^2,

which is an extension of varextropy.

It is easy to see that IV(0,t_2)=V(X_[t_2]), IV(t_1,∞)=V(X_t_1) and IV(0,∞)=V(X) are  the past varextropy, residual varextropy and varextropy, respectively.

In the following, a few examples are given to illustrate the interval extropy and the interval varextropy for significant distributions used in reliability and  survival analysis.

Let X be a random variable having exponential distribution, that is, X ∼ Exp(λ), λ>0. For 0<t_1<t_2, we have IJ(t_1,t_2)=-λ/4e^-λ t_1+e^-λ t_2/e^-λ t_1-e^-λ t_2, and   IV(t_1,t_2)=λ ^2/48(e^-λ t_1+e^-λ t_2)^2/(e^-λ t_1-e^-λ t_2)^2.
 

Suppose that  X have the Pareto-I distribution with the survival function F(x)=(a/x)^b;    0<a<x,  b>0.  We have, for a<t_1<t_2; 

    IJ(t_1,t_2)=b^2/2(2b+1)(t_1t_2)(t_1^2b+1-t_2^2b+1)/(t_2^b-t_1^b)^2,


    IV(t_1,t_2)=b^3/4(t_2^b-t_1^b)^3(t_1^2t_2^2)((t_2^3b+2-t_1^3b+2)/(3b+2)-b/(2b+1)^2(t_1^2b+1-t_2^2b+1)^2/(t_2^b-t_1^b)).

  
  
 Let X follow power distribution having F(x)=(x/a)^b,  0<x<a,  b>0. Then, for 0<t_1<t_2<a, we have 
 
    IJ(t_1,t_2)=-b^2/2(2b-1)(t_2^b-t_1^b)^2(t_2^2b-1-t_1^2b-1),


    IV(t_1,t_2)=b^3/4(t_2^b-t_1^b)^3[(t_2^3b-2-t_1^3b-2)/(3b-2)-b/(2b-1)^2(t_2^2b-1-t_1^2b-1)^2/(t_2^b-t_1^b)].

   
  
Suppose that  X  is a random variable with distribution function  F(x)=x^2,   x ∈ (0,1). Then, for 0<t_1<t_2<1, 
 
    IJ(t_1,t_2)=-2(t_2^2+t_1t_2+t_1^2)/3(t_2^2-t_1^2)(t_2+t_1),


    IV(t_1,t_2)=1/(t_2^2-t_1^2)^2[1/2(t_2^2+t_1^2)-4/9(t_2^2+t_1t_2+t_1^2)^2/(t_2+t_1)^2].

   

Suppose that  X  is a random variable with density and distribution function 

    F(x)=exp(-1/2-1/x)       if  0 ≤ x≤ 1 
    exp(-2+x^2/2)      if 1 ≤ x ≤ 2,


    f(x)=1/x^2exp(-1/2-1/x)        if x ∈ (0,1] 
     xexp(-2+x^2/2)      if  x ∈ [1,2).

   
   



Since the expression of the interval varextropy is not given in terms of elementry functions, we plot the interval varextropy as a function of t_1 for fixed t_2 (see Figure <ref>) and a function of t_2 for fixed t_1 (see Figure <ref>).  Figure <ref> shows that the interval varextropy for this distribution is not monotone in t_2 for some fixed t_1 and not monotone in t_1 for some fixed t_2.


 §.§ Some Properties for Interval Varextropy

In this subsection, we investigate some properties of the interval varextropy.


In the next theorem, we look for interval varextropy under a linear transformation.

Suppose that X is a non-negative and absolutely continuous random variable. Let Y=aX+b, with a>0,   b>0. Then, their interval varextropies are related by 
    IV_Y(t_1,t_2)=1/a^2IV_X(t_2-b/a, t_1-b/a).



 
 The proof is similar as that of Proposition 3 of <cit.>.
 

<cit.> defined the Generalized Failure Rate (GFR) functions of random variable X in t_1 and t_2 as 

    h_i(t_1, t_2)=f(t_i)/(F(t_2)-F(t_1)), i=1,2, (F(t_2)-F(t_1)>0),

where f and F are density and distribution function of random variable X, respectively.

In the following theorem, we show that under some conditions on IV(t_1, t_2), IJ(t_1, t_2) can be obtained in terms of GFR functions.


Let the interval varextropy IV(t_1,t_2) be constant, that is, IV(t_1,t_2)=v≥ 0, for all (t_1,t_2) ∈ D. Then, 
    IJ(t_1,t_2)=-1/4(h_1(t_1,t_2)+h_2(t_1,t_2))
.



Since  IV(t_1,t_2)=v, is constant, we have 

    ∂ IV(t_1,t_2)/∂ t_1   =   3h_1(t_1,t_2)v-h_1(IJ(t_1,t_2))^2
          -1/4(h_1(t_1,t_2))^3-(h_1(t_1,t_2))^2 IJ(t_1,t_2)=0,


    ∂ IV(t_1,t_2)/∂ t_2   =   -3h_2(t_1,t_2)v+h_2(t_1,t_2)(IJ(t_1,t_2))^2
          +(h_2(t_1,t_2))^2 IJ(t_1,t_2)+1/4(h_2(t_1,t_2))^3=0.

By solving these two equations simultaneously, the desired result is easily obtained. 



Let  X be a random variable with support (0,∞), differentiable and strictly positive density function f and distribution function F. If IV(t_1,t_2)=v ≥ 0  for all (t_1,t_2) ∈ D, then X is exponentially distributed.



The proof of this theorem is obtained according to Theorem 2 and Theorem 4 of <cit.>.


In the following, we give some bounds for the interval varextropy. Suppose that m(t_1,t_2) and σ ^2(t_1, t_2) express the mean and variance of X in the interval (t_1,t_2) which are defined as 

    m(t_1,t_2)=E(X|t_1<X<t_2)=∫_t_1^t_2xf(x)/F(t_2)-F(t_1) dx,

and

    σ^2(t_1,t_2)=Var(X|t_1<X<t_2)=∫_t_1^t_2x^2f(x)/F(t_2)-F(t_1)-(m(t_1,t_2))^2 dx.

A lower bound for IV(t_1,t_2) in terms of m(t_1,t_2) and σ^2(t_1,t_2) is given in the following theorem.

Let  f_t_1,t_2(x)=f(x)/F(t_2)-f(t_1) be  the density function of random variable X_t_1,t_2=(X|t_1<X<t_2). We have  

    IV(t_1,t_2) ≥1/4σ^2 (t_1,t_2) ( E_t_1,t_2[( W'_t_1,t_2(X_t_1,t_2)f_t_1,t_2(X_t_1,t_2)])^2,


where

    σ^2 (t_1,t_2) W_t_1,t_2(x)f_t_1,t_2(x)=∫_0^x(m(t_1,t_2)-z)f_t_1,t_2(z) dz.



From <cit.>, we can write 

    IV(t_1,t_2)= 1/4 Var(f_t_1,t_2(X_t_1,t_2)) ≥1/4σ^2 (t_1,t_2)( E_t_1,t_2(W_t_1,t_2(X_t_1,t_2))f'_t_1,t_2(X_t_1,t_2))^2,

where W_t_1,t_2(x) is defined as

    σ^2(t_1,t_2)W_t_1,t_2(x)f_t_1,t_2(x)=∫_0^x(m(t_1,t_2)-z)f_t_1,t_2(z) dz.

On differentiating both sides of (<ref>), we can see that:

    W_t_1,t_2(x)f'_t_1,t_2(x)=(m(t_1,t_2)-x)f_t_1,t_2(x)/σ^2 (t_1,t_2)-W'_t_1,t_2(x)f_t_1,t_2(x).

Therefore,

    IV(t_1,t_2) ≥1/4σ^2 (t_1,t_2) ( E_t_1,t_2[(W'_t_1,t_2(X_t_1,t_2)f_t_1,t_2(X_t_1,t_2)])^2.



Many density functions belong to the exponential family, such as the normal density function, gamma, exponential, Chi-squared, Beta, Dirichlet, etc. In the following theorem, we find an upper bound for varextropy of a doubly truncated random variable if its density belong to the exponential family.

Let  X be  a random variable such that its density function belongs to the exponential family that is 

    f(x)=exp(η(θ)τ(x)+Q(θ)+R(x)),     x ∈
 
where η(θ), τ(x), Q(θ) and R(x) are known functions, η(θ)≠ 0 and R(x) is non-negative. Under conditions A(t_1,t_2)=∫_t_1^t_2τ(x) dx<∞ and B(t_1,t_2)=∫_t_1^t_2 R(x) dx<∞.

We can see that

    IV(t_1,t_2) ≤1/4(F(t_2)-F(t_1))^3-1/4(F(t_2)-F(t_1))^4[2η(θ)A(t_1,t_2)+2Q(θ)(t_2-t_1)+2B(t_1,t_2)+(t_2-t_1)]

 
 
 
 Using the inequality 1+x< e^x, -∞<x< ∞, we can write:
 
    IV(t_1,t_2)   =   1/4(F(t_2)-F(t_1))^3∫_t_1^t_2exp(3η(θ)τ(x)+3Q(θ)+3R(x)) dx
          -1/4(F(t_2)-F(t_1))^4(∫_t_1^t_2exp(2η(θ)τ(x)+2Q(θ)+2R(x)) dx)^2
          ≤G(t_2)-G(t_1)/4(F(t_2)-F(t_1))^3-(∫_t_1^t_2(2η(θ)τ(x)+2Q(θ)+2R(x)+1) dx)^2/4(F(t_2)-F(t_1))^4
          ≤1/4(F(t_2)-F(t_1))^3-[2η(θ)A(t_1,t_2)+2Q(θ)(t_2-t_1)+2B(t_1,t_2)+(t_2-t_1)]/4(F(t_2)-F(t_1))^4

  where G(x) is the distribution function corresponding to the density function of 
    g(x)=exp(3η(θ)τ(x)+3Q(θ)+3R(x)).
 




§ SOME ESTIMATORS FOR THE INTERVAL VAREXTROPY

In this section, we propose some estimators for the interval varextropy defined in (<ref>) and  investigate their some asymptotic properties.

∙   First Estimator

Let X_1:n≤ X_2:n≤⋯≤ X_n:n be the ordered sample and m be an integer satisfying 1≤ m≤ n. 

Set 
 
    IV_mn(t_1,t_2)   =   1/4(F_n(t_2)-F_n(t_1))^31/2(n-m)∑_j=1^n-m(m/n+1(X_j+m:n-X_j:n)^-1)^2 
          ×(I(t_1≤ X_j:n≤ t_2)+I(t_1≤ X_j+m:n≤ t_2))-1/4(F_n(t_2)-F_n(t_1))^4
          ×[1/2(n-m)∑_j=1^n-m(m/n+1(X_j+m:n-X_j:n)^-1)(I(t_1≤ X_j:n≤ t_2) 
          + I(t_1≤ X_j+m:n≤ t_2))]^2,

 where F_n(x)=1/n∑_i=1^nI(X_i ≤ x) is the empirical distribution function. In the following theorem, we establish the a.s. consistency of  IV_mn(t_1, t_2).
 
 For all bounded densities f provided m,n→∞, m/log(n)→∞ and m/n→ 0, we  have for 0<t_1<t_2<∞, 
    |IV_mn(t_1,t_2)-IV(t_1,t_2)|→ 0     a.s.



 Set
  
    T_mn= 1/2(n-m)∑_j=1^n-m[m/n+1(X_j+m:n-X_j:n)^-1]^2(I(t_1 ≤ X_j:n≤ t_2) + I(t_1≤ X_j+m:n≤ t_2)),

and 
  
    T^'_mn= [1/2(n-m)∑_j=1^n-m(m/n+1(X_j+m:n-X_j:n)^-1) (I(t_1≤ X_j:n≤ t_2)+ I(t_1≤ X_j+m:n≤ t_2))]^2.

Then, we can write

    |IV_mn(t_1,t_2)-IV(t_1,t_2)| ≤  |I_n|+|II_n|,

where

    I_n=1/4(F_n(t_2)-F_n(t_1))^3 T_mn-1/4(F(t_2)-F(t_1))^3∫_t_1^t_2(f(x))^3  dx,

and

    II_n=1/4(F_n(t_2)-F_n(t_1))^4 T^'_mn-1/4(F(t_2)-F(t_1))^4(∫_t_1^t_2(f(x))^2  dx)^2.

On the other hand,


    I_n=1/4T_mn A_n+1/4(F(t_2)-F(t_1))^3(T_mn-∫_t_1^t_2f^3(x)  dx),

where

    A_n=1/(F_n(t_2)-F_n(t_1))^3-1/(F(t_2)-F(t_1))^3.

But,

    |A_n| ≤6 sup_x |F(x)-F_n(x)|/(F_n(t_2)-F_n(t_1))^2(F(t_2)-F(t_1))^2.

The Glivenko-Cantelli theorem ensures that 

    lim_n →∞|A_n|=0, a.s.

 (<ref>),  (<ref>)  and Theorem <ref> of  <cit.> ensure that:
 
    lim_n →∞|I_n|=0, a.s.

Also, 
 
    II_n=1/4T^'_mn B_n+1/4(F(t_2)-F(t_1))^4(T^'_mn-(∫_t_1^t_2f^2(x)  dx)^2),

where

    B_n=1/(F_n(t_2)-F_n(t_1))^4- 1/(F(t_2)-F(t_1))^4.

The Glivenko-Cantelli theorem ensures that 

    lim_n →∞|B_n|=0, a.s.

From (<ref>), (<ref>) and Theorem <ref> of  <cit.>, we can see that 

    lim_n →∞|II_n|=0, a.s.

(<ref>), (<ref>) and (<ref>) complete the proof.


∙  Second Estimator
Set 
 
    IV(t_1,t_2)= 1/4(F_n(t_2)-F_n(t_1))^3∫_t_1^t_2f_n^3(x) dx-1/4(F_n(t_2)-F_n(t_1))^4(∫_t_1^t_2f_n^2(x) dx)^2,

 where  f_n(x)=1/nh_n∑_j=1^n K(x-X_i/h_n) is the kernel density estimation.
 
 In the following theorem, we prove a.s. consistency of IV(t_1,t_2).
  
Assume that density function  f  have a support on (0, ∞) and f^' is bounded. Let K be a kernel density function and 
∫_-∞^∞ |t|K(t) dt<∞. 
 Then under conditions   h_n →  0, n h_n →∞ and h_n^-1 n^-1/2(loglog n)^1/2→ 0 as n →∞,   we can see that for every 0<t_1<t_2< ∞:
 
    lim_n →∞|IV(t_1,t_2)-IV(t_1,t_2)|=0,  a.s.



 
    |IV(t_1,t_2)-IV(t_1,t_2)| ≤ |I_n|+|II_n|,

 where
 
    I_n=1/4(∫_t_1^t_2f_n^3(x)  dx) A_n+1/4(F(t_2)-F(t_1))^3(∫_t_1^t_2f_n^3(x)  dx-∫_t_1^t_2f^3(x)  dx),

and
 
    II_n=1/4(∫_t_1^t_2f_n^2(x)  dx) ^2 B_n+1/4(F(t_2)-F(t_1))^4([∫_t_1^t_2f_n^2(x)  dx]^2-[∫_t_1^t_2f^2(x)  dx]^2).

We use the following inequality in the proof of the theorem.

Let 1 ≤ p < ∞, then for functions q and u in L_p, we have
 
    ∫_0^∞| |q(t)|^p-|u(t)|^p|  dμ(t)   ≤     p2^p-1∫_0^∞|q(t)-u(t)|^p dμ(t)+p2^p-1( ∫_0^∞|u(t)|^p dμ(t))^1-1/p
       ×   ( ∫_0^∞|q(t)-u(t)|^p dμ(t))^1/p,

 where μ is a measure on the Borel sets on . Using this inequality we observe that
  
    | ∫_t_1^t_2f_n^3(x)  dx-∫_t_1^t_2f^3(x)  dx |    ≤   ∫_0^∞ | f_n^3(x) -f^3(x)  | dx≤    12 ∫_0^∞ | f_n(x) -f(x)   |^3 dx
       +    12 (∫_0^∞f^3(x) dx)^2/3(∫_0^∞|f_n(x)-f(x)|^3 dx)^1/3.

However,
  
    ∫_0^∞ | f_n(x)  dx-f(x)  |^3 dx   ≤   (sup_x  |f_n(x)-f(x)|^2) ∫_ | f_n(x)- f(x) |  dx
       :=   Y_n,1× Y_n,2.

Now, by using the Chung's law of the iterated Logarithms (LIL), we can write
  
    sup_x  |f_n(x)-f(x)|   ≤   sup_x  |f_n(x)-E(f_n(x))|+sup_x  |E(f_n(x))-f(x)| 
       ≤   sup_x  |h_n^-1∫_ K(x-y/h_n) dF_n(y)-h_n^-1∫_ K(x-y/h_n) dF(y) |
       +   sup_x  |∫_(f(x-h_n t)-f(x))K(t) dt| ≤sup_x  |f^'(x) | h_n^-1sup_x  |F_n(x)-F(x)|
       +   sup_x  |f^'(x) | h_n  ∫_ |t|K(t) dt
       =   h_n^-1O(n^-1/2(loglog n )^1/2)+O(h_n), a.s.

Clearly, by (<ref>),
 
    Y_n,1=O([(h_n^-1n^-1/2(loglog n  )^1/2)∨ h_n]^2), a.s.

Also, by the results on the a.s. behavior of L_1-norms of kernel density estimators (see page 149 of  <cit.>),
 
    Y_n,2=O(h_n^2+(n h_n)^-1/2+(n^-1log(n))^1/2), a.s.

(<ref>),(<ref>),(<ref>),(<ref>),(<ref>) and (<ref>), ensure that 
 
    lim_n →∞|I_n|=0,  a.s.

 In a similar way to the above discussion and using formula (<ref>), it  can be shown that 
 
    lim_n →∞|II_n|=0,  a.s.

 (<ref>),(<ref>) and (<ref>) complete the proof.


∙   Third Estimator

Set 
 
    IV(t_1,t_2)= 1/4(F_n(t_2)-F_n(t_1))^3∫_t_1^t_2f_n^2(x) dF_n(x)-1/4(F_n(t_2)-F_n(t_1))^4(∫_t_1^t_2f_n(x) dF_n(x))^2.

 
In the following theorem we show  a.s.  consistency of IV(t_1,t_2) under some conditions.
  
Let f and f^' be bounded. If h_n→ 0  as n→∞, then for every 0<t_1<t_2<∞, we can see that 
  
    lim_n →∞|IV(t_1,t_2)-IV(t_1,t_2)|=0,  a.s.




  
    |IV(t_1,t_2)-IV(t_1,t_2)| ≤ |I_n|+|II_n|,

 where
  
    I_n=1/4 A_n ∫_t_1^t_2f_n^2   dF_n(x) +1/4(F(t_2)-F(t_1))^3[∫_t_1^t_2f_n^2   dF_n(x)-∫_t_1^t_2f^2(x)  dF(x)],

 and
   
    II_n=B_n/4( ∫_t_1^t_2f_n  dF_n(x))^2 -1/4(F(t_2)-F(t_1))^4([∫_t_1^t_2f(x)  dF(x)]^2-[∫_t_1^t_2f_n(x)  dF_n(x)]^2).

 On the other hand,
   
    |∫_t_1^t_2f_n^2 dF_n(x)-∫_t_1^t_2f^2(x)  dF(x)|    ≤   ∫_t_1^t_2 |f_n^2(x)-f^2(x)|  dF_n(x)+| ∫_t_1^t_2f^2(x)  d(F_n(x)-F(x))| 
       ≤   sup_x  |f_n^2(x)-f^2(x) |(F_n(t_2)-F_n(t_1))
       +   | ∫_t_1^t_2(F_n(x)-F(x))f(x)f^'(x)  dx| 
       ≤   sup_x  |f_n(x)-f(x) |^2+2(sup_x  f(x)) sup_x  |f_n(x)-f(x) |
        +    sup_x  |F_n(x)-F(x) |(sup_x  f(x))sup_x  |f^'(x)| (t_2-t_1),

the Chung's LIL, (<ref>) and (<ref>) ensure that 

    lim_n →∞ |∫_t_1^t_2f_n^2 dF_n(x)-∫_t_1^t_2f^2(x)  dF(x)|=0, a.s.

From (<ref>), (<ref>) and (<ref>), we have 
  
    lim_n →∞|I_n|=0, a.s.

 In a similar way we can see that
 
    lim_n →∞|II_n|=0, a.s.

 (<ref>), (<ref>) and (<ref>) complete the proof.




§ SIMULATION STUDY

In this section, the results of the Monte Carlo studies on biases and mean squared errors (MSEs) of our proposed estimators are presented. We consider the gamma, uniform and exponential distributions, which are considered in many references. For each sample size, 10,000 samples were generated and MSEs  and biases of estimators were computed. We used the m=[√(n)+0.5]  formula to estimate the  interval varextropy suggested by <cit.>. They used this heuristic formula for entropy estimation. In addition, we choose the  Epanechnikov kernel K(x)=3/4(1-x^2)I(|x|<1)  as the kernel and its corresponding optimal value of h_n=1.06sn^-1/5, where s is the sample standard deviation. Tables <ref>-<ref> give simulated biases and MSEs  of IV_m, n(t_1, t_2), IV(t_1,t_2) and IV(t_1,t_2)  for gamma, uniform and exponential distributed samples, respectively. 
For gamma and  exponential distributions, we use t_1=0 and t_2=3 and for uniform distribution, we use t_1=0 and t_2=0.5.






It is apparent from the tables that  both the absolute bias and MSE  decrease as the sample size increases. The bold type in these tables indicates the  interval varextropy estimator achieves the minimal MSE (bias). In the case of gamma distribution, Table <ref>, it is observed that, the estimator IV(0,3) has the best performance in terms of bias and MSE. Under the uniform distribution, Table <ref>, it can be seen that, the MSEs and biases of IV(0,0.5) are always smaller than other proposed estimators. 

According to Table <ref>,  under exponential distribution, the estimator IV(0,3) performs better than other proposed estimators in terms of MSE and bias. 


§ SOME TESTS OF UNIFORMITY

In this section, we introduce some goodness-of-fit tests of uniformity. These tests are based on our proposed interval varextropy estimators and their percentage points and power values are obtained by Monte Carlo simulation.


Consider the class of continuous distribution functions F with density function f(x) concentrated on the interval [0, 1]. 
Since the interval varextropy of a random variable is non-negative, we have IV(t_1,t_2)=var(-1/2f(X_t_1,t_2))≥ 0.

If  f is a standard uniform density function, it is easy to see that for each  (t_1, t_2) ∈ D={(u,v)∈ _+^2: F(u)<F(v)}, IV(t_1, t_2)=0.

On the other hand, if there exists a (t_1, t_2) ∈ D where IV(t_1, t_2)=0, then we have f(x_t_1, t_2)=f(x)/F(t_2)-F(t_1)=c on [0, 1], where c is a constant.  From the property of density function f(x_t_1, t_2), it is obvious that f(x_t_1, t_2)=c=t_2-t_1, where implies that 

    f(x)=(t_2-t_1)(F(t_2)-F(t_1)),    ∀   x ∈ [0,1].

Also, the Mean Value theorem ensures that there exists  a l ∈ (t_1, t_2), such that 

    f(l)=F(t_2)-F(t_1)/t_2-t_1.

 (<ref>) and (<ref>) imply that f(l)=(t_2-t_1)(F(t_2)-F(_1))=F(t_2)-F(t_1)/t_2-t_1, i.e.,  t_2-t_1=1 or t_2=1, t_1=0. 
 
 This result and (<ref>) ensure that f is a standard uniform density function.
  
The above discussion gives us the idea to use the proposed interval  varextropy estimators as the test statistics of goodness-of-fit tests of uniformity. 

Let X_1, …, X_n be a random sample from a continuous distribution function F(x) on [0,1]. The null hypothesis is H_0: F(x)=x and the alternative hypothesis (denoted H_1) is the opposite of H_0. Given any significance level α, our hypothesis-testing procedure can be defined by the critical region:  

    G_n=IV_n(t_1, t_2)≥ C_1-α,

where IV_n(t_1, t_2) is one of the proposed estimators and C_1-α is the critical value for the test with level α. Notice when IV_n(t_1, t_2)p→IV(t_1, t_2), under H_0: G_np→0 and H_1: G_n converges  to a number larger than zero in probability.

Due to the complexity of the calculations, it is not easy to determine the distribution of the test statistics under H_0, so the critical values are calculated using the Monte Carlo method. Based on three different interval  varextropy estimators, we can propose the following test statistics for testing the uniformity:  

    GV_n   =IV_m, n(0, 1),
    
    GD_n   =IV(0,1),
    
    GB_n   =IV(0,1).

We compare the powers of our proposed test statistics with Kolmogorov-Smirnov statistic (see <cit.> and <cit.>).
The test statistic of Kolmogorov-Smirnov is given as follows:

    KS=max(max_1≤ i≤ n{i/n-X_(i)}, max_1≤ i≤ n{X_(i)-i-1/n}),

where X_(1),  …, X_(n) are order statistics. 





For power comparisons, we compute the powers of  our proposed tests and the powers of tests mentioned above under the following alternative distributions:

    A_k   : F(x)=1-(1-x)^k, 0≤ x≤ 1 ( for k=1.5, 2);
    
    B_k   : F(x)={[           2^k-1x^k,           0≤ x≤ 0.5;                     ( for k=1.5, 2, 3);;     1-2^k-1(1-x)^k,           0.5≤ x≤ 1                     ].
    
    
    C_k   : F(x)={[ 0.5-2^k-1(0.5-x)^k,           0≤ x≤ 0.5;                        ( for k=1.5, 2).; 0.5+2^k-1(x-0.5)^k,           0.5≤ x≤ 1                     ].

<cit.> uses these alternative distributions in his study on power comparisons of some uniformity tests. 

Based on 100,000 repetitions, the critical values are estimated as shown in Table <ref>.
Table <ref> contains the results of simulations of our proposed test statistics and competitive test statistic with 100,000 repetitions for n=10, 20, 30 and α=0.05. The bold type in Table <ref> indicates the statistic achieving the maximal power. 
According to Table <ref>, the performance  of tests depends on alternative distributions.
 
Against alternative A, our proposed test GD_n and  KS-based test  give competitive powers to different samples.  For sample size n=10, the test based on   GD_n is the best. For moderate sample size n=20, if alternative distribution is A_1.5, GD_n is better. For 
n=30, KS test   performs better than the others. For alternative distribution B, GD_n and GB_n are the best tests and other tests have low powers in this case.  If the alternative is C, the test based on KS is the best test.




§ REAL DATA

In this section, we illustrate the use of proposed estimators in the real case. The data set considered here represents the remission times (in months) of a random sample of 128 bladder cancer patients. These data were analyzed by <cit.>.
The data are:

0.08, 2.09, 3.48, 4.87, 6.94, 8.66, 13.11, 23.63, 0.20, 2.23, 3.52, 4.98, 6.97, 9.02, 13.29,
0.40, 2.26, 3.57, 5.06, 7.09, 9.22, 13.80, 25.74, 0.50, 2.46, 3.64, 5.09, 7.26, 9.47, 14.24, 25.82, 0.51,
2.54, 3.70, 5.17, 7.28, 9.74, 14.76, 6.31, 0.81, 2.62, 3.82, 5.32, 7.32, 10.06, 14.77, 32.15, 2.64, 3.88,
5.32, 7.39, 10.34, 14.83, 34.26, 0.90, 2.69, 4.18, 5.34, 7.59, 10.66, 15.96, 36.66, 1.05, 2.69, 4.23, 5.41,
7.62, 10.75, 16.62, 43.01, 1.19, 2.75, 4.26, 5.41, 7.63, 17.12, 46.12, 1.26, 2.83, 4.33, 5.49, 7.66, 11.25,
17.14, 79.05, 1.35, 2.87, 5.62, 7.87, 11.64, 17.36, 1.40, 3.02, 4.34, 5.71, 7.93, 11.79, 18.10, 1.46, 4.40,
5.85, 8.26, 11.98, 19.13, 1.76, 3.25, 4.50, 6.25, 8.37, 12.02, 2.02, 3.31, 4.51, 6.54, 8.53, 12.03, 20.28,
2.02, 3.36, 6.76, 12.07, 21.73, 2.07, 3.36, 6.93, 8.65, 12.63, 22.69.

<cit.> fitted this data using an exponential distribution with λ=0.106773. Considering this model, we have obtained the values of IV(t_1,t_2) and estimators for different (t_1,t_2) in Table <ref>. The closeness of the estimated results to the values of  IV(t_1,t_2) shows that our estimators are good in the real scenario.




§ CONCLUDING REMARKS

In this paper, we introduced the concept of interval varextropy where  is a varextropy measure for doubly truncated random variables. We investigated changes of this measure under linear transforms.
We showed that if IV(t_1,t_2) is constant, then IJ(t_1,t_2) can be obtained in terms of GFR functions. Also, we showed that if IV(t_1,t_2) is constant, then under some conditions, the desired distribution is exponential. We obtained lower and upper bounds for IV(t_1,t_2).
We proposed some estimators for IV(t_1,t_2) and proved some asymptotic properties for these estimators.

In the simulation study, we investigated the behavior of estimators based on bias and MSE. By using the proposed estimators, we proposed some tests for uniformity tests. Finally, we evaluated estimators for a  real data set.
 

§ ACKNOWLEDGMENTS

The authors would like to thank the referees for their careful reading and their useful comments which led to this considerably improved version.
plain

[Alizadeh Noughabi  and Shafaei Noughabi(2024)]NoughabiandNoughabi2024
Alizadeh Noughabi, H.  and Shafaei Noughabi, M. (2024).
On the Estimation of Varextropy.
Statistics, doi: 10.1080/02331888.2024.2370329.

[Balakrishnan et al.(2022)]Balakrishnanetal2022
Balakrishnan, N.,    Buono, F. and  Longobardi, M. (2022).
 On Tsallis Extropy with an Application to Pattern Recognition.
Statistics & Probability Letters, 180, 109241.

[Becerra et al.(2018)]Becerraetal2018
Becerra, A.,  De la Rosa, J. I.,  González, E. , Pedroza, A. D.  and  Escalante, N. I. (2018).
Training Deep Neural Networks with Non-Uniform Frame-Level Cost Function for Automatic Speech Recognition.
Multimedia Tools and Applications, 77(20), 27231–27267.

[Buono et al.(2023)]Buonoetal2023
Buono, F.,  Kamari, O.  and Longobardi, M. (2023).
Interval Extropy and Weighted Interval Extropy.
Ricerche di Matematica , 72, 283–298.

[Buono and Longobardi(2020)]BuonoandLongobardi2020
Buono, F. and  Longobardi, M. (2020).
A Dual Measure of Uncertainty: The  Deng  Extropy.
Entropy, 22(5), 582.

[Cacoullos and Papathanasiou(1989)]CacoullosandPapathanasiou1989
 Cacoullos, T. and  Papathanasiou, V. (1989).
Characterizations of Distributions by Variance Bounds.
Statistics & Probability Letters, 7(5), 351–356.

[Grzegorzewski and Wieczorkowski(1999)]GrzegorzewskiandWieczorkowski1999
Crzcgorzewski, P.  and Wirczorkowski, R. (1999).
Entropy-Based Goodness-of-Fit Test for Exponentiality.
Communications in Statistics–Theory and Methods, 28(5), 1183–1202.

[Di Crescenzo and Paolillo(2021)]DiCrescenzoandPaolillo2021
Di Crescenzo, A. and Paolillo, L. (2021).
Analysis and Applications of the Residual Varentropy of Random Lifetimes.
Probability in the Engineering and Informational Sciences, 35(3), 680–698.

[Eggermont and Lariccia(2001)]EggermontandLariccia2001
Eggermont, P. P. B. and LaRiccia, V. N. (2001).
Maximum Penalized Likelihood Estimation. 
Springer Series in Statistics, New York, USA.

[Goodarzi(2022)]Goodarzi2022
Goodarzi,  F. (2022).
Characterizations of some Discrete Distributions and Upper Bounds on Discrete Residual Varentropy.
JIRSS , 21(2), 231–248.

[Kamari  and  Buono(2021)]KamariandBuono2021
Kamari, O. and  Buono, F. (2021).
On Extropy of Past Lifetime Distribution.
 Ricerche di Matematica , 70(2), 505–515.

[Kolmogorov(1933)]Kolmogorov1933
Kolmogorov, A. N. (1933).
Sulla determinazione empirica di una legge didistibuziane.
Giornale dell’Istituto Italiano degli Attuari , 4, 83–91.

 [Krishnan et al.(2020)]Krishnanetal2020
 Krishnan, A. S.,  Sunoj, S.M. and Unnikrishnan Nair, N. (2020).
Some Reliability Properties of Extropy for Residual and Past Lifetime Random Variables.
 Journal of the Korean Statistical Society, 49(2), 457–474.

[Lad et al.(2015)]Ladetal2015
Lad, F.,    Sanfilippo,  G. and  Agró, G. (2015).
Extropy: Complementary Dual of Entropy.
Statistical Science, 30(1), 40–58.

[Lee and Wang(2003)]LeeandWang2003
Lee, E. T. and  Wang, J. W. (2003).
Statistical Methods for Survival Data Analysis. 
John Wiley & Sons, New York, USA.

[Martinas  and Frankowicz(2000)]MartinasandFrankowicz2000
 Martinas, K. and Frankowicz, M. (2000).
Extropy-Reformulation of the Entropy Principle.
Periodica Polytechnica Chemical Engineering, 44(1), 29–38.

[Navarro and Ruiz(1996)]NavarroandRuiz1996
 Navarro, J. and  Ruiz, J. M. (1996).
Failure-Rate Functions for Doubly-Truncated Random Variables.
IEEE Transactions on Reliability, 45(4), 685–690.

[Qui(2017)]Qui2017
Qiu, G.(2017).
The Extropy of Order Statistics and Record Values.
Statistics & Probability Letters, 120(C), 52–60.

[Qiu and Jia(2018)]QiuandIia2018
Qiu, G. and  Jia, K. (2018).
The Residual Extropy of Order Statistics.
Statistics & Probability Letters, 133, 15–22.

[Qui et al.(2019)]Qiuetal2019
Qiu, G.,   Wang, L. and Wang, X. (2019).
On Extropy Properties of Mixed Systems.
Probability in the Engineering and Informational Sciences, 33(3), 471–486.

[Raqab and Qui(2019)]RaqabandQiu2019
 Raqab, M. Z. and  Qiu, G. (2019).
On Entropy Properties of Ranked Set Sampling.
Statistics, 53(1), 210–226.   
  
[Shanker et al.(2015)]Shankeretal2015
 Shanker, R., Fesshaye, H. and Selvaraj, S. (2015).
On Modeling of Lifetimes Data using Exponential and Lindley Distributions.
Biometrics & Biostatistics International Journal , 2(5), 140–147.

[Shannon(1948)]Shannon1948 
Shannon, C. E. (1948).
A Mathematical Theory of Communication.
Bell System Technical Journal, 27(3), 379–423. 



[Sharma and Kundu(2023)]SharmaandKundu2023
Sharma, A. and Kundu, C. (2023).
Varentropy of Doubly Truncated Random Variable.
Probability in the Engineering and Informational Sciences, 37(3), 852–871.

[Smirnov(1939)]Smirnov1939
Smirnov, N. V. (1939).
Estimate of Derivation between Empirical Distribution Functions in two Independent Samples.
Bulletin Moscow University, 2(2), 3–16.

[Song(2001)]Song2001
Song, K. S. (2001).
Rényi Information, Loglikelihood and an Intrinsic Distribution Measure.
Journal of Statistical Planning and Inference, 93(1-2), 51–69.

[Stephens(1974)]Stephens1974
Stephens, M. A. (1974).
 EDF  Statistics for Goodness of Fit and Some Comparisons.
Journal of the American Statistical Association, 69(347), 730–737.

[van Es(1992)]VanES1992
van Es, B.(1992).
Estimating Functionals Related to a Density by a Class of Statistics Based on Spacings.
Scandinavian Journal of Statistics, 19(1), 61–72.
 
[Vaselabadi et al.(2021)]Vaselabadietal2021
 Vaselabadi, N. M., Tahmasebi, S.,   Kazemi, M. R. and   Buono, F. (2021).
Results on Varextropy Measure of Random Variables.
Entropy, 23(3), 356.

[Zamini et al.(2023)]Zaminietal2023
 Zamini, R.,  Goodarzi, F. and  Hashemi, F. (2023).
Some Kernel Estimators for Varextropy Function under Lengthbiased Sampling.
Communications in Statistics-Theory and Methods, doi: 10.1080/03610918.2023.2289354.

 




 Paper Title: A Ball Divergence Based Measure For Conditional Independence Testing.



On the use of calibration data in error-aware compilation techniques for NISQ  devices
    Handy Kurniawan


Universidad Complutense de Madrid

Madrid, Spain 

handykur@ucm.es
Laura Rodríguez-Soriano


Universitat Politècnica de València

Valencia, Spain 

lrodsor1@disca.upv.es
Daniele Cuomo


Universitat Politècnica de València

Valencia, Spain 

cuomo.daniele@outlook.com
Carmen G. Almudever


Universitat Politècnica de València
Valencia, Spain 

cargara2@disca.upv.es
Francisco García Herrero


Universidad Complutense de Madrid
Madrid, Spain 

francg18@ucm.es

    August 5, 2024
================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

§ ABSTRACT  

In this paper we introduce a new measure of conditional dependence between two random vectors  X and Y given another random vector Z using the ball divergence. Our measure characterizes conditional independence and does not require any moment assumptions. We propose a consistent estimator of the measure using a kernel averaging technique and derive its asymptotic distribution. Using this statistic we construct two tests for conditional independence, one in the model-X framework and the other based on a novel local wild bootstrap algorithm. In the model-X framework, which assumes the knowledge of the distribution of X| Z, applying the conditional randomization test we obtain a method that controls Type I error in finite samples and is asymptotically consistent, even if the distribution of X| Z is incorrectly specified up to distance preserving transformations. More generally, in situations where X| Z is unknown or hard to estimate,  we design a double-bandwidth based local wild bootstrap algorithm that asymptotically controls both Type I error and power. We illustrate the advantage of our method, both in terms of Type I error and power, in a range of simulation settings and also in a real data example. A consequence of our theoretical results is a general framework for studying the asymptotic properties of a 2-sample conditional V-statistic, which is of independent interest.  


Keywords:  Ball divergence, conditional independence testing, kernel density estimation, model-X framework, local wild bootstrap. 





§ INTRODUCTION

Conditional independence testing plays a central role in a variety of statistical applications such as graphical modeling <cit.>, causal discovery <cit.>, variable selection <cit.>, dimensionality reduction <cit.>, and computational biology <cit.>. Formally, given random vectors X,  Y,  Z, the conditional independence problem involves testing the hypothesis:   

    H_0:  X Y |  Z versus  H_1: X Y |  Z ,

based on i.i.d. samples {( X_i,  Y_i,  Z_i)}_1 ≤ i ≤ n from the joint distribution of ( X,  Y,  Z). This problem is well understood for parametric models, such as the multivariate Gaussian (where the classical partial correlation characterizes conditional independence) and for discrete data <cit.>. However, testing conditional independence is a particularly challenging  problem for continuous distributions in the nonparametric setting. For instance,  
<cit.> showed that if Z is continuous, then there is no unbiasedly estimable measure that characterizes conditional independence. Moreover, <cit.> (see also <cit.> and <cit.>) showed that for absolutely continuous (with respect to the Lebesgue measure) random variables it is not possible to have a non-trivial test with level asymptotically uniformly bounded by some given α∈ (0, 1) over the entire null hypothesis space H_0. Hence, pointwise control of Type I error is the best one can hope for, unless additional assumptions are made on the distribution of ( X,  Y,  Z). 

Numerous nonparametric conditional independence measures, with different moment and smoothness assumptions on the underlying distribution, have been proposed over the years. These include measures based on estimating conditional cumulative distribution functions <cit.>, conditional characteristic functions <cit.>, conditional probability density functions <cit.>, empirical likelihood <cit.>, kernel methods <cit.>, mutual information and entropy <cit.>, Rosenblatt transformations  <cit.>, copulas <cit.>, maximal nonlinear conditional correlation <cit.>, distance correlation <cit.>, estimating regression functions <cit.>, geometric graph-based measures <cit.>, among several others (see <cit.> for a more detailed review). 

To use a conditional dependence measure for testing the hypothesis (<ref>) one has to choose a rejection threshold for the corresponding  test statistic, which, as mentioned above, is a delicate matter that usually requires additional assumptions. An approach that has gained popularity recently is the model-X framework <cit.>, where one assumes that  _ X|  Z, the conditional distribution of X|  Z, is known. In this case, the conditional randomization test (CRT) can be implemented by repeatedly resampling from _ X|  Z to uniformly control the finite sample Type I error of any conditional dependence measure. This, in particular, has been applied to calibrate the Azadkia-Chatterjee conditional dependence measure <cit.> and the Kernel Partial Correlation (KPC) of <cit.>. <cit.> characterized the Type I error inflation of the CRT when an estimate of _ X|  Z is considered, and also demonstrate situations where this estimation error is asymptotically negligible. They also introduced a variant based on non-uniform permutations which is known as the Conditional Permutation Test (CPT). The CRT, however, is inapplicable in applications where _ X|  Z is unknown. In such situations there are two common approaches: (1) approximating the asymptotic null distribution <cit.> and (2) local bootstrap algorithms <cit.>. The former provides pointwise asymptotic Type I error control, although strong assumptions on the  estimation errors of the nuisance parameters are sometimes required for the results to hold. The latter, in its discrete variant  <cit.>, proceeds by sampling, for each Z_j, the observations in { X_i}_1 ≤ i ≤ n with probability proportional to their `distance' (in terms of some kernel) from Z_j, for 1 ≤ j ≤ n. However, theoretical understanding of the discrete local bootstrap for conditional independence testing is limited. Another approach that circumvents the estimation of _ X| Z are local permutation methods <cit.>, where one randomly perturbs the observations in { Y_i }_1 ≤ i ≤ n that are `close' to the conditioning variable Z_j, for 1 ≤ j ≤ n.  


In this paper, we propose a new measure of conditional dependence for Euclidean random vectors using the ball divergence <cit.>, a recently introduced robust measure of difference between 2 probability distributions. Our construction relies on the observation that X  Y|  Z if and only if the distributions of X |  Y,  Z and X |  Z are the same almost surely. Hence, a natural approach to constructing a conditional dependence measure is to quantify the difference between _ X|  Y,  Z and _ X|  Z, and averaging it (with a possible weight function) over the marginal distribution of ( Y,  Z). In this paper, we operationalize this approach using the ball divergence to measure the difference between the corresponding conditional distributions. We refer to this measure as the conditional ball divergence  (cBD) which is zero if and only if X  Y|  Z, that is, it characterizes conditional independence. Moreover, the cBD, unlike several other conditional dependence measures, such as the conditional distance correlation <cit.> and the generalized covariance measure <cit.>, does not require any moment assumptions, hence it is robust against outliers and extreme values. We illustrate the efficacy of the cBD in measuring and testing conditional independence through the following results: 

 


  * In Section <ref>, we propose an estimate of the cBD using a kernel averaging technique and study its large sample properties. To begin with we show that the cBD estimate is consistent, that is, the estimate converges to the population measure as the sample size increases (Theorem <ref>). Next, we derive the asymptotic distribution of both the pointwise and the averaged cBD estimates (see Theorem <ref> and <ref>, respectively). The latter, in particular, shows that the cBD estimate with an appropriate weight function is asymptotically normal under H_0. 


  * In Section <ref>, we calibrate the cBD estimate to develop tests for conditional independence in the model-X framework and using a novel local wild bootstrap algorithm. In the model-X framework, by repeatedly resampling from the known distribution of X |  Z one can implement the conditional randomization test (CRT) which controls level in finite samples and is also consistent with only finite number of resamples (Proposition <ref>). Furthermore, since the cBD estimate depends only on the ordering of the pairwise distances between the observations, our test remains consistent even if _ X |  Z is misspecified up to distance preserving transformations, such as, location shift, rotation, or homogeneous scaling.  The same also holds for the conditional permutation test (CPT) (see Proposition <ref>). In general, when _ X|  Z is not known, we propose a novel local wild bootstrap for calibrating the cBD statistic. Here, given Z_j, instead of directly resampling an X_i (based on their distance from Z_j), for 1 ≤ i, j ≤ n, we sample from a Gaussian distribution with mean X_j and a small variance (bandwidth), for 1 ≤ j ≤ n. This allows us to design a double-bandwidth strategy, where a larger bandwidth is used for computing the test statistic and a smaller bandwidth is used for the sampling, such that both Type I error and power are asymptotically controlled (Theorem <ref>). 


  * In Section <ref>, we compare the performance of the cBD test (calibrated using the local wild bootstrap) with other competing state-of-the-art methods both in simulations and on a benchmark data set. The cBD outperforms the other tests in a variety of simulation settings. Our method is especially powerful in detecting non-linear geometric dependencies and negative correlation between the pairwise distances. Our experiments also show that the discrete local bootstrap for calibrating conditional independence tests do not always control the Type I error in finite samples. In the real data analysis we are also able to achieve higher power with a smaller sample size compared to competing methods. 



A consequence of our theoretical analysis, is a general result about the asymptotic properties of a 2-sample conditonal V-statistic (Section <ref>). This framework can be applied more generally for studying the limiting properties of kernel density based estimates for conditional inference. As an illustration, we outline in Remark <ref> the derivation of the pointwise limit of the conditional distance correlation. We discuss possible extensions and directions for future research in Section <ref>. The proofs of the results are deferred to the Appendix. 



§ CONDITIONAL BALL DIVERGENCE

 

Suppose X,  Y, Z are random vectors in ℝ^d_X, ℝ^d_Y, and ℝ^d_Z, respectively. Throughout we will assume that the joint distribution of ( X,  Y,  Z) has density p_ X,  Y,  Z in ℝ^d_X+d_Y+d_Z, with marginal densities denoted by p_ X, p_ Y, p_ Z, p_ X,  Y, p_ Y,  Z, and p_ X,  Z, with respect to Lebesgue measures in the appropriate dimensions. We begin by recalling the definition of the ball divergence, specialized to the case of probability measures on Euclidean spaces <cit.>. Throughout we will assume that the Euclidean space is equipped with a metric ρ such that it becomes a separable metric space. 


The ball divergence between two probability measures P and Q in ^d is defined as

    Θ^2(P, Q)=∫∫ (P-Q)^2 B̅(  u, ρ(  v,  u)) {dP(  u) d P ( v)+ d Q( u) d Q(  v)},

where B̅(  u, ε) is the closed ball of radius ε centered at u in ^d. (Here, for any measurable set A ⊆^d, (P-Q)^2 (A) = (P(A) - Q(A))^2.)
 

A key property of the ball divergence is that Θ^2(P, Q) = 0 if and only if P=Q <cit.>. Now, note that X Y |  Z if and only if the distributions of X |  Y,  Z and X|  Z are the same almost surely. This motivates the following measure of conditional dependence.  
 
 
Suppose X,  Y, Z are random vectors in ℝ^d_X, ℝ^d_Y, and ℝ^d_Z, respectively, and a : ^d_Y×^d_Z→ [0, ∞) is a bounded, measurable weight function which is positive on the support of p_ Y, Z. Then the conditional ball divergence (cBD) between X and Y given Z, with weight function a, is defined as 

    ζ_a( X,  Y|  Z)
        = [ Θ^2(_ X|  Y,  Z, _ X|  Z) a( Y,  Z) ] ,

where the expectation is with respect to the joint distribution of ( Y,  Z). 
 
  
 
The following proposition shows that the cBD characterizes conditional dependence without any additional assumptions on the underlying distribution.

  For any weight function a as in Definition <ref>, ζ_a( X,  Y|  Z)≥ 0. Moreover, the equality holds if and only if X Y |  Z.



    The non-negativity of ζ_a( X,  Y|  Z) is obvious. Also, note that if X Y |  Z, then the distributions _ X|  Y,  Z and _ X|  Z are the same almost surely. Hence, by the distribution characterization property of the ball divergence <cit.>, Θ^2(_ X|  Y,  Z,_ X|  Z) = 0 almost surely. This implies, recalling (<ref>), ζ_a( X,  Y |  Z)=0, whenever X Y |  Z. 
    
    Conversely, if ζ_a( X,  Y |  Z)=0, then by the positivity of a we have, Θ^2(_ X|  Y,  Z, _ X|  Z)=0 almost surely. This implies, by the property of the ball divergence, _ X|  Y,  Z = _ X|  Z almost surely, hence, X Y |  Z.




Note that the measure ζ_a( X,  Y|  Z) is asymmetric in X and Y. It is easy to symmetrize the measure by considering the average/maximum of ζ_a( X,  Y|  Z) and ζ_a( Y,  Z|  Z). The subsequent results will continue hold for the symmetrized variants, however, for analytic simplicity, we study the asymmetric measure in this article. 
 


 
Similar to the Chatterjee-Azadkia conditional dependence measure <cit.>  and the Kernel Partial Correlation (KPC)  <cit.>, the cBD can be normalized to be within [0, 1], where 0 characterizes conditional independence and 1 characterizes complete conditional dependence (see Section <ref> and Proposition <ref> for further details).  Hence, the cBD, unlike covariance-based measures such as the conditional distance correlation <cit.>, has the ability to measure the `strength' of (possibly) non-linear dependence between X and Y given Z. 








 §.§ Consistent Estimation
 
 


We now describe a procedure for estimating the cBD based on i.i.d. samples {( X_i,  Y_i,  Z_i)}_1 ≤ i ≤ n from the joint distribution p_ X,  Y,  Z, using a kernel averaging method. Hereafter, we will choose  ρ to be the Euclidean metric, to be denoted by ·, although most of our results will continue to hold for general L_p metrics. Also, suppose K: [0,∞) → [0,∞) is a non-negative symmetric kernel function and h_1, h_2 > 0 are bandwidth parameters. Throughout, unless specified otherwise, vectors will be denoted in row format. Then for any Borel set A ⊂ℝ^d_X define, 

    ℙ_ X|  Y =  y,  Z =  z(A) :=∑_i=1^n K(( y,  z) -( Y_i,  Z_i) /h_1)I_A{ X_i}/∑_i=1^n K(( y,  z)  - ( Y_i,  Z_i)  /h_1)

and 

    ℙ_ X|  Z =  z(A) :=∑_i=1^n K( z-  Z_i/h_2)I_A{ X_i}/∑_i=1^n K( z -  Z_i/h_2)  ,
 
as the sample analogues of ℙ_ X|  Z =  z(A) and ℙ_ X| Y =  y,  Z =  z(A), respectively. Here, I_A( t) =  1 { t ∈ A} denotes the indicator function of the event A. Then the cBD measure (recall (<ref>)) can be estimated as follows:  

    ζ̂_a, n = 1/n∑_s=1^n Θ^2(ℙ_ X|   Y_s,  Z_s,ℙ_ X|  Z_s)a(   Y_s,  Z_s).

The estimate (<ref>) can be expanded out using the definition of the ball divergence (recall (<ref>)). For this, define 

    w_( Y_i,  Z_i)( y,  z) := K(( y,  z)  - ( Y_i,  Z_i)  /h_1)  and  w_ Z_i( z) := K( z -  Z_i /h_2) ,

and w( y,  z) := ∑_i=1^n w_( Y_i,  Z_i)( y,  z) and w( z) := ∑_i=1^n w_ Z_i( z). 
Then denoting δ( x,  y,  z) =  1 { z ∈B̅ (  x,  x -  y )}, the indicator that z lies inside in the ball  B̅ (  x,  x -  y ),  we can express (<ref>) as:  

    ζ̂_a, n    = 1/n∑_s=1^n ( A_n, s + C_n, s) a( Y_s,  Z_s) ,

where 

    A_n, s := ∑_1 ≤ u, v ≤ n( ∑_r = 1^n {w_( Y_r,  Z_r)( Y_s,  Z_s)/w( Y_s,  Z_s)   - w_ Z_r( Z_s)/w( Z_s)}δ( X_u,  X_v,  X_r) )^2
     w_ ( Y_u,  Z_u)  ( Y_s,  Z_s) w_ ( Y_v,  Z_v) ( Y_s,  Z_s) / w( Y_s,  Z_s)^2 ,
 
 and 
 
    C_n, s := ∑_1 ≤ u, v ≤ n( ∑_r = 1^n {w_( Y_r,  Z_r)( Y_s,  Z_s)/w( Y_s,  Z_s)   - w_ Z_r( Z_s)/w( Z_s)}δ( X_u,  X_v,  X_r) )^2
     w_ Z_u  ( Z_s) w_ Z_v( Z_s) / w( Z_s)^2 .
 

To derive the asymptotic properties of ζ̂_a, n we will need the underlying distributions to be smooth in a certain sense. This is formalized in the following definition: 



Let f( x,  y) be a density function with marginal densities denoted as f_ X and f_ Y, respectively. For any sequence δ_n that converges to zero define

    R_n( x,  y;  u) := 1/δ_n^2{f( x,  y+δ_n u)/f( x,  y)-1-δ_n ( ∂/∂ y f( x, y)/f( x, y))  u ^⊤}.

We say f is nice with respect to Y, if 

    sup_ u: u=1lim sup_n→∞∫(R_n( x,  y; u))^2 f( x, y)d x   is uniformly bounded in  y.




A sufficient condition for f( x,  y) to be nice in Y is that the variation of the first and second order partial derivatives of the logarithm of f_ Y| X are bounded (see Lemma <ref> in Appendix <ref> for details). 

To derive the large sample properties of ζ̂_a, n we will make the following assumptions on the distributions, the kernels, and the bandwidths: 


    
  *  The joint densities p_ X,  Z and p_ X,  Y,  Z are nice with respect to Z and ( Y, Z), respectively.  Moreover, p_ X,  Y,  Z and its marginals are uniformly bounded.
    
    
  *  The kernel K is uniformly bounded, non-negative, symmetric about zero, and compactly supported. Moreover,
       
    ∫_ℝ^d K( u) d u = 1  and ∫_ℝ^d u^⊤ u  K( u) d u= τ I_d ,
 
    where I_d is denotes the d × d identity matrix and τ > 0 is some constant. Here, d=d_Y+d_Z or d=d_Z depending on the context. 

    
  *  The bandwidths h_1,h_2 depend on the sample size n in such a way that 
    
    h_1^d_1/h_2^d_2→ 0, nh_i^d_i→∞,  and  nh_i^d_i+4→ 0,   for i = 1,2
 
    as n →∞, where d_1 = d_Y+d_Z and d_2 = d_Z.  (Here, d_Y and d_Z denote the dimensions of Y and Z, respectively). 




Assumption <ref> is standard in the nonparametric kernel regression and density estimation literature. For example, it is satisfied by the commonly used Epanechnikov kernel which we will also use in our experiments. Assumption <ref> ensures that h_1 and h_2 are neither too large nor too small, and they maintain a relative ordering between themselves. Under these assumptions we have the following result (see Appendix <ref> for the proof). 

 Suppose Assumptions <ref>, <ref>, and <ref> hold. Then, n →∞, 

    ζ̂_a, nP→ζ_a( X,  Y| Z),
  that is, ζ̂_a, n is a consistent estimator of ζ_a( X,  Y| Z). 








 §.§ Asymptotic Distribution
 



We now derive the asymptotic distribution of the cBD estimate. We begin by  reviewing some properties of the ball divergence. Towards this, let δ( x,  y,  z) =  1 { z ∈B̅ (  x, ρ( x,  y)} =  1 {ρ(  x,  z) ≤ρ( x,  y) } and define 
    η( x,   y,  z,  z') := δ( x,  y,  z) δ( x,  y,  z').
 
Then given 2 distributions P and Q and i.i.d. samples U_1,  U_2,  U_3,  U_4 and V_1,  V_2,  V_3,  V_4 from P and Q, respectively, one can write the ball divergence (<ref>) as (from the proof of <cit.>): 

    Θ^2(P, Q) = 𝔼 [ ϕ( U_1,  U_2,  U_3,  U_4;  V_1,  V_2,  V_3,  V_4)  ] ,
 
where 

    ϕ( u_1,  u_2,  u_3,  u_4;  v_1,  v_2,  v_3,  v_4) = ϕ_A( u_1,   u_2,  u_3,  u_4,  v_3,  v_4) + ϕ_C( v_1,  v_2,   v_3,  v_4,  u_3,  u_4) ,
 
with 

    ϕ_A( u_1,   u_2,  u_3,  u_4,  v_3,  v_4)    := η( u_1,   u_2,  u_3,  u_4) + η( u_1,   u_2,  v_3,  v_4) - η( u_1,   u_2,  u_3,  v_3) - η( u_1,   u_2,  u_4,  v_4) , 
    ϕ_C( v_1,  v_2,   v_3,  v_4,  u_3,  u_4)     := η( v_1,   v_2,  v_3,  v_4) + η( v_1,   v_2,  u_3,  u_4) - η( v_1,   v_2,  v_3,  u_3) - η( v_1,   v_2,  v_4,  u_4).
 
Note that |ϕ| ≤ 2. This shows that Θ^2(F, G) is a 2-sample divergence measure with a bounded core function ϕ of degree (4,4). Although function ϕ is not symmetric, it can be easily symmetrized as follows: 

    Θ^2(μ,ν) = 𝔼 [ ϕ'( U_1,  U_2,  U_3,  U_4;  V_1,  V_2,  V_3,  V_4)  ],

where ϕ' is the symmetrized version of the core function ϕ defined as, 

    ϕ'(  u;  v) = 1/4!^2∑_τ∈ S_4∑_σ∈ S_4ϕ( u_τ;  v_κ ) .

Here, S_4 denotes the set of all permutations of {1, 2, 3, 4}, u_τ = (  u_τ(1),  u_τ(2),  u_τ(3),  u_τ(4) ), and v_σ =( v_σ(1),  v_σ(2),  v_σ(3),  v_σ(4) ), for τ, σ∈ S_4. <cit.> showed that when P=Q, the function ϕ' is a first-order degenerate kernel, that is, 

    𝔼[ϕ'( U_1,  U_2,  U_3,  U_4;  V_1,  V_2,  V_3,  V_4)|  U_1] = 0  and 𝔼[ϕ'( U_1,  U_2,  U_3,  U_4;  V_1,  V_2,  V_3,  V_4)|  V_1] = 0 ,
 
almost surely, but [ϕ^'( U_1,…, U_4;  V_1,…,  V_4)| U_1, U_2] ≠ 0.



Using (<ref>), the cBD measure (<ref>) can be expressed as: 

    ζ_a, n = [
    ϕ'( X_1,  X_2,  X_3,  X_4;  X'_1,  X'_2,  X'_3,  X'_4)  a( Y,  Z) ],

where ( X,  Y,   Z) is distributed according to p_ X,  Y,  Z, X_1,  X_2,  X_3,  X_4 are i.i.d. _ X|  Y ,  Z, and X_1',  X_2',  X_3',  X_4' are i.i.d. _ X|  Z. 
We know from Theorem <ref> that ζ_a, n can be consistently estimated using   

    ζ̂_a, n = 1/n∑_s=1^n Θ^2(_ X|  Y_s, Z_s, _ X|  Z_s) a( Y_s, Z_s).

In the following theorem we obtain the limiting distribution of the individual terms in (<ref>), conditioned on Y=  y and Z =  z,  both under H_0 and H_1. Throughout, we interpret stochastic integrals in the Wiener sense <cit.> (see also <cit.>), where, unlike in the Wiener-Itô integral, the `diagonal' is not removed. 




  Assume the conditions of Theorem <ref> holds. Then we have the following results:

  
      
  (1) Under H_0, that is, X Y| Z, for any fixed y∈ℝ^d_Y and z∈ℝ^d_Z  with p_ Y,  Z( y,  z) > 0,
       
    nh_1^d_Y+d_ZΘ^2(ℙ_ X| Y =  y,  Z =  z, ℙ_ X |  Z=  z) 
       D→4 2∫ Q_0 ( x,  x';  y,  z) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x') ,
 
       where 
        
       
       
  * Q_0 ( x,  x';  y,  z) = [ϕ'( x, x',  X_1, X_2 ;  X_3, X_4,  X_5, X_6)], 
       where the expectations are taken with respect to X_1,…,  X_6 ∼ℙ_ X| Y =  y,  Z =  z and ϕ' is defined in (<ref>).
       
       
  * 𝔾_ℙ_ X|  Y=  y,  Z =  z is a Gaussian process indexed by L^2 functions in ℝ^d_X with covariance function as in (<ref>). 
       

        


       
  (2) Under H_1, that is, X Y| Z, for any fixed y∈ℝ^d_Y and z∈ℝ^d_Z  with p_ Y,  Z( y,  z) > 0, we have 
       
    √(nh_1^d_Y+d_Z)(Θ^2(ℙ_ X| Y =  y,  Z =  z, ℙ_ X |  Z=  z)-Θ^2(ℙ_ X| Y =  y,  Z =  z, ℙ_ X |  Z=  z)) 
       D→ 4∫ Q_1 ( x;  y,  z) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x) D = 4 N(0, Var__ X| Y=  y,  Z=  z[Q_1 ( X;  y,  z)) ] ,
 
       where 𝔾_ℙ_ X|  Y=  y,  Z =  z is as defined before and 
       
    Q_1 ( x;  y,  z) = [ϕ'( x, X_1, X_2, X_3;  X_1', X_2', X_3', X_4')]   ,

       with X_1,  X_2, X_3 ∼ℙ_ X| Y =  y,  Z =  z, X_1',…,  X_4' ∼ℙ_ X|  Z= z. 
  
  


The proof of Theorem <ref> is given in Appendix <ref>. The proof follows from a more general result about the limiting distribution of conditional V-statistics that might be of independent interest (see Section <ref>). 


 
Theorem <ref> shows that the pointwise limit of the cBD estimate has a Gaussian distribution under H_1 (see (<ref>)). 
However, under H_0, the asymptotic variance of the Gaussian is zero and the limit in (<ref>) is degenerate. This is because the ball divergence core function ϕ' is conditionally first-order degenerate under H_0 (see Definition <ref>). As a result, the limiting distribution in this case is given by a bivariate stochastic integral  (see (<ref>)). To express the limit in (<ref>) in an alternative form, note that by the spectral theorem, for y ∈ℝ^d_Y,  z ∈ℝ^d_Z fixed, 

    Q_0 ( x,  x';  y,  z) = ∑_i=1^∞λ_i ϕ_i ( x) ϕ_i( x'),

where {λ_i}_i ≥ 1 and {ϕ_i}_i ≥ 1 are the eigenvalues and the eigenvectors of the operator 

    ℋ(f)( x):= ∫ Q_0 ( x,  x';  y,  z) f(  x') d_ X| Y=  y,  Z=  z( x') ,

for f ∈ L_2(^d_X, _ X| Y=  y,  Z=  z). 
Denote U_i := ∫ϕ_i ( x) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x), for i ≥ 1. By the orthogonality of the eigenvectors, {U_i}_i ≥ 1 is a collection of independent Gaussian random variables with mean zero and variance Var_ℙ_ X|  Y=  y,  Z =  z[ϕ_i ( X)]. Hence, from (<ref>), 

    ∫ Q_0 ( x,  x';  y,  z) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x') D = ∑_i=1^n λ_i U_i^2 ,
 
which is the familiar representation of the distribution of a degenerate V-statistics as an infinite weighted sum of independent χ^2_1 random variables. 



Now, we proceed to derive the limiting distribution of the cBD estimate (<ref>). One challenge towards this is the random denominators in (<ref>) arising from the kernel density estimates. This issue is usually handled either by assuming the density of Y,  Z is bounded below or by choosing an appropriate weight function. Here, we implement the latter; as in <cit.>. The idea is to choose a weight function such that the random denominator in the corresponding estimate is cancelled. This leads us to define the following cBD measure with weight function a( y,  z) = a^*( y,  z) := p_ Y,  Z( y,  z)^4 p_ Z( z)^4: 

    ζ^*( X, Y| Z):=ζ_a^*( X, Y| Z) = [ Θ^2(_ X|  Y,  Z, _ X|  Z) p_ Y,  Z( Y,  Z)^4 p_ Z( Z)^4 ].

The natural empirical estimate of ζ^* is given by: 

    ζ̃_n^*    := 1/n∑_s=1^n Θ^2(ℙ_ X|   Y_s,  Z_s,ℙ_ X|  Z_s)p̂_ Y, Z^4( Y_s,  Z_s)p̂_ Z^4 ( Z_s) ,

where ϕ' is as defined in (<ref>) and p̂_ Y, Z( y, z) =w( y, z)/(nh_1^d_Y+d_Z) and p̂_ Z = w( z)/(nh_2^d_Z) are the kernel density estimators of p_ Y, Z and p_ Z, respectively (recall the discussion after (<ref>)). 
This estimate can be expanded out as a V-statistic, which incurs some bias  in the estimation procedure, due to the repetition of the indices in the summation. To avoid this, for the asymptotic results it is convenient to work with the following estimator, which takes the form of an U-statistic, 

    ζ̂_n^* := 1/(n)_9  ∑_ i ∈ ([n])_9φ_n(( X_i_1, Y_i_1, Z_i_1),( X_i_8, Y_i_8, Z_i_8),…,( X_i_9, Y_i_9, Z_i_9)) ,

where (n)_9 = n(n-1)⋯(n-8), (n)_9 is the set of all elements in [n]^9 with distinct indices, 

    φ_n(( X_1, Y_1, Z_1),…,( X_9, Y_9, Z_9)) 
        = 1/h_1^4(d_Y+d_Z) h_2^4d_Z∏_ℓ=2^5 w_( Y_ℓ, Z_ℓ)( Y_1, Z_1) ∏_ℓ=6^9w_ Z_ℓ( Z_1) ϕ' ( X_2,…,  X_5;  X_6, … X_9) ,
 
and ϕ' is the ball divergence core function (<ref>). The following theorem establishes the large sample properties of ζ̂_n^*.



 
  Assume the conditions of Theorem <ref> holds. Then, we have the following results: 
  
 
  (1) ζ^*( X, Y| Z)≥ 0 and the equality holds if and only if X Y| Z.
 
  (2) ζ̂_n^* is a consistent estimator of ζ^*( X, Y| Z), that is, 
 ζ̂_n^*P→ζ^*( X, Y| Z), as n→∞.

  (3) Under H_0:  X Y|  Z, if additionally we have n^2h_1^d_Y+d_Z+4→ 0 and h_2/h_1→ 0 as n→∞, then   
    
    nh_1^(d_Y+d_Z)/2ζ̂_n^*D→ N(0, 144  𝒞(K) σ^2),

    where 𝒞(K) := ∫(∫ K(( s,  t))K(( u,  v) - ( s,  t)) d s d t)^2 d ud v, 
    
    σ^2= ∫ Q_0^2( x_1,  x_2; y,  z) p_ X,  Y,  Z( x_1,  y,  z )p_ X,  Y,  Z( x_2,  y,  z ) p_ Y,  Z^6( y,  z) p_ Z^8( z) d x_1d x_2d yd z,
  
    with Q_0 as defined in Theorem <ref>. 
     
    


The proof of Theorem <ref> is given in Appendix <ref>. Note from (<ref>) that ζ̂_n^* is a U-statistics of order 9 with core function φ_n. The proof of Theorem <ref> proceeds by writing out the Hoeffding's projections of ζ̂_n^*. Under H_0, the first-order projection turns out to be asymptotically negligible and the limiting distribution is determined by terms arising from the second-order projection, which we analyze using results from <cit.> on degenerate U-statistics of order 2. For large samples, by estimating σ^2, we can use (<ref>) to obtain a test for conditional independence with precise asymptotic level. However, this approximation might not be very accurate for small samples. Moreover, although the U-statistic based estimate (<ref>) has a tractable limiting distribution, its computational complexity is very high. On the other hand, the  estimator proposed in Section <ref>, which can be expanded as a V-statistic, is computationally more efficient and it can be calibrated using a bootstrap or permutation method as discussed in Section <ref>, leading to a test that can detect conditional dependence even in moderately large samples. 



Another approach to reduce the computational complexity of the U-statistic based estimate (<ref>) is to consider the following estimator of ζ^*( X, Y| Z) which averages φ_n only over disjoint 9-tuples:

    ζ̂_n^L = 1/⌊ n/9 ⌋∑_i=1^⌊ n/9 ⌋φ_n(( X_9i-8, Y_9i-8, Z_9i-8),…,( X_9i, Y_9i, Z_9i)).

Clearly, ζ̂_n^L can be computed in 𝒪(n) time. Moreover, using similar arguments as in the proof of Theorem <ref> it can be shown that ζ̂_n^L is a consistent estimator of ζ^*( X, Y| Z) (under the same set of assumptions). However, the resulting test for conditional independence will be less powerful than the tests based on (<ref>) or (<ref>).  






§  LIMITING DISTRIBUTION OF CONDITIONAL V-STATISTICS

 
As is evident from the discussion in the previous sections, to derive the asymptotic properties in the cBD estimate (<ref>) one has to understand the limiting distribution of U/V-statistics that involve empirical estimates of the conditional distributions ℙ_ X|  Z =  z and ℙ_ X|  Y =  y,  Z =  z. Similar quantities appear in estimation of other nonparametric measures for conditional inference, such as the conditional distance covariance <cit.>, the tests in <cit.>, the conditional energy distance <cit.>, among others. In this section we distill the main techniques from our proofs and present a general result about the limiting distribution of a 2-sample conditional V-statistic, which can be broadly applied for deriving large sample properties of kernel density based estimates for conditional inference. 


We begin with a convergence result about a conditional empirical process. As before, we assume ( X_1,  Y_1,  Z_1), …, ( X_n,  Y_n,  Z_n) are i.i.d. samples with joint distribution _ X,  Y,  Z with density p_ X,  Y,  Z. Fixing y ∈^d_Y and z ∈^d_Z, let ℙ_ X|  Z =  z and ℙ_ X|  Y =  y,  Z =  z be the estimates of the conditional distributions ℙ_ X|  Z =  z and ℙ_ X| Y =  y,  Z =  z, respectively, as defined in Section <ref>. For bandwidths h_1 and h_2 satisfying Assumption <ref>, consider the conditional empirical processes, 

    S_n^ z :=    {√(nh_2^ d_Z )(∫ f( x) dℙ_ X|  Z =  z( d x)-_ℙ_ X|  Z =  z [ f( X) ] )| f∈ℱ_ z} , 
        
    
        S_n^ y, z :=    {√(nh_1^d_Y+d_Z)(∫ f( x) dℙ_ X |  Y= y,  Z =  z( d x)-_ℙ_ X|  Y=  y,  Z =  z [ f( X) ] )| f∈ℱ_ y, z},

where ℱ_ z := {f : ℝ^d_X→ℝ | ∫ f ( x )^2 dℙ_ X|  Z= z ( x) < ∞} and 
ℱ_ y,  z := {f : ℝ^d_X→ℝ | ∫ f ( x )^2 dℙ_ X|  Y =  y,  Z= z ( x) < ∞}. The following proposition establishes the finite-dimensional convergence of these processes. 



Suppose Assumptions <ref>, <ref>, and <ref> hold. Fix y ∈^d_Y and z ∈^d_Z such that p_ Z( z)>0 and p_ Y, Z( y, z)>0. Then the finite dimensional distributions of the processes
(S_n^ z, S_n^ y,  z) jointly converge to the finite dimensional distributions of 

    ( {∫ f( x) d𝔾_ℙ_ X|  Z =  z( x)| f∈ℱ_ z} , {∫ f( x) d𝔾_ℙ_ X|  Y=  y,  Z =  z( x)| f∈ℱ_ y,  z}) ,

where 𝔾_ℙ_ X|  Z =  z and 𝔾_ℙ_ X|  Y=  y,  Z =  z are independent Gaussian processes defined as follows: 




  * 𝔾_ℙ_ X|  Z =  z is a centered Gaussian process with covariance kernel, for f, g ∈ℱ_ z, 

    Cov[∫ f d𝔾_ℙ_ X|  Z =  z,∫ g d𝔾_ℙ_ X|  Z =  z] = c_K ( z) Cov_ℙ_ X|  Z =  z[ f( X), g( X)] ,
  
and c_K( z) := ∫_^d_Z K^2( u) d u/p_ Z( z).  


  * 𝔾_ℙ_ X|  Y=  y,  Z =  z is a centered Gaussian process with covariance kernel, for f, g ∈ℱ_ y,  z, 

    Cov[∫ f d𝔾_ℙ_ X|  Y=  y,  Z =  z,∫ g d𝔾_ℙ_ X|  Y=  y,  Z =  z]     = c_K ( y,  z) Cov_ℙ_ X|  Y=  y,  Z =  z[ f( X), g( X)] ,

where c_K( y, z) := ∫_^d_Y+d_Z K^2( u) d u/p_ Y, Z( y, z). 

   
 



The proof of Proposition <ref> is given in Appendix <ref>. 
Invoking this result we can derive the asymptotic properties of functionals of the conditional empirical process.  Specifically, given a core function ϕ:(^d_X)^r_1× (^d_X)^r_2→, for integers r_1, r_2≥ 0, consider the problem of estimating 

    θ := 𝔼[ϕ(  X_[r_1+r_2]) |{ Y_s =  y }_ 1  ≤ s ≤ r_1, { Z_s =  z }_1 ≤ s ≤ r_1+r_2  ] ,
 
where X_[r_1+r_2] = (  X_1, …,  X_r_1+r_2). 
     The plug-in empirical estimator of θ is given by: 
     
    θ̂_n     = ∫ϕ(x_1,…, x_r_1 ;  x_r_1+1,…, x_r_1+r_2) ∏_i=1^r_1d_ X| Y =  y, Z =  z ( x_i) ∏_j=r_1+1^r_1+r_2d_ X|  Z= z  ( x_j) .

     This is a 2-sample conditional V-statistic with a core function of order (r_1, r_2). To derive the asymptotic distribution of θ̂ we need to compute its Hoeffding's decomposition. For this, denote the symmetrization ϕ by: 
    
    ϕ'( x_1,…,  x_r_1+r_2) = 1/r_1! r_2!∑_τ∈ S_r_1∑_σ∈ S_r_2ϕ(  x_τ;  x_ r_1+ σ) ,
 
    where, for any integer r ≥ 1, S_r is the set of all permutations of {1, 2, …, r}, x_τ = (  x_τ(1), … x_τ(r_1) ), for τ∈ S_r_1, 
    and x_r_1+ σ = (  x_r_1+σ(1), … x_r_1+ σ(r_2) ), for σ∈ S_r_2. (Note that if ϕ is symmetric in the first r_1 and last r_2 coordinates, then ϕ' = ϕ.)  

 For 0 ≤ i ≤ r_1 and 0 ≤ j ≤ r_2, the (i,j)-th order (conditional) Hoeffding's projection of ϕ  is defined as: 

    ϕ_i, j ( x_1, …,  x_i;  x_1', …,  x_j') 
        := [ϕ'( x_1, …,  x_i,  X_i+1, …,  X_r_1;  x_1', …,  x_j',  X_r_1+j+1, … X_r_1+r_2)| 𝒢_ y,  z^(i, j)] ,

where 𝒢_ y,  z^(i, j) := {{ Y_s =  y,  Z_s =  z }_i+1 ≤ s ≤ r_1, { Z_s =  z }_ r_1+j+1 ≤ s ≤ r_1 + r_2} and ϕ' is the symmetrization of ϕ. 

 


  * The function ϕ is said to be conditionally non-degenerate, if either ϕ_1, 0 0 or ϕ_1, 0 0, almost surely. 


  * For k ≥ 1, the function ϕ is said to be conditionally degenerate of order k, if ϕ_i, j = 0, almost surely, for all a+b ≤ k and there exists a_0, b_0 such that a_0+b_0 = k+1 and ϕ_a_0, b_0≠0. 
 

  * The function ϕ is said to be conditionally completely degenerate, if ϕ_i, j = 0, almost surely, for all a+b < r_1+r_2. 

 




The following result gives the asymptotic distribution of θ̂, depending on the order of degeneracy of ϕ. Analogous results for one-sample conditional  U-statistics are derived in <cit.>. 

 Let θ and θ̂ be as defined in (<ref>) and (<ref>). Further, suppose 

    𝔼[ϕ^2(  X_[r_1+r_2]) |{ Y_s =  y }_ 1  ≤ s ≤ r_1, { Z_s =  z }_1 ≤ s ≤ r_1+r_2  ] < ∞,

and Assumptions <ref>, <ref>, <ref> hold. Then we have the following: 




  (1) θ̂_n = θ + R_n, where R_n=O_P((nh_1^d_Y+d_Z)^-1/2). 
          
          
  (2) If ϕ is conditionally non-degenerate, then 
              
    (nh_1^d_Y+d_Z)^1/2 (θ̂_n - θ ) D→  r_1∫ϕ_1, 0( x)  d𝔾_ℙ_ X|  Y=  y,  Z =  z( x)  D = N(0, r_1^2 σ^2) ,
 
     where σ^2 = c_K ( y,  z) Var_ℙ_ X|  Y=  y,  Z =  z [ ϕ_1, 0( X)]  and c_K( y, z) is defined in Proposition <ref>. 


          
          
  (3) If ϕ conditionally degenerate of order k, for some k ≥ 1, then 
     
    (nh_1^d_Y+d_Z)^(k+1)/2θ̂_n    D→r_1 k+1∫ϕ_k+1, 0( x_1…,  x_k+1)∏_i=1^k+1𝔾_ℙ_ X|  Y=  y,  Z =  z( x_i) ,
 
          where ϕ_k+1, 0 is the (k+1, 0)-th order Hoeffding's projection of ϕ. 
    

    



The proof of Proposition <ref> is given in Appendix <ref>. Proposition <ref>, in particular, gives us the asymptotic distribution of pointwise ball divergence Θ^2(ℙ_ X| Y =  y,  Z =  z, ℙ_ X |  Z=  z), both under the null and the alternative, as in Theorem <ref>. Moreover, as mentioned before, we expect this result to be useful in obtaining the pointwise limiting distribution of other nonparametric conditional independence or 2-sample estimates. We illustrate this in the following remark for the conditional distance covariance <cit.>. 



    The conditional distance covariance between 2 random vectors X and Y given another random vector Z =  z is defined as the square root of (see <cit.>): 
    
    𝒟( X, Y| Z= z) = 1/c_d_X c_d_Y∫_ℝ^d_X + d_Y|ϕ_ X, Y | Z =  z( t, s) - ϕ_ X| Z =  z( t)ϕ_ Y| Z =  z( s)|^2/ t^p+1 s^q+1d t d s ,
 
    where ϕ_ X |  Z =  z  (·), ϕ_ Y|  Z =  z  (·), and ϕ_ X,  Y|  Z =  z  (·),  
  are the conditional characteristic functions of X|  Z=  z, Y|  Z=  z, and ( X,  Y)|  Z=  z, respectively; and c_d_X = π^d_X/2/Γ((d_X+1)/2) and c_d_Y = π^d_Y/2/Γ((d_Y+1)/2). Following the derivation of <cit.>, (<ref>) can be alternatively expressed as:
    
    𝒟( X, Y| Z= z) = [φ(( X_1, Y_1),( X_2, Y_2),( X_3, Y_3),( X_4, Y_4))|{ Z_i =  z}_1≤ i≤ 4] ,
 
    where ( X_1, Y_1,  Z_1),( X_2, Y_2,  Z_2),( X_3, Y_3,  Z_3) are i.i.d. _ X, Y,  Z and 
   
    φ(( X_i, Y_i)_1≤ i≤ 4) =  X_1- X_2 Y_1- Y_2 +  X_1- X_2 Y_3- Y_4 - 2 X_1- X_2 Y_1- Y_3 .

Hence, the natural plug-in estimate of (<ref>) is: 
  
    𝒟_n^ z    = ∫φ(( x_i, y_i)_1≤ i≤ 4) ∏_i=1^4d_ X,  Y |  Z =  z ( x_i,  y_i)  .
 
It is known that 𝒟_n^ z is a consistent estimator of 𝒟( X, Y| Z= z) (see <cit.>), however, to the best of our knowledge, the asymptotic distribution of 𝒟_n^ z has not been previously explored. This can be addressed using Proposition <ref>.  
To  this end, note that the symmetrization φ^' of φ (as in (<ref>) with r_1=4 and r_2=0) is first-order degenerate under H_0 and is non-degenerate under H_1. Hence, using Proposition <ref> we have the following results. For this, we assume 
    
    𝔼[φ^2(( X_i, Y_i)_1≤ i≤ 4) |{ Z_s =  z }_1 ≤ s ≤ 4  ] < ∞

    and z ∈ℝ^d_Z is such that p_ Z( z) > 0.  
    
    
        
  * Under H_0: X Y| Z =  z we have,
        
    nh_1^d_Y+d_Z𝒟_n^ zD→4 2∫φ_2(( x_1,  y_1), ( x_2,  y_2)) ∏_i=1^2 d𝔾__ X, Y |  Z= z ( x_i,  y_i) D=6∑_i=1^∞λ_i U_i^2,

        where φ_2 is the second-order Hoeffding's projection of φ^' with respect to the distribution _ X, Y| Z= z as in Definition <ref>, {λ_i}_i ≥ 1 is a square integrable sequence, and {U_i}_i ≥ 1 is a sequence of i.i.d standard normal random variables. The distributional equality of the limiting random variable follows from spectral decomposition theorem as in Remark <ref>.  

        
  * Under H_1: X Y|  Z =  z we have,
        
    √(nh_1^d_Y+d_Z)(𝒟_n^ z - 𝒟( X, Y| Z= z))D→ 4 ∫φ_1( x ,  y )d𝔾__ X, Y| Z= z( x ,  y )D=𝒩(0,16σ^2),

        where φ_1 is the first-order Hoeffding's projection of φ^' with respect to the distribution _ X, Y| Z= z and σ^2= c_K( z) Var__ X, Y| Z= z[φ_1(  X ,  Y )].   
    




§ TESTING CONDITIONAL INDEPENDENCE
 



In this section, we will discuss methods to calibrate the cBD estimate for testing conditional independence. Throughout, we fix a weight function a and, suppressing the dependence on a,  denote the corresponding cBD measure by ζ( X,  Y| Z) and the estimate by ζ̂_n. 



 §.§ Tests under the Model-X framework



In the model-X framework <cit.> since the distribution of X|  Z is assumed to be known, one can implement the conditional randomization test (CRT) by repeatedly resampling from the distribution X|  Z to approximate the null distribution of the cBD estimate ζ̂_n. To describe this formally, denote the observed sample by 𝒟_0 := { (X_i, Y_i, Z_i)}_i=1^n.   The CRT calibration of the cBD estimate then proceeds as follows: 



  * Generate new observations X_1',  X_2',…,  X_n', where X_i^'∼ X|  Z_i, for 1 ≤ i ≤ n, and construct a new data set 𝒟' = { (X_i^',  Y_i,  Z_i)}_i=1^n. Repeat this M times to obtain resampled data sets 𝒟'_1,𝒟'_2…,𝒟'_M.  


  * Report the p-value 

    p_CRT = 1+∑_j=1^M 1{ζ̂_n(𝒟_j')≥ζ̂_n(𝒟_0)}/1+M,

where ζ̂_n(𝒟_j') denotes the test statistic computed on the j-th resampled data set 𝒟_j', for  1 ≤ j ≤ M. 

This procedure controls Type I error in finite samples and is also asymptotically consistent with a finite number of resamples as shown below (see Appendix <ref> for the proof). Throughout, we fix α∈ (0, 1). 


Suppose the assumptions of Theorem <ref> hold. Then the test function ϕ_CRT = 1{p_CRT<α}, controls Type I error in finite samples, that is, _H_0(ϕ_CRT) ≤α. Moreover, lim_n →∞_H_1(ϕ_CRT) =1,  for M>(1-α)/α. 



We can also calibrate the test using the conditional permutation test <cit.>, a variant of the CRT where one permutes the observations X_1,  X_2, …,  X_n non-uniformly in such a way that the dependence between X and Z is preserved. Specifically, we generate M permutations π_1,π_2,…,π_M of {1,2,…,n} independently from the distribution: 

    (Π = π| X,  Y,  Z) = ∏_ℓ = 1^n p( X_π(ℓ)|  Z_ℓ)/∑_π'∈ S_n∏_ℓ=1^n p( X_π'(ℓ)|  Z_ℓ) ,

where S_n is the set of all permutations of {1,2,…,n}, and p( x|  z) is the density function of the distribution of X|  Z= z. Then we construct resampled data sets 𝒟^π_j = {( X_π_j(i),  Y_i,  Z_i)}_i=1^n, for each 1 ≤ j ≤ M and compute the p-value: 

    p_CPT = 1+∑_j=1^M 1{ζ̂_n(𝒟^π_j)≥ζ̂_n(𝒟_0)}/1+M.

Here, as before, 𝒟_0 := { (X_i, Y_i, Z_i)}_i=1^n denotes the original data set and ζ̂_n(𝒟^π_j) is the test statistic computed on 𝒟^π_j, for 1 ≤ j ≤ M. 
This method also has the desired level and power properties as stated in the following proposition (see Appendix <ref> for the proof).  


Suppose the assumptions of Theorem <ref> hold. Then the test function ϕ_CPT = 1{p_CPT<α}, controls Type I error in finite samples, that is, _H_0(ϕ_CPT) ≤α. Moreover, lim_n →∞_H_1(ϕ_CPT) =1,  for M>(1-α)/α. 




 

In the model-X framework the knowledge of X|  Z plays a crucial role in controlling the Type I and Type II error rates. Naturally, misspecification of the distribution of X|  Z may lead to incorrect inferences. To investigate the effect of model misspecification on the CRT, suppose one resamples from the model ^*_ X|  Z, where _ X|  Z^* ≠_ X|  Z. Note that even if the model is misspecified, the CRT will have the desired error control properties if the distribution of ζ̂_n(𝒟') matches with the null distribution of ζ̂_n(𝒟_0). Towards this, note that the function ϕ' in (<ref>) depends only on the ordering of the pairwise distances of the observations { X_1,  X_2, …,  X_n}. Hence, the resampled test statistic and the observed test statistic will have the same distribution if _X |  Z can be obtained from _X |  Z^*  using a distance preserving transformation, that is, a translation, rotation or homogeneous scale transformation (the same change in scale along all coordinate axes). This is because the joint distribution of 1{ X_i- X_j≤ X_k- X_j}, for 1≤ i< j< k≤ n, w_( Y_i, Z_i)( Y_j,  Z_j), for 1≤ i< j≤ n, and w_ Z_i( Z_j), for 1≤ i< j≤ n,  will remain unchanged under such transformations. To illustrate this consider the following numerical example: Suppose 

    Z∼ N(0,1),   Y = Z + ε_1,  and  X = Z + rY+ ε_2,
 
where ε_1,ε_2 are i.i.d N(0,1) random variables. It is easy to see that when r=0, X Y| Z and when r≠0, X Y| Z. Here, we consider the cases r=0 and r=1. Note that for r=0, X| Z∼ Z + N(0,1) and for r=1, X| Z∼ 2Z+N(0,2). 
To compute our test statistic we set the bandwidths as: h_1 = c_1 n^-1/(d_Y+d_Z+2) and h_2 = c_2n^-1/(d_Z+2), where c_1 and c_2 are the averages of the coordinate-wise IQR of the observations { (Y_1, Z_1), …, (Y_n, Z_n) } and { Z_1,  …, Z_n}, respectively. Throughout the nominal level is set to α= 0.05. 




 Here, we consider _ X| Z^* ∼ 5Z + η, where η∼ N(10,25/(r+1)), to generate the resampled data when r=0 and r=1. Note that this is a scale and location shift of the actual distribution _ X| Z, hence the distributions of ζ̂_n(𝒟_0) and ζ̂_n(𝒟') should be the same. To check this we use the Kolmogorov-Smirnov (KS) test to evaluate the closeness of the distributions of ζ̂_n(𝒟_0) and ζ̂_n(𝒟'). The empirical power of the KS-test (computed based on 500 trials) for different sample sizes is reported in Figure <ref>. The power was close to the size 0.05 for all sample size for both r=0 and r=1, confirming that the distributions of ζ̂_n(𝒟_0) and ζ̂_n(𝒟') are indeed the same in the case, despite model misspecification. 





 
In this case we consider ^*_ X | Z =Unif(-|Z|,|Z|) to generate the resampled data. As before the empirical power of the KS-test (computed based on 500 trials) for different sample sizes is reported in Figure <ref>. The plots clearly shows that here the resampled test statistic had a significantly different distribution than that obtained from correctly specified _ X| Z. 









The above discussion shows that although the CRT with the cBD statistic is robust to distance preserving transformations, if the model is completely misspecified the CRT may lead to incorrect conclusions. In such cases one has to rely on a data-driven approach for approximating _ X| Z. This is discussed in the following section.  





 §.§ Local Wild Bootstrap Tests



In this section we propose a new bootstrap algorithm that approximates the density of _ X|  Z using kernel smoothing methods and generates the resampled data from this estimated distribution. 
Towards this, fix bandwidths h_0 and h_2' and consider the following estimate of the probability density function of X| Z, based on the data { ( X_i,  Z_i) }_1 ≤ i ≤ n: 

    p̂_ X|  Z= z( x|  z) = p̂_ X, Z( x, z)/p̂_ Z( z)    = 1/nh_0^d_Xh_2^' d_Z∑_s=1^n K(z - Z_s /h_2') ϕ( x- X_s/h_0)/1/nh_2^' d_Z∑_s=1^n K(z - Z_s /h_2')
        = ∑_s=1^n κ_s( z) 1/h_0^d_Xϕ( x- X_s/h_0) ,

where ϕ denotes the probability density function of the standard normal distribution and 

    κ_s( z) :=  K(z - Z_s /h_2')/∑_j=1^n K( z- Z_ j/h_2')  ,

for 1 ≤ s ≤ n. Let _ X|  Z= z be the probability distribution function corresponding to p̂_ X|  Z= z( x| z). Note that (<ref>) takes the form of a Gaussian mixture model. Specifically, the law of a random variable X̂∼_ X|  Z= z can be described as follows:  X̂ is distributed as 𝒩( X_s, h_0^2  I_d_X) with probability κ_s( z), for 1 ≤ s ≤ n. Now, the local wild bootstrap method for calibrating the cBD statistic can be described as follows: 





  * Given the observed data 𝒟_0, generate X̂_i∼_ X|  Z_i, independently for each 1 ≤ i  ≤ n,  to get the resampled data set 𝒟̂ = {(X̂_i,  Y_i,  Z_i)}_i=1^n. Repeat this procedure M times to obtain resampled data sets 𝒟̂_1,𝒟̂_2,…,𝒟̂_M.



  * Report the p-value 

    p_LWB = 1+∑_j=1^M 1{ζ̂_n(𝒟̂_j)≥ζ̂_n(𝒟_0)}/1+M,

where ζ̂_n(𝒟̂_j) is our test statistic computed based on the data 𝒟̂_j, for 1 ≤ j ≤ n.  


Note that although the random variables {ζ̂_n(𝒟̂_j)}_1 ≤ j ≤ M are exchangeable, they do not have the same distribution as ζ̂_n(𝒟̂_0) under H_0. However, we can choose the bandwidths h_0 and h_2' in a such a way that the distributions of ζ̂_n(𝒟̂_1) and ζ̂_n(𝒟_0) become asymptotically close (in the total variation distance). As a consequence, the level of the local wild bootstrap test can be controlled at a desired level asymptotically and the test is also consistent in large samples.  This is formalized in the following theorem. 

Before proceeding with statement of the theorem, we consider a slight generalization. Note that one can consider implementing the local wild bootstrap method with any collection of (random) weight functions {γ_s}_1 ≤ s ≤ n where γ_s: ℝ^d_Z→ [0, 1] is such that ∑_s=1^n γ_s( z) = 1, for all z ∈ℝ^d_Z and 1 ≤ s ≤ n, and the weight functions only depend on the data through { Z_i}_1 ≤ i ≤ n. (Choosing γ_s = κ_s as in (<ref>) gives the method described above.) Of course, not all weight functions γ_s are meaningful, only those which `concentrate' around the data point Z_s and decay with distance from Z_s are statistically relevant. In the following theorem we show the asymptotic validity of the local wild bootstrap for weight functions satisfying a general condition. Implications of this condition are discussed after the theorem. 
 


    Suppose Assumptions <ref>, <ref>, <ref> hold, nh_0^2 → 0, 
    and p_ X, Z is nice with respect to X. Also, let {γ_s}_1 ≤ s ≤ n be a collection of weight functions such that  ∏_s=1^n γ_s( Z_s) P→ 1, 
    as n →∞. Then the test function ϕ_LWB = 1{ p_LWB<α} satisfies the following: 
    

  (a) lim_n, M →∞_H_0(ϕ_LWB) ≤α, that is, ϕ_LWB controls Type I error at α.

    
  (b) If M>(1-α)/α, lim_n →∞_H_1(ϕ_LWB) =1,  that is, the power of ϕ_LWB converges to 1, as n →∞. 
  
   


The proof of Theorem <ref> is given in Appendix <ref>. The result shows that the local wild bootstrap asymptotically controls Type I error and is consistent  when the weights functions satisfy ∏_s=1^n γ_s( Z_s) P→ 1. To understand this condition, consider, independently for each 1 ≤ s ≤ n, a random variable Ẑ_̂ŝ which takes value Z_i with probability γ_s( Z_i), for 1 ≤ i ≤ n. Then ∏_s=1^n γ_s( Z_s) is the probability that {Ẑ_s=  Z_s: 1≤ s ≤ n}. Hence, the condition ∏_s=1^n γ_s( Z_s) P→ 1 ensures that the resampled and the observed configurations are asymptotically unchanged, that is, each Z_s is resampled exactly once with high probability, for 1 ≤ s ≤ n. There are 2 natural ways to ensure the condition ∏_s=1^n γ_s( Z_s) P→ 1 holds: (1) choosing γ_s= κ_s as in (<ref>) with the bandwidth h_2' satisfying h_2' = o(n^-2/d_Z) (see Lemma <ref> in Appendix <ref>), and (2) choosing γ_s = δ_ Z_s, for 1 ≤ s ≤ n, where δ_ z denotes the point mass at the point z. We adopt the latter choice in our numerical studies in Section <ref>. 


Another interesting consequence of Theorem <ref> is that it allows us to choose the bandwidth h_0 separately from the bandwidths h_1, h_2 which are used for computing the test statistic.  This double-bandwidth approach is necessary to resolve the dichotomy between Type I error and power that is inherent in permutation/bootstrap calibration of conditional independence measures. The finer choice of h_0 ensures the Type I error control, while the coarser choices of h_1, h_2 ensure efficient estimation of the cBD estimate, leading to better power. This also bears resemblance with the double-binning local permutation method in  <cit.>, where a similar condition on bin-widths appears (see the following remark).

 
Recently, <cit.> proposed a local permutation method for conditional independence testing by partitioning the space of the conditioning random variable Z and permuting the variables ( X,  Y) within each bin, to approximate the null distribution. They proved asymptotic Type I error rate control of the local permutation method, in the total variation sense, when the bin width h is such that nh^2→ 0. However, to have good power a larger bin size is required for computing the test statistic. This leads to a double-binning approach, where they used a larger bin size to estimate the test statistic and a smaller bin size to calibrate the test. However, their method requires Z to have bounded support and only  the case where Z is univariate is considered. In contrast, our resampling algorithm is applicable even when Z has unbounded support or is multidimensional. 


To further investigate how well the local wild bootstrap performs under the null hypothesis, 
we test the closeness of the distribution of the resampled test statistic ζ̂_n(𝒟̂) and the distribution of the test statistic obtained using the CRT method ζ̂_n(𝒟^') (which assumes the knowledge of _ X |  Z) using the following simulation. For comparison we also implement the same for the discrete local bootstrap. Specifically, we consider the same setup as in Remark <ref>: 


    Z∼ N(0,1),   Y = Z+ϵ_1,  and X = Z +ϵ_2,
 where ϵ_1,ϵ_2 are i.i.d. N(0,1) random variables. Here, also we take the bandwidths h_1 = c_1n^-1/(d_Y+d_Z+2) and h_2 = c_2 n^-1/(d_Z+2), where c_1 and c_2 are the averages of the coordinate-wise IQR of the observations {(Y_1,Z_1),…, (Y_n,Z_n)} and {Z_1,…, Z_n}, respectively. For every n, we repeat the resampling algorithms 200 times and use the Kolmogorov-Smirnov (KS) test to evaluate the closeness of the respective distributions. The nominal level is kept fixed at α = 0.05. Our findings are reported in Figure <ref>. We observe that the null distribution of the local wild bootstrapped test statistic is close to the distribution of the CRT resampled test statistic. 
On the other hand, the distribution of the discrete local bootstrapped test statistic is drastically different. 

This difference diminishes with increasing sample size, but one may require a very large sample size for this difference to be negligible. 










 §.§ Choice of the Weight Function



Recall that in Section <ref> we considered the cBD measure with weight function a( y, z) = p_ Z( z)^4 p_ Y, Z( y, z)^4. This choice is convenient because the corresponding cBD estimate is asymptotically Gaussian under H_0. Although the limiting null distribution of the cBD estimate might be intractable for general weight functions, Theorem <ref> provides a way to calibrate the cBD estimate that is asymptotical consistent for any choice of the weight function a(·, ·). This raises the question: How does the choice of the weight function effect the finite-sample power of the cBD based tests? We explore this through a small simulation with three possible weight functions (a) a( y, z) =1, (b) a( y, z) = p_ Y, Z( y, z)^2, and (c) a( y,  z) = p_ Z( z)^4p_ Y, Z( y, z)^4. For ease of computation, here we calibrate the test using the V-statistic cBD estimates. 




Consider the model 
    Y = Z + η_1  and  X = Z + rY+η_2,
 
where η_1,η_2 and Z are i.i.d. N(0, 1). For each of the 3 weight functions, the empirical power of the corresponding cBD test is computed for different values of r keeping the sample size fixed at n=50. The bandwidths h_1, h_2 for computing the cBD statistic are set as in Example <ref>. Also, the bandwidth for the local bootstrap is chosen to be h_0 = 20 c_2 n^-1/1.95,  where c_2 is the IQR of {Z_1, … , Z_n}, which satisfies the assumption nh_0^2→ 0 as in Theorem <ref>. The results are reported in Figure <ref>. We see that the test corresponding to a( Y, Z) = 1 has the highest power, followed by a( y, z) = p_ Y, Z( y, z)^2 and a( Y, Z) = p_ Z( z)^4p_ Y, Z( y, z)^4. This shows that although the weight function a( Y, Z) = p_ Z( z)^4p_ Y, Z( y, z)^4 leads to a simple asymptotic null distribution, other choices can have better finite-sample performance. In particular, this example shows that choosing a( y, z)=1 might be beneficial in practice. 




§ ANALYSIS OF SIMULATED AND BENCHMARK DATA SETS
 


In this section, we evaluate the empirical performance of our test on simulated experiments and also on a benchmark data set. We compare our test with the tests based on the Conditional Distance Covariance <cit.>, the Generalized Covariance Measure <cit.>, and the weighted Generalized Covariance Measure <cit.>, which are referred to as the cDC test, GCM tests, and wGCM test, respectively. There are multiple ways to carry out the GCM test depending upon the method used to estimate the regression function. Here, we use the generalized additive model for that purpose. <cit.> proposed two different tests using the wGCM where one can either choose a weight function apriori and use it for the entire data or estimate the optimal weight function based on a training data set and apply the test on the validation data set. Here, also we use generalized additive models to estimate the regression functions and the corresponding tests are referred to as wGCM.fix test and wGCM.est test, respectively. 

In our experiments we choose h_1 = c_1 n^-1/(d_Y+d_Z+2) and h_2 =c_2 n^-1/(d_Z+2) to estimate the test statistic, where c_1 and c_2 are the averages of the coordinate-wise IQR of the observations { ( Y_1,  Z_1), …, ( Y_n,  Z_n) } and { Z_1,  …,   Z_n}, respectively.  For the local smooth bootstrap we let γ_s = δ_ Z_s, for 1 ≤ s ≤ n, which ensures ∏_s=1^n γ_s( Z_s) = 1 and choose h_0 = 20   c_2 n^-1/1.95. We refer to our test as the cBD test. In our simulated experiments, all p-values are computed based on 500 replications and the empirical power is estimated by the proportion of times H_0 is rejected over 500 repetitions. Throughout the nominal level is set at 5%. 




 §.§ Analysis of Simulated Data


In this section we compare the performance of the proposed test with existing methods in simulation settings. We begin with a simple one dimensional regression model:  

 Consider the regression model 
    Y = Z + η_1  and  X = Z + rY+η_2,
 where η_1 and η_2 are independent error terms and Z follows a univariate distribution. Here, X depends on both Y and Z when r≠0 and the dependence becomes stronger as |r| increases. We consider two cases: 


  (a) Z,η_1,η_2 are i.i.d. N(0,1) random variables and 

  (b) Z,η_1,η_2 are i.i.d. standard Cauchy random variables. 
 
Powers of different tests are computed as a function r with sample size fixed at n=50. The results are reported in Figure <ref>. In Example <ref> (a), the GCM test has the best performance, closely followed by the wGCM tests and the cDC test. Our test also has satisfactory performance, but its power is relatively lower compared to other tests. On the other hand, in Example <ref> (b), our test and the cDC test have competitive performance and they significantly outperform the regression-based tests (the GCM and wGCM tests). This shows the lack of robustness of the regression-based tests against outliers and extreme values generated from heavy-tailed distributions. While the cDC test has more power when |r| is small, the cBD test outperforms the cDC test for large |r|. 












 
Here, we consider bivariate distributions with geometric dependence structures.  Specifically, suppose Z∼ Unif(0,1), X = Z + η_1, and Y= Z + η_2, where (η_1,η_2) are generated from one of the six unusual bivariate distributions described in <cit.>. The scatter plot of observations from these six bivariate distributions (referred to as (a) Four Clouds, (b) W, (c) Diamond, (d) Parabola, (e) Two Parabolas, and (f) Circle)  are shown in Figure <ref>. It is easy to see that in the Four Clouds example, X Y| Z while for the rest we have XY| Z. Here we compute the power of the different tests for varying sample sizes, which are reported in Figure <ref>. Note that in each of these examples, the Pearson's correlation coefficient between (η_1,η_2) is zero. Hence, the residuals in the GCM and wGCM tests are expected to be uncorrelated, rendering these tests powerless. This is validated in Figure <ref>, where the GCM and wGCM tests have power close to the nominal level of 0.05 for all the examples. In the Four Clouds example, the powers of all tests are close to the nominal level α=0.05, as expected. In the W example, none of the tests have satisfactory performance. In the Diamond example, the power curve of our test gradually increases with the sample size, whereas those for the other tests remain near the nominal level. For the Parabola and Two Parabolas examples, the power curves of our test and cDC test both increase with the sample size, but our test significantly outperforms the cDC test. In the Circle example, our test also has significantly higher power compared to the other tests even when the sample size is moderately large. Overall, our test has the highest power among competing methods in detecting these unusual dependencies. 
 

 Suppose X, Y and Z are as in Example <ref>, with (η_1,η_2) generated from an equal mixture of two bivariate normal distributions. These bivariate distributions have mean zero and variance-covariance matrix Σ_1 and Σ_2, respectively. We consider two cases: 
    (a) Σ_1 = [ 1 0; 0 1 ], Σ_2 = [ 10  0;  0 10 ]   and    (b) Σ_1 = [  1  0;  0 10 ],  Σ_2 = [ 10  0;  0  1 ].
 
Figure <ref> shows the power of the tests for varying sample sizes. Here, the regression-based tests have very poor performance, because the  covariance between (η_1,η_2) is zero. In Example <ref> (a) our test slightly outperforms the cDC test. However, in Example <ref> (b) the cBD test significantly outperforms the cDC and the other tests. Note that in this case the correlation between pairwise distances of η_1 and  η_2 is negative, where distance correlation based methods tend to perform poorly. For instance, in the context of independence testing <cit.> noted that if the correlation between the pairwise distances is negative, the distance correlation test performs poorly even for moderately large sample size. 

 



 Here, we consider the bivariate analogue of Example <ref>. Specifically, suppose 
    Y =  Z + η_1 and  X=  Z+r Y+η_2,
 
where η_1 and η_2 are independent error terms and Z follows a bivariate distribution. We consider two scenarios: 
 

  (a) Z,η_1,η_2 are i.i.d. observations from a bivariate standard normal distribution, 

  (b) Z,η_1,η_2 are i.i.d. and the components are independent standard Cauchy random variables. 

We compute the power of the tests for different values of r and report them in Figure <ref>. In Example <ref> (a) our test has slightly lower power compared to the other tests (similar to the univariate case considered in Example <ref> (a)). However, in Example <ref> (b) only our test and the cDC test have satisfactory performances. Although, the cDC test has an edge over our test in this case, the probability of Type I error (the power at r=0) of the cDC test is 0.192, which is significantly higher than the nominal level of 0.05. On the other hand, our test controls Type I error at the nominal level. In that sense, the power of cDC and cBD tests are not comparable in this scenario. 
 



 


 
Now, we consider a multivariate analogue of Example <ref>. In particular, suppose Z = (Z_1,Z_2,Z_3) is generated uniformly from the unit cube [0,1]^3 and  
    (X,Y) = (max{Z_1,Z_2,Z_3}+ε_1, max{Z_1,Z_2,Z_3}+ε_2),
 where (ε_1,ε_2) are generated from the (a) Circle, (b) Parabola, and (c) Two parabola distributions as in Example <ref>. We compute the power of the tests for varying sample sizes and report them in Figure <ref>. Here, the cDC test has extremely poor performance with power equal to zero in all instances. The regression-based tests also perform poorly, since the errors are uncorrelated. Only our test is able to successfully detect this form of conditional dependence, and its power increases with the sample size. 



 





 §.§ Analysis of Benchmark Datasets
 


In this section, we analyze the `marks' data set available in the R package `bnlearn'. This data set contains marks obtained by students in the subjects Mechanics (M), Vectors (V), Analysis (An), Algebra (Al), and Statistics (S), all measured on a scale of 0-100. This dataset has been previously analyzed in <cit.> and <cit.> where the authors found that the marks obtained by the students in Statistics is conditionally dependent on that of Analysis given the marks on the rest of the subjects. They also found dependence of the marks in Mechanics and Vectors given the marks on the rest of the subjects. Therefore, considering these studies we consider testing (a) H_0: S An | (M,V,Al) and (b) H_0: M V| (S,An,Al) where we have evidence against the null. To compare the performance of the different methods we generate random sub-samples from the original data and carry out the tests based on those sub-samples. This procedure is repeated 500 times to estimate the powers of the tests by the proportions of times they reject  H_0. We evaluate the performance for different sub-sample sizes, and the results are given in Figure <ref>. In case (a), the GCM test has a better performance when the sample size is small, but our cBD test outperforms all other tests for relatively larger sample size. In case (b), when the sample size is small, the cDC test and GCM test has a better performance than our test, but when the sample size is large the cBD test significantly outperforms its competitors.    









§ EXTENSIONS AND FUTURE DIRECTIONS




In this paper, we propose a new measure of conditional dependence using the ball divergence. We estimate the measure using a kernel averaging method, study its asymptotic properties, and develop a test for conditional independence using a novel bootstrap approach. Our test exhibits improved performance both in simulations and real-data examples. 

Many extensions and variations of our method are possible. For instance, there has been a series of recent results, beginning with the breakthrough work of <cit.>, on measures of  dependence which are normalized between [0, 1], and takes the values 0 or 1, respectively, if and only if the corresponding pair of random variables is independent or one is a measurable function of the other almost surely (see <cit.> for a survey of recent developments). For  measuring conditional dependence, <cit.> proposed a correlation coefficient with similar properties, that is, it lies between 0 and 1, takes value 0 if and only if X  Y|  Z, and takes value 1 if and only if X is almost surely a measurable function of Y given Z. The kernel partial correlation (KPC) of <cit.> extends this to general topological spaces. In a similar vein, we can also construct a normalized measure for conditional dependence using the ball divergence. Specifically, taking the weight function a=1 in (<ref>), define the normalized cBD as follows: 

    ℛ( X,  Y|  Z) :=   [Θ^2(_ X|  Y,  Z, _ X|  Z) ) ]/𝔼 [∫Θ^2(_δ_ x, _ X|  Z) d_ X| Y, Z ( x)   ] .

Similar to the conditional dependence measures in <cit.> and <cit.> the normalized cBD satisfies the following trifecta of properties (see Proposition <ref>): (1) ℛ( X,  Y|  Z) ∈ [0, 1], (2) ℛ( X,  Y|  Z) = 0 if and only if X Y |  Z, and (3)  ℛ( X,  Y|  Z) = 1
if and only if X is a measurable function of Y and Z almost surely.

One can also consider defining a conditional dependence measure using the ball covariance of <cit.>. Given a pair of random vectors ( X,  Y) in ℝ^d_X+d_Y with joint distribution _ X,  Y
and marginal distributions _ X and _ Y, respectively, the ball covariance 
with weight functions ω_1 : ℝ^d_X×ℝ^d_X→ℝ_≥ 0 and ω_2 : ℝ^d_Y×ℝ^d_Y→ℝ_≥ 0, is defined as: 

    BCov_ω_1, ω_2^2( X,  Y) = ∫ (P_ X,  Y -P_ X⊗ P_ Y )^2    ( B̅(u_1, ρ(u_1, u_2)) ×B̅(v_1, ρ(v_1, v_2)) ) 
       ω_1(u_1, u_2) ω_2(v_1, v_2) {dP_ X,  Y(u_1, v_1) dP_ X,  Y(u_2, v_2)  } ,

where B̅(  u, ε) is the closed ball of radius ε centered at u in the appropriate dimension. The BCov characterizes (unconditional) independence under certain assumptions on the support of the distribution (see <cit.>). Using the BCov, one can define a conditional dependence measure between X and Y, given Z as follows: 

    cBCov_ω_1, ω_2, a^2( X,  Y |  Z) = [ BCov_ω_1, ω_2^2( X |  Z,  Y |  Z) a ( Z) ] ,

for some weight function a : ℝ^d_Z→ℝ_≥ 0. It can be shown that cBCov = 0 if and only if X Y |  Z, however, the conditions required on the support for this to hold can be restrictive in the context of conditional independence testing.  The cBCov can be estimated using a kernel averaging method similar to that in Section <ref>,  and its large sample properties can be studied using the framework developed in Section <ref>. While the cBCov resembles a classical covariance measure, the cBD can
measure the strength of dependence, as discussed in the previous paragraph. Exploring how the cBCov compares with the cBD is an interesting future direction. 
 






Acknowledgements: B. B. Bhattacharya was supported by NSF CAREER grant DMS 2046393, NSF grant DMS 2113771, and a Sloan Research Fellowship. 



apalike














§ PROOF OF THEOREM <REF>




Recall, from (<ref>), that ζ( X,  Y |  Z) = [Θ^2( ℙ_ X|  Y,  Z, ℙ_ X|  Z) a( Y,  Z)], where Θ^2(F, G) is the ball divergence between 2 probability distributions F and G as defined in (<ref>). Let ϕ be the core function of the ball divergence as in (<ref>). For notational convenience denote u = ( u_1,  u_2,  u_3,  u_4) and v = ( v_1,  v_2,  v_3,  v_4), and note that,

    Θ^2(_ X|  Y=  y,  Z =  z, _ X|  Z =  z) = ∫ϕ( u;  v) ∏_i=1^4 d_ X|  Y =  y ,  Z =  z ( u_i) ∏_i=1^4 d_ X|  Z =  z ( v_i).

The corresponding estimate is given by

    Θ^2(_ X|  Y =  y , Z =  z , _ X|  Z =  z ) = ∫ϕ( u;  v) ∏_i=1^4 d_ X|  Y =  y ,  Z =  z ( u_i) ∏_i=1^4 d_ X|  Z =  z ( v_i),

where 
ℙ_ X|  Y =  y ,  Z =  z (A) and ℙ_ X|  Z =  z (A) are as defined in (<ref>) and (<ref>), respectively. Now, using Proposition <ref> we have,

    Θ^2(_ X|  Y =  y , Z =  z , _ X|  Z =  z ) = Θ^2(_ X|  Y =  y , Z =  z , _ X| Z =  z ) + O_P(1/√(nh_1^(d_Y+d_Z))) .















































Therefore,   

    Θ^2(ℙ_ X|  Y =  y,  Z =  z,ℙ_ X|  Z =  z) a( y,  z) P→Θ^2( ℙ_ X|  Y =  y,  Z =  z, ℙ_ X|  Z =  z)a( y,  z) .
 
Then by the Dominated Convergence Theorem (DCT),

    Θ^2(ℙ_ X|  Y_1 ,  Z_1 ,ℙ_ X|  Z_1) a(  Y_1,  Z_1) P→Θ^2( ℙ_ X|  Y_1 ,  Z_1 , ℙ_ X|  Z_1 )a( Y_1,  Z_1) .
 
Moreover, since |Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1)| ≤ 2 and a( y, z) is bounded, 

    𝔼[ (
    Θ^2(ℙ_ X|  Y_1 ,  Z_1 ,ℙ_ X|  Z_1 ) a( Y_1,  Z_1) )^2 ]  ≤ 
    4[ a^2( Y_1,  Z_1) ] <∞.
 Hence, the sequence Θ^2(ℙ_ X|  Y,  Z,ℙ_ X|  Z) a( Y,  Z) is asymptotically uniformly integrable and from (<ref>) we have,   

    [ζ̂_a, n] = [ Θ^2(ℙ_ X |  Y_1,  Z_1,ℙ_ X|  Z_1) a( Y_1,  Z_1) ] →[Θ^2( ℙ_ X|  Y_1,  Z_1, ℙ_ X|  Z_1)a( Y_1,  Z_1)] = ζ_a( X, Y|  Z),
 
as n→∞. Therefore, the estimate ζ̂_a, n is asymptotically unbiased. 

To complete the proof of consistency, we will show that [ζ̂_a, n ]→ 0, as n→∞. To begin with, observe that 

    [ζ̂_a, n] = T_n^(1) + T_n^(2) ,
 
where 

    T_n^(1)    := 1/n[Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1) a( Y_1,  Z_1) ] , 
     
       T_n^(2)    : =  n(n-1)/n^2Cov[Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1) a( Y_1,  Z_1),Θ^2(ℙ_ X|  Y_2,  Z_2,ℙ_ X|  Z_2) a(  Y_2,  Z_2) ] .

Using |Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1)| ≤ 2 and the boundedness of a( y, z), it follows that 
    [Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1) a( Y_1,  Z_1)] = O(1),
 and hence, T_n^(1)→ 0. Therefore, it suffices to show that T_n^(2)→ 0. For this, using boundedness note that 

    [ ( Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1) a( Y_1,  Z_1) Θ^2(ℙ_ X|  Y_2,  Z_2,ℙ_ X|  Z_2) a( Y_2,  Z_2) )^2 ] ≤ 16 ( [ a^2( Y_1,  Z_1) a^2(  Y_2,  Z_2)] ) <∞.

Hence, (<ref>), uniform integrability, and DCT gives, 

    lim_n→∞𝔼[Θ^2(ℙ_ X|  Y_1,  Z_1,ℙ_ X|  Z_1) a( Y_1,  Z_1) Θ^2(ℙ_ X|  Y_2,  Z_2,ℙ_ X|  Z_2) a(  Y_2,  Z_2) ]  =  ζ_a( X,  Y |  Z)^2 .

Combining the above with (<ref>) gives, T_n^(2)→ 0. This implies, [ζ̂_a, n] → 0, as n→∞, which together with (<ref>) completes the proof of the Theorem <ref>.  




§ PROOF OF THEOREM <REF>






    
  (1) 
Note that under H_0:  X Y| Z, the core function of the ball divergence ϕ' (as in (<ref>)) is conditionally degenerate of order 1, in the sense of Definition <ref>. Hence, by Proposition <ref> (3) with r_1=r_2=4 and k=1, under H_0, 
    
    nh_1^d_Y+d_ZΘ^2(_ X| Y =  y,  Z =  z , _ X| Z= z) D→4  2∫ϕ_2, 0( x,  x') d𝔾__ X| Y=  y,  Z =  z( x)d𝔾__ X| Y=  y,  Z =  z( x') ,

    where ϕ_2, 0 is the (2, 0)-th order Hoeffding's decomposition of ϕ. Recalling (<ref>), it easy to see that 
ϕ_2, 0( x,  x') = Q_0( x,  x';  y, z),  
where Q_0 is as defined in Theorem <ref>. 

    
    
  (2) Under H_1: X Y| Z, the core function of the ball divergence ϕ' is conditionally non-degenerate.  Hence, by Proposition <ref> (2), 
    
    √(nh_1^d_Y+d_Z)(Θ^2(_ X| Y =  y,  Z =  z , _ X| Z= z)-Θ^2(_ X| Y =  y,  Z =  z, _ X| Z= z)) 
       D→ 4 ∫ϕ_1, 0( x) d𝔾__ X| Y=  y,  Z =  z( x) .

    From (<ref>),we have ϕ_1, 0( x) = Q_1( x;  y, z), where Q_1 is as defined in Theorem <ref>. 
    




§ PROOF OF THEOREM <REF>
 





 §.§ Proof of Theorem <ref> (1)
 

Recall from (<ref>)    

    ζ^*( X, Y| Z) = [Θ^2(_ X| Y, Z,_ X| Z)p_ Z^4( Z)p_ Y, Z^4( Y, Z)].

    Since Θ^2(_ X|  Y, Z,_ X| Z)≥ 0 we have ζ^*( X, Y| Z)≥ 0. The equality holds if and only if Θ^2(_ X|  Y, Z,_ X| Z)=0 almost surely, that is, if and only if _ X|  Y, Z=_ X| Z almost surely. This proves the result in Theorem <ref> (1).  
    
    


 §.§ Proof of Theorem <ref> (2)
 
    
   Recall the definition of ζ̂_n^* from (<ref>). The core function φ_n in (<ref>) can be expressed as: 
   
    φ_n(( X_1, Y_1, Z_1),( X_2, Y_2, Z_2),…, ( X_9, Y_9, Z_9))
        = 1/ h_1^4(d_Y+d_Z) h_2^4d_Z K_12⋯ K_15K_16'⋯ K_19' ϕ'( X_2,…,  X_5;  X_6,…,  X_9),

   where K_ts = K(( Y_t, Z_t)-( Y_s, Z_s)/h_1) and K_ts' = K( Z_t- Z_s/h_2). For notational convenience, let U_i:=( X_i, Y_i, Z_i). Now note,
    
    [ζ̂_n^*] = [φ_n( U_1, U_2,…,  U_9)]

    and
    
    [ζ̂_n^*] = n 9^-1∑_c=1^9 9 cn-9 9-c[φ_n^(c)( U_1,…,  U_c)] ,

    where
    
    φ_n^(c)( u_1,…,  u_c) = [1/9!∑_π∈ S_9φ_n( U_π(1),…,  U_π(9))| U_1= u_1,…,  U_c =  u_c ] ,

    is the c-th order Hoeffding's projection of φ_n (see Section 1.3 in <cit.> for the derivation of the above expressions). By Lemma <ref> we have, 
    
    [ζ̂_n^*]→ζ^*( X, Y| Z).
 
    Therefore, the result in Theorem <ref> (2) will follow if we can show [ζ̂_n^*] → 0, as n→∞. This is established in the following lemma: 
    
     For 1 ≤ c ≤ 9,  
    
    1/n^c[φ_n^(c)( U_1,…,  U_c)] → 0 ,

    as n→∞. Consequently, from (<ref>), [ζ̂_n^*] → 0. 
    
    
     Suppose U = ( X,  Y,  Z) is distributed with density p_ X,  Y,  Z. For c=1, 
    
    [φ_n^(1)( U)]     = [φ_n^(1)( U)^2] - ([φ_n^(1)( U)])^2.
 
    By Lemma <ref>, 
    ([φ_n^(1)( U)])^2 = ([ζ̂_n^*])^2→(ζ^*( X,  Y| Z))^2,  as n→∞. 
    Therefore, we need to find the rate of convergence of [φ_n^(1)( U)^2]. Note that, from (<ref>), we can write,
    
    φ_n^(1)( u)     = 1/9{[φ_n( u,  U_2,…,  U_9)] + 4 [φ_n( U_1,  u, U_3,…,  U_9)] + 4 [φ_n( U_1,  U_2,…,  U_5,  u,  U_7,…,  U_9)]}
        := 1/9{ϑ_1^(1)( u) + 4 ϑ_2^(1)( u)+ 4 ϑ_3^(1) ( u)} .
 
    By Lemma <ref>,
    
    [ϑ_1^(1)( U)^2]     = [[φ_n( U,  U_2,…,  U_9)][φ_n( U,  U_2',…,  U_9')]]*(where U,  U_2,…,  U_9, U_2',…,  U_9' are i.i.d.)
        =  [φ_n( U,  U_2,…,  U_9)φ_n( U,  U_2',…,  U_9')] 
       →∫[[ϕ'( X_2,…,  X_5;  X_6,…,  X_9)] p_ Z( z)^4 p_ Y, Z( y, z)^4]^2 p_ Y, Z( y, z)d y d z ,
 
    where the expectation is with respect to X_2,…,  X_5 i.i.d. from ℙ_ X |  Y= y,  Z =  z and X_6,…,  X_9 i.i.d. from _ X|  Z = z. Similarly, it can shown that [ϑ_2^(1)( U)^2] = O(1) and [ϑ_3^(1)( U)^2] = O(1). Hence, from (<ref>) and using (a+b+c)^2≤ 3(a^2+b^2+c^2), it follows that     
     [φ_n^(1)( U)^2] = O(1). Hence, 1/n[φ_n^(1)( U)] → 0, as n→∞. 
    
    Next, suppose c=2. Note from (<ref>) that,
    
    φ_n^(2)( u, u') = 1/72{ϑ_1^(2)( u, u') + ϑ_2^(2)( u, u') } ,

    where
    
    ϑ_1^(2)( u, u') :=      4 ϑ_1, 1^(2)( u, u') + 4 ϑ_1, 2^(2)( u, u') + 16 ϑ_1, 3^(2)( u, u') + 6 ϑ_1, 4^(2)( u, u') + 6 ϑ_1, 5^(2)( u, u') ,
 
    where 
    
    ϑ_1, 1^(2)( u, u')     := [φ_n( u,  u',  U_3,…,  U_9)] , 
    ϑ_1, 2^(2)( u, u')     := [φ_n( u,  U_2,…,  U_5;  u',  U_7,…,   U_9)] , 
    ϑ_1, 3^(2)( u, u')     :=  [φ_n( U_1,  u, U_3,…,  U_5;  u',  U_7,…,   U_9)] , 
    ϑ_1, 4^(2)( u, u')     := [φ_n( U_1,  u,  u',  U_4,  U_5;  U_6, …,   U_9)] ,
    ϑ_1, 5^(2)( u, u')     :=  [φ_n( U_1,  U_2, …,  U_5;  u,  u',  U_8,   U_9)]  ;

        and ϑ_2^(2)( u, u') = ϑ_1^(2)( u', u). For U,  U' are i.i.d. with density p_ X,  Y,  Z, it follows from (<ref>), (<ref>), and (<ref>),  
        
    h_1^(d_Y+d_Z)[φ_n^(2)( U, U')] →τ_2^2 ,
 
  for some constant τ_2^2 > 0. Hence, 1/n^2[φ_n^(2)( U,  U')] → 0, as n→∞. 

    Similarly, for any c=3,…, 9 it can be shown that there exists a constant τ_c^2 > 0 such that 
    
    h_1^(c-1)(d_Y+d_Z)[φ_n^(c)( U_1,…,  U_c)]→τ_c^2 ,

    as n→∞. This implies 1/n^c[φ_n^(c)( U_1,…,  U_c)] → 0, as n→∞. 
    
        
        The above lemma together with (<ref>) completes the proof of Theorem <ref> (2). 
        
        
        


 §.§ Proof of Theorem <ref> (3)
 

    
        
        To begin with, note that ζ̂_n^* is a U-statistics with kernel φ_n of degree 9. Let δ_n^(1)( u_1) = φ_n^(1)( u_1) and, for 2 ≤ c ≤ 9, 
  
    δ_n^(c)( u_1,…,  u_c) = φ_n^(c)( u_1,…,  u_c) - ∑_a=1^c-1∑_1≤ i_1≤⋯≤ i_a≤ cδ_n^(a)( u_i_1,…, u_i_a) ,
 
   where φ_n^(c)( u_1,…,  u_c) is as defined in (<ref>). From Section 1.6 in <cit.> we have,
   
    ζ̂_n^* = ∑_c=1^9 9 cΔ_n^(c) ,

   where Δ_n^(c) = n c^-1∑_1≤ i_1≤⋯≤ i_c ≤ nδ_n^(c)( U_i_1,…,  U_i_c) satisfies the following properties: For 1 ≤ c ≤ 9, 
   
       
  (a) The U-statistics Δ_n^(1), Δ_n^(2),…, Δ_n^(9) are uncorrelated.
       
  (b) [Δ_n^(c)] = n c^-1[δ_n^(c)( U_1,…,  U_c)].
       
  (c) [δ_n^(c)( U_1,…,  U_c)] = ∑_a=1^c (-1)^c-ac a[φ_n^(a)( U_1,…,  U_a)], for 1 ≤ c ≤ 9.
   
   
   We begin by showing that δ_n^(1) is asymptotically negligible. 
   
    Under H_0:  X  Y| Z, δ_n^(1)(  u) = O(h_1^4), almost surely. Moreover, 
    nh_1^(d_Y+d_Z)/2Δ_n^(1)L_2→ 0.

    

 
Note that 
   
    δ_n^(1)( u)     =φ_n^(1)( u) = 1/9{ϑ_1^(1)( u) + 4 ϑ_2^(1)( u)+ 4 ϑ_3^(1) ( u)} ,
 
   where ϑ_1^(1), ϑ_2^(1), and ϑ_3^(1) are as defined in (<ref>). 
   
    Note that 

    ϑ_1^(1)( u)     = [ φ_n( u, U_2,…, U_5;  U_6,…,  U_9)] 
        = 1/ h_1^4(d_Y+d_Z) h_2^4d_Z[∏_s=2^5 w_ Y_s, Z_s( y, z) ∏_s=6^9 w_ Z_s( z)ϕ'( X_2,…,  X_9)] 
        = p_ Y,  Z (  y,  z)^4  p_ Z ( z)^4[ ϕ'( X_2,  X_3,…,  X_9) |{ Y_s=  y }_2 ≤ s ≤ 5, { Z_s=  z }_2 ≤ s ≤ 9 ] +  O(h_1^2 h_2^2)  * (by Remark <ref>)
        = O(h_1^2 h_2^2) = O(h_1^4) ,
 
since ϕ' is conditionally degenerate of order 1 under H_0 and h_2/h_1→ 0 by assumption. Similarly, it can be shown that ϑ_2^(1)( u) = O(h_1^4) and ϑ_3^(1)( u) = O(h_1^4), almost surely. This implies, δ_n^(1)(  u) = O(h_1^4). 

Also, we have from Lemma <ref>, 

    [ϑ_1^(1)( U)]     = [ φ_n ( U, U_2,…, U_5;  U_6,…,  U_9)] 
       →∫ p_ Y,  Z (  y,  z)^5  p_ Z ( z)^4[ ϕ'( X_2,  X_3,…,  X_9) |{ Y_s=  y }_2 ≤ s ≤ 5, { Z_s=  z }_2 ≤ s ≤ 9 ] d y d z +  O(h_1^2 h_2^2)   
        = O(h_1^2 h_2^2) = O(h_1^4) .
 
Similarly, [ϑ_2^(1)( U)] = O(h_1^4) and [ϑ_3^(1)( U)] = O(h_1^4). Hence, 

    nh_1^(d_Y+d_Z)/2[Δ_n^(1)(  U)] = nh_1^(d_Y+d_Z)/2[ δ_n^(1)(  U) ] = O(nh_1^(d_Y+d_Z)/2 + 4) → 0 ,

since n^2 h_1^d_Y+d_Z+4→ 0.  
Next, from (<ref>) and Lemma <ref> we have [ϑ_1^(1)( U)^2] = O(h_1^8). Similarly, it can be shown that [ϑ_2^(1)( U)^2] = O(h_1^8) and [ϑ_3^(1)( U)^2] = O(h_1^8). This implies, 

    n^2h_1^(d_Y+d_Z)Var[ Δ_n^(1)(  U)] = nh_1^(d_Y+d_Z)Var[ δ_n^(1)(  U) ] = O(nh_1^(d_Y+d_Z + 8) )  → 0,

since n^2 h_1^d_Y+d_Z+4→ 0. Combining (<ref>) and (<ref>) shows that nh_1^(d_Y+d_Z)/2Δ_n^(1)L_2→ 0. 
  
   
   The above lemma shows that ζ̂_n^* is an approximately first-order degenerate U-statistics. Now, we consider the second-order Hoeffding's projection. 
   
       Under H_0:  X  Y| Z, 
             
    nh_1^(d_Y+d_Z)/2Δ_n^(2)D→ N(0, 144  σ_2^2 𝒞_2(K)) ,

              where σ_2^2 = σ^2 and 𝒞_2(K) = 𝒞(K) are as in the statement of Theorem <ref>. 
         


  
We have from (<ref>), δ_n^(2)( u,  u') = φ_n^(2)( u,  u') -  δ_n^(1)( u) - δ_n^(1)( u'). This means, 
   
    Δ_n^(2)    = 1/n  2∑_1 ≤ i < j ≤ nφ_n^(2)( U_i,  U_j) -  2/n∑_i=1^n δ_n^(1)( U_i) 
        = 1/n  2∑_1 ≤ i < j ≤ nφ_n^(2)( U_i,  U_j) -  o_P(1/nh_1^(d_Y+d_Z)/2) ,
 
by Lemma <ref>. Hence, it suffices to derive the limiting distribution 
  of n  2^-1∑_1 ≤ i < j ≤ nφ_n^(2)( U_i,  U_j). Towards this, recall from (<ref>), 
   
    φ_n^(2)( u, u') = 1/72{ϑ_1^(2)( u, u') + ϑ_2^(2)( u, u') } ,

where ϑ_1^(2) and ϑ_2^(2) are as defined in (<ref>).  Note that ϑ_1^(2)( u,  u') has 5 terms. The proof of Lemma <ref> now proceeds as follows: 



  * First we show that the U-statistics corresponding to the first 2 terms in (<ref>) are asymptotically negligible:   

    nh_1^(d_Y+d_Z)/2/n(n-1)∑_1 ≤ i  j ≤ nϑ_1, 1^(2)( U_i,  U_j) = o_L_2(1) ,  nh_1^(d_Y+d_Z)/2/n(n-1)∑_1 ≤ i  j ≤ nϑ_1, 2^(2)( U_i,  U_j) = o_L_2(1) ,
 
where o_L_2(1) is a term that converges to zero in L_2, as n →∞. 



  * Next, we show the following about the U-statistics corresponding to the last three terms in  (<ref>): 

    nh_1^(d_Y+d_Z)/2/n(n-1)∑_1≤ i  j ≤ nϑ_1, 4^(2) ( U_i, U_j)    → N(0, σ_2^2 𝒞_2(K)) ,
 
where σ_2^2 and 𝒞_2(K) are as in Lemma <ref>;   
and 

    nh_2^d_Z/2/n(n-1)∑_1≤ i  j ≤ nϑ_1, 3^(2) ( U_i, U_j)    → N(0, τ_3^2),  nh_2^d_Z/2/n(n-1)∑_1≤ i  j ≤ nϑ_1, 5^(2) ( U_i, U_j) → N(0, τ_5^2) ,
 
for some positive constants τ_3^2 and τ_5^2. Here, the convergence in (<ref>) and (<ref>) hold in distribution and in the first 2 moments. 



The proofs of the above results are given below. To complete the proof of Lemma <ref>, observe that (<ref>) implies, since h_2/h_1 → 0, 

    nh_1^(d_Y+d_Z)/2/n(n-1)∑_1 ≤ i  j ≤ nϑ_1, 1^(3)( U_i,  U_j) = o_P(1) ,  nh_1^(d_Y+d_Z)/2/n(n-1)∑_1 ≤ i  j ≤ nϑ_1, 2^(5)( U_i,  U_j) = o_P(1) .
 
Combining this with (<ref>) and (<ref>) the result in Lemma <ref> follows. 

 


  
    Recall the definition of ϑ_1, 1^(2) from (<ref>). Then by Lemma <ref> and Remark <ref>, 
       
    1/n(n-1)∑_1 ≤ i  j ≤ n[ ϑ_1, 1^(2)( U_i,  U_j) ] 
        =  [ϑ_1, 1^(2)( U_1,  U_2) ] 
    
               = 1/h_2^4d_Zh_1^4(d_Y+d_Z)∫[  ∏_s=2^5 w_ Y_s, Z_s( y_1,  z_1) ∏_s=6^9 w_ Z_s( z_1)ϕ'( X_2,  X_3,…,  X_9)] p_ Y,  Z( y_1,  z_1) d y_1 d z_1 
       →∫ p_ Y,  Z (  y,  z)^5  p_ Z ( z)^4[ ϕ'( X_2,  X_3,…,  X_9) |{ Y_s=  y }_2 ≤ s ≤ 5, { Z_s=  z }_2 ≤ s ≤ 9 ]  d y d z +  O(h_1^2 h_2^2)  
        = O(h_1^2 h_2^2) = O(h_1^4) .
 
This means, 

    nh_1^(d_Y+d_Z)/2/n(n-1)∑_1 ≤ i  j ≤ n [ ϑ_1, 1^(2)( U_i,  U_j) ] = O(nh_1^(d_Y+d_Z)/2 +4 ) → 0.
 




Next, we compute the variance of ∑_1 ≤ i  j ≤ nϑ_1, 1^(2)( U_i,  U_j). As before, by Lemma <ref> and Remark <ref>, 

    h_1^(d_Y+d_Z)/n^2∑_1 ≤ i  j ≤ n[ ϑ_1, 1^(2)( U_i,  U_j) ] 
       ≲ h_1^(d_Y+d_Z)[ ϑ_1, 1^(2)( U_1,  U_2) ^2 ] 
        = h_1^(d_Y+d_Z)[[φ_n( U_1,  U_2,  U_3, …,  U_9)][φ_n( U_1,  U_2,  U_3', …,  U_9')]]*(where U_1,…,  U_9, U_3',…,  U_9' are i.i.d.)
        = h_1^(d_Y+d_Z)[φ_n( U_1,  U_2,  U_3, …,  U_9)φ_n( U_1,  U_2,  U_3', …,  U_9')] 
       →{ C_K ∫([ γ'( X_2,  X_3, …,  X_9;  X_3', …,  X_9') ] p_ Z( z)^8 p_ Y, Z( y, z)^7 ) p_ Y, Z( y, z) d y d z  + O(h_1^2 h_2^2) }
        = O(h_1^2 h_2^2) = O(h_1^4) → 0 ,
 
where γ' is the symmetrized version of the function γ : (^d_X)^15→, 

    γ( x_2,  x_3, …,  x_9;  x_3', …,  x_9') :=ϕ^'( x_2,  x_3, …,  x_9) ϕ^'( x_2,  x_3', …,  x_9'),

the expectation is with respect to X_2,…,  X_5,  X_3', …,  X_5' i.i.d. from ℙ_ X |  Y= y,  Z =  z and X_6,…,  X_9,  X_6',…,  X_9' i.i.d. from _ X|  Z = z, and C_K is some constant depending on K. Since ϕ^' is conditionally first order degenerate under H_0, 

    [ γ'( X_2,  X_3, …,  X_9;  X_3', …,  X_9') ] = 0 ,

which explains the last step in (<ref>). Now, we consider the covariance terms, 

    h_1^(d_Y+d_Z)/n^2∑_1 ≤ i  j  k ≤ nCov[ ϑ_1, 1^(2)( U_i,  U_j),  ϑ_1, 1^(2)( U_j,  U_k)  ] 
       ≲ n h_1^(d_Y+d_Z)[ ϑ_1, 1^(2)( U_1,  U_2) ϑ_1, 1^(2)( U_1,  U_3) ] 
        = n h_1^(d_Y+d_Z)[[φ_n( U_1,  U_2,  U_3', …,  U_9')][φ_n( U_1,  U_3,  U_3”, …,  U_9”)]]  *(where U_1,  U_2,  U_3,  U_3', …,  U_9',  U_3”, …,  U_9” are i.i.d.)
        = n h_1^(d_Y+d_Z)[φ_n( U_1,  U_2,  U_3', …,  U_9') φ_n( U_1,  U_3,  U_3”, …,  U_9”) ]  
        = O(nh_1^(d_Y+d_Z)+4) → 0,
 
by Lemma <ref> and Remark <ref>.  Collecting (<ref>) and (<ref>) it follows that

    h_1^(d_Y+d_Z)/n^2[∑_1 ≤ i  j ≤ nϑ_1, 1^(2)( U_i,  U_j) ] → 0 .

 From (<ref>) and (<ref>) the first result in (<ref>) follows. The second result in (<ref>) can be proved similarly. 


   

We will only prove (<ref>). The results in (<ref>) can proved similarly. To begin with recall the definition of ϑ_1, 4^(2) from (<ref>). Then 

    ϑ_1, 4^(2)( u,  u') 
        = [φ_n( U_1,  u,  u',  U_4,  U_5;  U_6, …,   U_9)] 
        = 1/h_1^4(d_Y+d_Z)h_2^4d_Z[ w_ y, z( Y_1, Z_1) w_ y', z'( Y_1, Z_1)∏_s = 4^5 w_ Y_s,  Z_s( Y_1, Z_1) ∏_s = 6^9 w_ Z_s( Z_1) ϕ'( x, x', X_4, …, X_9)] ,
 
       where u = ( x,  y,  z) and u' = ( x',  y',  z'). Then note that, 
       
    [ϑ_1, 4^(2)( U,  U')^2]
        = 1/h_1^8(d_Y+d_Z)h_2^8d_Z
           [ w_ Y, Z( Y_1, Z_1) w_ Y', Z'( Y_1, Z_1) ∏_s = 4^5 w_ Y_s,  Z_s( Y_1, Z_1) ∏_s = 6^9 w_ Z_s( Z_1) ϕ'( X, X', X_4, …, X_9)
                w_ Y, Z(Ỹ_1,Z̃_1) w_ Y', Z'(Ỹ_1,Z̃_1)∏_s = 4^5 w_Ỹ_s, Z̃_s(Ỹ_1,Z̃_1) ∏_s = 6^9 w_Z̃_s(Z̃_1) ϕ'( X, X',X̃_4, …,X̃_9)] ,
 
       where {Ũ_i = (X̃_i, Ỹ_i, , Z̃_i) }_1 ≤ i ≤ 9 are i.i.d. with density p_ X,  Y,  Z which is independent of  { U_i = (  X_i,  Y_i, ,  Z_i) }_1 ≤ i ≤ 9. To compute (<ref>), we first condition on U_1,Ũ_1, U, and U' to get, 
              
    [ϑ_1, 4^(2)( U,  U')^2| U, U',  U_1, Ũ_̃1̃]
        = 1/h_1^8(d_Y+d_Z)h_2^8d_Z w_ Y, Z( Y_1, Z_1) w_ Y', Z'( Y_1, Z_1) w_ Y, Z(Ỹ_1,Z̃_1) w_ Y', Z'(Ỹ_1,Z̃_1) 
       ∫{∏_s = 4^5 w_ y_s,  z_s( Y_1, Z_1) ∏_s = 4^5 w_ỹ_s, z̃_s(Ỹ_1,Z̃_1) ∏_s = 6^9 w_ z_s( Z_1) ∏_s = 6^9 w_z̃_s( Z_1) ϕ'( X, X', x_4, …, x_9) ϕ'( X, X',x̃_4, …,x̃_9) 
             ∏_s=4^5 p_ X,  Y,  Z( x_s,  y_s,  z_s) ∏_s=4^5 p_ X,  Y,  Z(x̃_s, ỹ_s, z̃_s)  ∏_s=6^9 p_ X,  Z(  x_s,  z_s)  ∏_s=6^9 p_ X,  Z(x̃_s, z̃_s) }d𝒱 ,
 
         where d𝒱 := ∏_s=4^9 d x_s ∏_s=4^9 dx̃_s ∏_s=4^5 d y_s ∏_s=4^5 dỹ_s ∏_s=4^9 d z_s ∏_s=4^9 dz̃_s. Now, by the change of variables: 
    y_s =  Y_1 + h_1  u_s,  z_s =  Z_1 + h_1  v_s, ỹ_s = Ỹ_1 + h_1 ũ_s, z̃_s = Z̃_1 + h_1 ṽ_s,
 for s ∈{4, 5}, and 
         
    z_s =  Z_1 + h_2  v_s, z̃_s = Z̃_1 + h_2 ṽ_s,
 
         for s ∈{6, 7, 8, 9} we get, 
              
    [ϑ_1, 4^(2)( U,  U')^2| U, U',  U_1, Ũ_̃1̃] 
        = 1/h_1^4(d_Y+d_Z) w_ Y, Z( Y_1, Z_1) w_ Y', Z'( Y_1, Z_1) w_ Y, Z(Ỹ_1,Z̃_1) w_ Y', Z'(Ỹ_1,Z̃_1) 
       ∫{∏_s = 4^5 K(  (  u_s,  v_s )   ) ∏_s = 4^5 K(  ( ũ_s, ṽ_s )   ) ∏_s = 6^9 K(  v_s  ) ∏_s = 6^9 K( ṽ_s  )  ϕ'( X, X', x_4, …, x_9) ϕ'( X, X',x̃_4, …,x̃_9) 
             ∏_s=4^5 p_ X,  Y,  Z( x_s,  Y_1 + h_1  u_s,  Z_1 + h_1  v_s) ∏_s=4^5 p_ X,  Y,  Z(x̃_s, Ỹ_1 + h_1 ũ_s, Z̃_1 + h_1 ṽ_s) 
             ∏_s=6^9 p_ X,  Z(  x_s,  Z_1 + h _2  u_s)  ∏_s=6^9 p_ X,  Z(x̃_s, Z̃_1 + h _2 ũ_s) }d𝒱 ,
 
 where d𝒱 := ∏_s=4^9 d x_s ∏_s=4^9 dx̃_s ∏_s=4^5 d u_s ∏_s=4^5 dũ_s ∏_s=4^9 d v_s ∏_s=4^9 dṽ_s.      
Denote the integral (<ref>) by γ( x, x^';  y_1, z_1,ỹ_1,z̃_1, h_1,h_2). Then taking expectation on both sides of (<ref>) gives, 
        
    [ϑ_1, 4^(2)( U,  U')^2] 
        = [1/h_1^4(d_Y+d_Z) w_ Y, Z( Y_1, Z_1) w_ Y', Z'( Y_1, Z_1) w_ Y, Z(Ỹ_1,Z̃_1) w_ Y', Z'(Ỹ_1,Z̃_1)γ( X, X^';  Y_1, Z_1,Ỹ_1,Z̃_1,h_1,h_2)]
        = 1/h_1^4(d_Y+d_Z)∫{ K(( y, z)-( y_1, z_1)/h_1) K(( y', z')-( y_1, z_1)/h_1) 
                                K(( y, z)-(ỹ_1,z̃_1)/h_1) K(( y, z)-(ỹ_1,z̃_1)/h_1) γ( x, x^';  y_1, z_1,ỹ_1,z̃_1 h_1,h_2)
                                p_ X,  Y,  Z( x, y, z)p_ X,  Y,  Z( x', y', z')p_ X,  Y,  Z( x_1, y_1, z_1)p_ X,  Y,  Z(x̃_1,ỹ_1,z̃_1) }d𝒯 ,
 
      where d𝒯 = d xd yd zd x'd y'd z'd x_1d y_1d z_1dx̃_1dỹ_1dz̃_1. By the change of variables: 
      ( y_1, z_1) = ( y,  z) + h_1 ( u_1,  v_1) and (ỹ_1, z̃_1) = (ỹ, z̃) + h_1 (ũ_1, ṽ_1) we get, 
      
    [ϑ_1, 4^(2)( U,  U')^2] 
        = 1/h_1^2(d_Y+d_Z)∫{ K(( u_1, v_1)) K(( y', z')-( y, z) - h_1( u_1, v_1)/h_1) 
                K((ṽ_1,ũ_1)) K(( y', z')-( y, z)-h_1(ṽ_1,ũ_1)/h_1) 
               γ( x, x^'; y + h_1 u_1, z+ h_1 v_1, y+h_1ũ_1, z + h_1ṽ_1, h_1,h_2) 
                p_ X,  Y,  Z( x, y, z) p_ X,  Y,  Z( x', y', z') p_ X,  Y,  Z( x_1, y + h_1  v_1, z+h_1 u_1) p_ X,  Y,  Z(x̃_1, y + h_1ṽ_1,  z + h_1ũ_1) }d𝒯̃ ,

      where d𝒯̃ = d xd yd zd x'd y'd z'd x_1d u_1d v_1dx̃_1dũ_1dṽ_1. Then by the change of variables: ( y',  z') = ( y,  z) + h_1 ( u' ,  v') we get, 
     
    [ϑ_1, 4^(2)( U,  U')^2] 
        = 1/h_1^(d_Y+d_Z)∫{ K(( u_1, v_1)) K(( u', v') - ( u_1, v_1))K((ũ_1,ṽ_1)) K(( u', v')-(ũ_1,ṽ_1)) 
               γ( x, x^';  y + h_1 u_1, z+ h_1 v_1, y+h_1ũ_1, z + h_1ṽ_1, h_1,h_2) p_ X,  Y,  Z( x, y, z) 
                p_ X,  Y,  Z( x', y + h_1  v',  z + h_1 u') p_ X,  Y,  Z( x_1, y + h_1  v_1, z+h_1 u_1) p_ X,  Y,  Z(x̃_1, y + h_1ṽ_1,  z + h_1ũ_1) }d𝒯' ,
 
       where d𝒯' = d xd yd zd x'd u'd v'd x_1d u_1d v_1dx̃_1dũ_1dṽ_1. 
                 Now, under Assumption <ref>, it can be shown that as n →∞, 
        
    h_1^d_Y+d_Z[ϑ_1, 4^(2)( U,  U')^2] 
       →𝒞_2(K) ∫[Q_0^2( x, x^') p( z)^4 p( y, z)^2]^2 p( x,  y, z) p( x', y, z) d x d x' d y d z = σ_2^2 𝒞_2(K) ,

        where Q_0 is as defined in Theorem <ref>, and
        
    𝒞_2(K) = ∫[∫ K(( u_1, v_1)) K(( u,  v) - ( u_1, v_1))d u_1d v_1]^2 d u d v.

    
  Now, define     
   
    Γ_n     := 1/n(n-1)∑_1≤ i  j≤ nϑ_1, 4^(2) ( U_i, U_j),

    where 
    
    ϑ_1, 4^(2) ( U_i, U_j) = {ϑ_1, 4^(2)( U_i, U_j)-_ U[ ϑ_1, 4^(2) ( U_i, U)]-_ U[ ϑ_1, 4^(2) ( U, U_j) ]+_ U, U'[ ϑ_1, 4^(2) ( U, U')]} ,

 with U,  U' i.i.d. with density p_ X,  Y,  Z. Note that Γ_n in (<ref>) is a degenerate U-statistics of degree 2. Hence, we can apply Theorem 1 from <cit.> to establish the convergence of Γ_n. 
 
   For this, note from (<ref>) and (<ref>) that 
    
    n^2 h_1^d_Y+d_Z[ Γ_n] = (1+ o(1)) h_1^d_Y+d_Z[ϑ_1, 4^(2) ( U_1, U_2)] = (1+ o(1)) h_1^d_Y+d_Z[ϑ_1, 4^(2)( U_1, U_2)^2]→σ_2^2  𝒞_2(K) .
  
Now, define 
    
    G_n( u, u') := [ϑ_1, 4^(2)( U_1,  u) ϑ_1, 4^(2)( U_1,  u')].

Using similar calculations we can show that h_1^(d_Y+d_Z)[G_n^2( U_1, U_2)] and h_1^3(d_Y+d_Z)[ϑ_1, 4^(2)( U_1,  U_2)^4] converges to a_1 and a_2, respectively, where a_1, a_2>0 are constants.  Therefore,
    
    [G_n^2( U_1, U_2)]/[[ϑ_1, 4^(2)( U_1, U_2)^2]]^2 = h_1^2(d_Y+d_Z)[G_n^2( U_1, U_2)]/[h_1^(d_Y+d_Z)[ϑ_1, 4^(2)( U_1, U_2)^2]]^2 = O(h_1^(d_Y+d_Z))→ 0,

    and 
    
    [ϑ_1, 4^(2)( U_1, U_2)^4]/n[[ϑ_1, 4^(2)( U_1, U_2)^2]]^2 = h_1^2(d_Y+d_Z)[ϑ_1, 4^(2)( U_1, U_2)^4]/n[h_1^(d_Y+d_Z)[ϑ_1, 4^(2)( U_1, U_2)^2]]^2    = h_1^3(d_Y+d_Z)[ϑ_1, 4^(2)( U_1, U_2)^4]/(nh_1^(d_Y+d_Z))[h_1^(d_Y+d_Z)[ϑ_1, 4^(2)( U_1, U_2)^2]]^2
        = O(1/nh_1^(d_Y+d_Z)) → 0,

    as n→∞. Hence, from Theorem 1 from <cit.> we have,
    
    nh_1^(d_Y+d_Z)/2Γ_n D→ N(0, σ_2^2 𝒞_2(K)) .
 
Also, by arguments as in (<ref>), 
    nh_1^(d_Y+d_Z)/21/n∑_i=1^n _ U[ϑ_1, 4^(2) (  U_i, U)] P→ 0 and nh_1^(d_Y+d_Z)/2_ U_1, U[ϑ_1, 4^(2) (  U_1, U)] → 0.
 Hence, the result in (<ref>) follows. 
 

With (<ref>), (<ref>), and (<ref>) proved, the proof of Lemma <ref> is complete. 
    


To complete the proof of Theorem <ref> (3), we need to show that 

    nh_1^(d_Y+d_Z)/2Δ^(c)P→ 0 ,
 
for c ≥ 3. Towards this, as in the proof Lemma <ref>, it can shown that, under H_0, [φ_n^(c)( U_1,…,  U_c)] = O(h_1^4), for c ≥ 1. 
 Hence, recalling (<ref>), 

    n h_1^(d_Y+d_Z)/2[Δ_n^(c)] =  O(n h_1^(d_Y+d_Z)/2 + 4) → 0.
  
  Moreover, by (<ref>) we have 

    [Δ_n^(c)] = 1/n c∑_a=1^c (-1)^c-ac a[φ_n^(a)( U_1,…,  U_a)] = O(1/n^c h_1^(c-1)(d_Y+d_Z)) .
 
Hence, 

    n^2 h_1^(d_Y+d_Z)[Δ_n^(c)] = O(1/n^c-2 h_1^(c-2)(d_Y+d_Z)) → 0 ,

for c ≥ 3. The result in (<ref>) follows from (<ref>) and (<ref>). Combining Lemma <ref>, Lemma <ref>, and (<ref>) completes the proof of Theorem <ref> (3).   




§ PROOFS FROM SECTION <REF>

 


In this section we present the proofs of the results from Section <ref>. The proof of Proposition <ref> 
is given in Section <ref> and the proof of Proposition <ref> is given in Section <ref>. In Section <ref> we prove Theorem <ref>.  





 §.§  Proof of Proposition <ref> 



    Note that by construction the observed dataset 𝒟_0 and the resampled datasets 𝒟_1',𝒟_2',…, 𝒟_M' are exchangeable under H_0. Therefore, ζ̂_n(𝒟_0) and {ζ̂_n(𝒟_i')}_1 ≤ i ≤ M are exchangeable under H_0. Using this exchangeability property we have the following results: 
    
        
  (i) Note that (M+1)p_CRT = ∑_i=1^M 1{ζ̂_n(𝒟_0)> ζ̂_n(𝒟_i')}+1 is the rank of ζ̂_n(𝒟_0) among the observations {ζ̂_n(𝒟_0),ζ̂_n(𝒟_1'),…, ζ̂_n(𝒟_M')}. Due to exchangeability (M+1)p_CRT is uniformly distributed over {1,2,…, M+1}. Therefore,
        
    (p_CRT≤α)     = ((M+1)p_CRT≤ (M+1)α) = ⌊ (M+1)α⌋/M+1.
 
        Hence, our test controls the Type I error uniformly under the CRT framework. 

        
  (ii)  Recall that, by construction, X_i'  Y_i| Z_i, for all 1 ≤ i ≤ n. Hence, by Theorem <ref>, ζ̂_n(𝒟_1') P→ 0, as n →∞. Theorem <ref> also tells us that under H_1, ζ̂_n(𝒟_0) P→ζ( X,  Y |  Z)>0. Therefore, 1{ζ̂_n(𝒟_i')>ζ̂_n(𝒟_0)}→ 0, as n →∞, for each 1 ≤ i ≤ M. Hence, for any M ≥ 1 fixed, 
           
    p_CRT = 1+∑_i=1^M 1{ζ̂_n(𝒟_0)>ζ̂_n(𝒟_i')}/1+MP→1/1+M,

        as n →∞. Therefore, if α>1/(M+1),  (p_CRT≤α) → 1 as n →∞. 
        
    




 §.§  Proof of Proposition <ref> 
 



    Here also under H_0, the observed data 𝒟_0 and 𝒟^π_1,𝒟^π_2,…, 𝒟^π_M are exchangeable (see <cit.>). Hence, ζ̂_n(𝒟_0) and {ζ̂_n(𝒟^π_j)}_1 ≤ j ≤ M are exchangeable. Now, using similar arguments as in the proof of Proposition <ref> we have our desired result.    





 §.§  Proof of Theorem <ref> 



    Let us define 𝒮 := {( Y_1, Z_1),( Y_2, Z_2),…, ( Y_n, Z_n)}, 𝒮_ Z := { Z_1, Z_2,…, Z_n}, 𝒮_ X := { X_1, X_2,…, X_n}, and 𝒮_ X' := { X_1', X_2',…, X_n'}. 
    
     Suppose the assumptions of Theorem <ref> hold. Then under H_0, 
        
    TV(ℒ(ζ̂_n(𝒟̂_1)|𝒮),ℒ(ζ̂_n(𝒟_0)|𝒮)) P→ 0 .

    
    
    
    Then by the definition of the total variation distance (denoted as TV) for any measurable function f we have,
    
    TV(ℒ(f( X_1,…, X_n)),ℒ(f( X_1',…,  X_n')))
        = sup_A∈ℬ()|(f( X_1,…, X_n) ∈ A)-(f( X_1',…,  X_n')∈ A)|
        = sup_A∈ℬ()|(( X_1,…, X_n) ∈ f^-1 (A))-(( X_1',…,  X_n')∈ f^-1(A))|
       ≤sup_A∈ℬ(^n)|(( X_1,…, X_n) ∈ A)-(( X_1',…,  X_n')∈ A)|
        = TV(( X_1,…,  X_n),( X_n',…,  X_n' )),

    where ℒ( V), for any random variable V, denotes the distribution of V.   
    Using (<ref>) we have
    
    TV(ℒ(ζ̂_n(𝒟̂_1)|𝒮),ℒ(ζ̂_n(𝒟_0)|𝒮))≤TV(ℒ(𝒮_ X'|𝒮),ℒ(𝒮_ X|𝒮)).

    However, under H_0:  X Y |  Z we have, ℒ(𝒮_ X|𝒮) = ℒ(𝒮_ X|𝒮_ Z) with density ∏_i=1^n p( x_i|  Z_i). Also, by the construction of the resampled data ℒ(𝒮_ X'|𝒮) = ℒ(𝒮_ X'|𝒮_ Z) with density [∏_i=1^n p̂( x_i|  Z_i)|  Z_1, …, Z_n] where p̂( x| z) is as in (<ref>). Therefore, under H_0, 
    
    TV(ℒ(ζ̂_n(𝒟̂_1)|𝒮),ℒ(ζ̂_n(𝒟_0)|𝒮)) 
       ≤TV(ℒ(𝒮_ X'|𝒮),ℒ(𝒮_ X|𝒮))
        = TV(ℒ(𝒮_ X'|𝒮_ Z),ℒ(𝒮_ X|𝒮_ Z))
        = 1/2∫|[∏_i=1^n p̂( x_i|  Z_i)|  Z_1, Z_2,…, Z_n] - ∏_i=1^n p( x_i|  Z_i)| ∏_i=1^n d x_i.

    Note that conditioned on Z_1, Z_2,…, Z_n we have,
    
    [∏_i=1^n p̂( x_i|  Z_i) |  Z_1,…,  Z_n] = ∑_ i∈ [n]^n∏_j=1^n γ_i_j(  Z_j )  [ ∏_j=1^n 1/h_0^d_Xϕ( x_j,  X_i_j, h_0  I_d_X)|  Z_1,…,  Z_n] .

    Observe that 
    [ ∏_j=1^n 1/h_0^d_Xϕ( x_j,  X_i_j, h_0  I_d_X)|  Z_1, Z_2,…, Z_n]
 is itself a valid density function, for any i = (i_1,…, i_n) ∈ [n]^n. 
   Using ∑_ i∈ [n]^n∏_j=1^n γ_i_j(  Z_j ) = ∏_j=1^n ∑_i=1^n γ_i( Z_j) = 1 gives, 
    
    ∫|[∏_i=1^n p̂( x_i|  Z_i)|  Z_1,…, Z_n ] - ∏_i=1^n p( x_i|  Z_i) | ∏_i=1^n d x_i
        = ∫|∑_ i ∈ [n]^n ∏_j=1^n γ_i_j(  Z_j )  [ ∏_j=1^n 1/h_0^d_Xϕ( x_j, X_i_j, h_0  I_d_X)|  Z_1,…,  Z_n]- ∏_i=1^n p( x_i|  Z_i) | ∏_i=1^n d x_i
       ≤∏_i=1^n γ_i( Z_i) ∫| [ ∏_i=1^n 1/h_0^d_Xϕ( x_i,  X_i, h_0  I_d_X)|  Z_1,…, Z_n]-∏_i=1^n p( x_i|  Z_i) |∏_i=1^n d x_i + U_n,

     where 
    
    U_n = ∑_π: [n] → [n] 
    π≠id∏_i=1^n γ_π(i)(  Z_i ) ∫| [ ∏_i=1^n 1/h_0^d_Xϕ( x_i,  X_π(i), h_0  I_d_X)|  Z_1,…, Z_n]-∏_i=1^n p( x_i|  Z_i)|∏_i=1^n d x_i ,
 
     with id: [n] → [n] denoting the identity map, that is, id(i) = i, for 1 ≤ i ≤ n.   

      For any π: [n] → [n] with π≠id note that,
     
    ∫| [ ∏_i=1^n 1/h_0^d_Xϕ( x_i,  X_π(i), h_0  I_d_X)|  Z_1,…, Z_n]-∏_i=1^n p( x_i|  Z_i) | ∏_i=1^n d x_i 
        = 2TV(ℒ( V_1,…, V_n), ℒ( X_1,…,  X_n)),

     for some random vector ( V_1,…,  V_n) having joint density function [ ∏_i=1^n 1/h_0^d_Xϕ( x_i,  X_π(i), h_0  I_d_X)|  Z_1,…, Z_n]. Bounding the TV distance by 1, we then have     
     
    U_n = O_P (  ∑_π: [n] → [n] 
    π≠id∏_i=1^n γ_π(i)(  Z_i )  ) = o_P(1) ,

         as n →∞, since ∏_s=1^n γ_s( Z_s) P→ 1.  
     Also, note that
     
    [ ∏_i=1^n 1/h_0^d_Xϕ( x_i,  X_i, h_0  I_d_X)|  Z_1,…, Z_n]     = ∫∏_i=1^n 1/h_0^d_Xϕ( x_i- w_i/h_0) ∏_i=1^n p( w_i|  Z_i) ∏_i=1^n d w_i
        = ∫∏_i=1^n ϕ( a_i) ∏_i=1^n p( x_i + h_0 a_i|  Z_i) ∏_i=1^n d a_i 
       = ∏_i=1^n ( ∫ϕ( a) p( x_i + h_0 a|  Z_i)  d a ) .

     Hence, by a telescoping argument, 
     
    A_n :=    ∫|[ ∏_i=1^n 1/h_0^d_Xϕ( x_i,  X_i, h_0 I_d_X)|  Z_1,…, Z_n] - ∏_i=1^n p( x_i|  Z_i)| ∏_i=1^n d x_i 
        = ∫| ∏_i=1^n ( ∫ϕ( a) p( x_i + h_0 a|  Z_i)  d a ) - ∏_i=1^n p( x_i|  Z_i)| ∏_i=1^n d x_i  
       ≤∑_i=1^n |∫ϕ( a) p( x_i + h_0 a |  Z_i) d a - p( x_i |  Z_i)| d x_i .
 
            Taking expectation with respect to Z_1,…, Z_n gives,          
            
    [A_n]      = n  [ |∫ϕ( a) p( x + h_0 a |  Z_1) d a - p( x |  Z_1)| d x ] 
        = n ∫|∫ϕ( a) p_ X, Z( x + h_0 a, z)/p_ Z( z)d a - p_ X, Z( x,  z)/p_ Z( z )| p_ Z( z)d x d z 
        = n ∫|∫ϕ( a) p_ X, Z( x + h_0 a, z) d a - p_ X, Z( x,  z)| d x d z 
        = O(nh_0^2) ,
 
            since p_ X, Z is nice in X. This implies, A_n P→ 0.  
     Now combining (<ref>),(<ref>),(<ref>), and (<ref>) the result in Lemma <ref> follows. 
         
    
    
     
     Now we prove the desired properties of the Type I error and power of the local wild bootstrap test ϕ_LWB = 1{ p_LWB<α}. 

    
        
  (a) Before proving the result in Theorem <ref> (a) let us first note that for two random variables T and W, 
        
    (T>x) = ∫_{ t>x}dℒ(T)(t)     = ∫_{t>x}d(ℒ(W) + ℒ(T)-ℒ(W))(t)
        = ∫_{t>x}dℒ(W)(t) + ∫_{t>x}d(ℒ(T)-ℒ(W))(t)
       ≤(W>x) + sup_A | ℒ(T)(A)-ℒ(W)(A)|
        = (W>x) + TV(ℒ(T), ℒ(W)),

        for any x∈. Using (<ref>) for ζ_n(𝒟̂_1) and ζ_n(𝒟_0) conditioning on 𝒮 we have,
        
    (ζ̂_n(𝒟_0)>x|𝒮)    ≤(ζ̂_n(𝒟̂_1)>x |𝒮) + TV(ℒ(ζ̂_n(𝒟̂_1)|𝒮),ℒ(ζ̂_n(𝒟_0)|𝒮)) .

       Taking expectation on both sides, applying Lemma <ref> and the Dominated Convergence Theorem gives       
       
    lim sup_n→∞(ζ̂_n(𝒟_0)>x)    ≤lim sup_n→∞(ζ̂_n(𝒟̂_1)>x) ,
 
   for any x∈. Now, define c_n^*(α) as the upper α-th quantile of ζ̂_n(𝒟̂_1) conditioned on 𝒟_0, that is, (ζ̂_n(𝒟̂_1) >c_n^*(α)|𝒟_0) ≤α. Also, define p^*:=(ζ̂_n(𝒟̂_1)≥ζ̂_n(𝒟_0) |𝒟_0). Let
        
    F(t) = {ζ̂_n(𝒟̂_1)≤ t|𝒟_0}   and    F_M(t) = 1/M{∑_i=1^M 1{ζ̂_n(𝒟̂_i)≤ t }}.

        Note that F and F_M are distribution functions conditioned on the observed data 𝒟_0. Then,
        
    |p^*-p_LWB|     = |(ζ̂_n(𝒟̂_1)≥ζ̂_n(𝒟_0)|𝒟_0) -1/M+1{∑_i=1^M 1{ζ̂_n(𝒟̂_i)≥ζ̂_n(𝒟_0) } + 1}|
        = |(ζ̂_n(𝒟̂)< ζ̂_n(𝒟_0)|𝒟_0) - 1/M+1{∑_i=1^M 1{ζ̂_n(𝒟̂_i)< ζ̂_n(𝒟_0)}}|
         = |F(ζ̂_n(𝒟_0)-)-M/M+1F_M(ζ̂_n(𝒟_0)-)|
       ≤|F(ζ̂_n(𝒟_0)-)-F_M(ζ̂_n(𝒟_0)-)| + |F_M(ζ̂_n(𝒟_0)-)/M+1|
       ≤sup_t∈|F(t)-F_M(t)|+1/M+1.
        
        Using the Dvoretzky-Keifer-Wolfwitz inequality now gives, 
        
    (|p_LWB-p^*|>ε|𝒟_0) ≤ 2exp{-2M(ε-1/M+1)^2 },

        for any ε>1/(M+1). Now, fix a large M and take any ε>1/(M+1). Then,
        
    (p_LWB≤α|𝒟_0)     = (p_LWB≤α, |p_LWB-p^*|>ε|𝒟_0) + (p_LWB≤α, |p_LWB-p^*|≤ε|𝒟_0) 
       ≤(|p_LWB-p^*|>ε|𝒟_0) +  1{p^*≤α+ε}
       ≤ 2exp{-2M(ε-1/M+1)^2} +  1{ζ_n(𝒟_0)>c_n^*(α+ε)} .

        Taking expectations on both sides gives, 
        
    (p_LWB≤α) ≤ 2exp{-2M(ε-1/M+1)^2} + (ζ_n(𝒟_0)>c_n^*(α+ε)).
  
        Now, taking limit as n→∞ and M→∞ we get,
        
    lim sup_n,M→∞(p_LWB≤α)    ≤lim sup_n →∞(ζ̂_n(𝒟_0)>c_n^*(α+ε) ) 
       ≤lim sup_n →∞(ζ̂_n(𝒟̂_1) >c_n^*(α+ε)) *(using (<ref>))
        = lim sup_n →∞[(ζ̂_n(𝒟̂_1) >c_n^*(α+ε)|𝒟_0) ]
       ≤lim sup_n →∞[α + ε]*(by the definition of c_n^*(α+ε))
         = α +  ε.

        Hence, since ε is arbitrary, under H_0, lim sup_n,M→∞_H_0(p_LWB≤α ) ≤α. This completes the proof of the first part of Theorem <ref>.   
        
        
  (b) Under H_1: X Y |  Z, using similar arguments as in the proof of  Lemma <ref> we can show that 
      
    TV(ℒ(ζ̂_n(𝒟̂_1)|𝒮),ℒ(ζ̂_n(𝒟_1')|𝒮))P→ 0,

     where 𝒟_1' is the resampled data obtained in the conditional randomization test. Therefore, even under H_1, ζ̂_n(𝒟̂_1) P→ 0, as n→∞, by Theorem <ref>. Moreover, ζ̂_n(𝒟_0) converges to ζ( X, Y |  Z) which is positive under H_1. Therefore, under H_1, p_LWB converges in probability to 1/(M+1), as n →∞. Hence, if α>1/(M+1) or M>(1-α)/α, lim sup_n→∞_H_1(p_LWB≤α ) =1, that is, the local wild bootstrap test is consistent. 
    

    



 §.§ Verifying the Weight Condition
 


Let γ_s( z) = κ_s( z) as defined in (<ref>). In the following lemma shows how to choose the bandwidth h_2' such that the condition ∏_s=1^n γ_s( Z_s)P→ 1 holds. 

 
Let γ_s( z) = κ_s( z) as defined in (<ref>). Also, suppose that the Assumptions <ref>, <ref>, <ref> hold. If h_2^'  = o(n^-2/d_Z) we have,
    
    ∏_s=1^n γ_s( Z_s)P→ 1 ,

    as n→∞. 
    


 First, recalling (<ref>), note that 
{γ_1( Z_1)<1 } implies that there exists Z_j, with j ≠1, such that K( Z_1- Z_j/h_2^')>0. Now, since the kernel K has compact support (by Assumption <ref>), this implies Z_j- Z_1< Ch_2^' for some constant  C>0. Therefore, by the union bound,
    
    (γ_1( Z_1)<1)    ≤∑_j=2^n ( Z_j- Z_1< Ch_2^')
        = (n-1) ( Z_2- Z_1< Ch_2^')  *(since Z_2, …,  Z_n are i.i.d.)
        = (n-1) ∫_ z_2- z_1< Ch_2^' p_ Z( z_1) p_ Z( z_2) d z_1 d z_2
       ≲  (n-1) ∫Vol(B( z_1, Ch_2^')) p_ Z( z_1) d z_1 *(since p_ Z is bounded by Assumption <ref>)
       ≲ (n-1)(h_2^')^d_Z.

    Here, Vol(B( z_1, Ch_2^')) denotes the volume of the ball of radius Ch_2^' which is centered at z_1 and the constants in the ≲ bounds depend on d_Z. Now, fix any arbitrary ϵ>0 and note that,
    
    (∏_s=1^n γ_s( Z_s)<1-ϵ)    ≤∑_s=1^n (γ_s( Z_s)<(1-ϵ)^1/n)
        = n (γ_1( Z_1)<(1-ϵ)^1/n) 
    
               ≤ n (γ_1( Z_1)<1) 
       ≲  n(n-1)(h_2^')^d_Z
       → 0 ,
 
    since h_2^' = o(n^-2/d_Z). This completes the proof. 






§ PROOFS FROM SECTION <REF> 
 


This section is organized as follows: In Appendix <ref> we collect the proofs of some technical lemmas which will be used in the analysis of the conditional empirical process. The proof of Proposition <ref> is given in Section <ref>. Proposition <ref> is proved in Section <ref>. 




 §.§ Auxiliary Lemmas






We begin with the following result: 

 Suppose ( X_1,  Z_1), ( X_2,  Z_2), …, ( X_r,  Z_r) are i.i.d. samples from p_ X,  Z. Further, suppose the following conditions hold:   


  * p_ X, Z satisfies assumption <ref>; 


  * the kernel K satisfies assumption <ref>; 


  * h_2 is a bandwidth satisfying the conditions in assumption <ref>. 


Fix an integer r ≥ 1. Then for a measurable, symmetric function ψ: (ℝ^d_X)^r →ℝ such that 
    𝔼[ψ^2(  X_1,  X_2, …,  X_r ) | { Z_s =  z}_1 ≤ s ≤ r ] < ∞,
 we have 

    lim_h_2→ 01/h_2^ r d_Z 𝔼[  ∏_s=1^r  K ( z -  Z_s /h_2) ψ( X_1, …,  X_r)  ] = p_ Z ( z)^r [ψ( X_1, …,  X_r)| { Z_s =  z}_1 ≤ s ≤ r].

Further, if

    sup_ z∈^d_Z𝔼[ψ^2(  X_1,  X_2, …,  X_r ) | { Z_s =  z}_1 ≤ s ≤ r ] < ∞,
 holds, we have,   

    lim_h_2→ 01/h_2^ r d_Z ∫𝔼[  ∏_s=1^r  K( z -  Z_s /h_2) ψ( X_1, …,  X_r)  ] p_ Z( z)d z 
        = ∫ p_ Z ( z)^r+1[ψ( X_1, …,  X_r)| { Z_s =  z}_1 ≤ s ≤ r] d z.
 

 

 
Denote X = ( X_1, …,  X_r) and x = ( x_1, …,  x_r). Then observe that  

    1/h_2^ r d_Z 𝔼[ ∏_s=1^r K( z -  Z_s /h_2) ψ(  X ) ]     = 1/h_2^ r d_Z ∫∫∏_s=1^r K( z - z̃_s /h_2) ψ( x) ∏_s=1^r p_ X,  Z ( x_s, z̃_s) d x_s dz̃_s 
        = ∫∫∏_s=1^r  K(  z_s ) ψ(  x) ∏_s=1^r p_ X,  Z (  x_s,  z + h_2 z_s) d x_s d z_s ,
 
with the change of variable z̃_s =  z+ h_2 z_s, for 1 ≤ s ≤ r. Denote 
    p^(r)(  x,  z )  := ∏_s=1^r p_ X,  Z ( x_s,  z)   and  p^(r)(   x,  z + h  z  )   := ∏_s=1^r p_ X,  Z ( x_s,  z + h_2 z_s) ,
 
where z = ( z_1, …,  z_r). Also, define Q( z) := ∏_s=1^r K( z_s) and 

    R( x_s, z; z_s, h_2) = 1/h_2^2{p_ X, Z( x_s, z + h_2 z_s)/p_ X, Z( x_s, z) -1 - h_2  z_s  ∂/∂ zp_ X, Z( x_s, z)/p_ X, Z( x_s, z)}.

Then, using the identity, 

    p_ X, Z( x_s,  z + h_2 z_s) = p_ X, Z( x_s,  z ) + h_2 z_s  ∂/∂ zp_ X, Z( x_s, z) + h_2^2 p_ X, Z( x_s, z) R( x_s, z;  z_s, h_2) ,
 
and noting that 
∫∫ K ( z_s) 
 z_s  ∂/∂ zp_ X, Z( x_s, z) d z_s = 0, 
since the kernel is symmetric about zero by Assumption <ref>, we have  

    A_n^ z    := ∫∫ Q ( z) ψ( x) p^(r)(   x,  z + h  z  )   ∏_s=1^r d x_s d z_s - ∫∫ Q( z) ψ(  x) p^(r)(  x,  z )  ∏_s=1^r d x_s d z_s  
        = ∑_k=1^r h_2^2 k∑_S⊆ [r]:
     |S| = k A_n^ z(S),
 
        where 
        
    |A_n^ z(S)|     := |∫∫ Q ( z) ψ( x) ∏_s∈ S^c p_ X,  Z ( x_s,  z) ∏_s∈ S( R( x_s, z; z_s, h_2) p_ X, Z( x_s, z)) ∏_s=1^r d x_s d z_s| 
       ≤∫ Q ( z) ( ∫|ψ( x)| ∏_s∈ S^c p_ X,  Z ( x_s,  z) ∏_s∈ S( |R( x_s, z; z_s, h_2)| p_ X, Z( x_s, z)) ∏_s=1^r d x_s ) ∏_s=1^r  d z_s 
       ≤∫ Q ( z)  ( ∫ψ( x)^2 p^(r)(  x,  z ) ∏_s=1^r d x_s )^1/2( ∫∏_s∈ S R( x_s, z; z_s, h_2)^2 p^(r)(  x,  z )  ∏_s=1^r d x_s )^1/2∏_s=1^r d z_s .
 
        Now, note that 
        
    lim sup_n→∞∫∏_s∈ S R( x_s, z; z_s, h_2)^2 p^(r)(  x,  z )  ∏_s=1^r d x_s
         = lim sup_n→∞∫∏_s∈ S( z_s^2 R( x_s, z; z_s/ z_s, h_2 z_s)^2 p^(r)(  x,  z) )  ∏_s=1^r d x_s*(by (<ref>))
       ≤(p( z))^r-|S|∏_s∈ S z_s^4  ( sup_ u: u=1lim sup_n→∞∫ R( x, z; u, h_2 z_s)^2 p( x,  z )  d x)^|S|≲∏_s∈ S z_s^4.

        Also, by Assumption <ref>, 
    ∫ Q( z) ∏_s∈ S z_s^2 ∏_s=1^r d z_s = (∏_s∈ S^c∫ K(  z_s  ) d z_s)( ∏_s∈ S∫ z_s^2 K(  z_s  ) d z_s) = (d_Zτ)^|S|.
 
                Hence, by Assumption <ref>, the finiteness of [ψ^2( X) | { Z_s =  z}_1 ≤ s ≤ r ]<∞, we have, 

    |A_n^ z(S)|≲ p_ Z( z)^r/2∫ Q ( z) ∏_s∈ S z_s^2 ∏_s=1^r  d z_s ≲ 1.
 
Hence, from (<ref>), we get A_n=O(h_2^2). Therefore, from (<ref>),  

    1/h_2^ r d_Z 𝔼[ ∏_s=1^r K( z -  Z_s /h_2) ψ(  X ) ]     = ∫∫∏_s=1^r  K(  z_s ) ψ(  x) ∏_s=1^r p_ X,  Z (  x_s,  z) d x_s d z_s + O(h_2^2) 
        =  ∫ψ(  x) ∏_s=1^r p_ X,  Z (  x_s,  z ) d x_s  + O(h_2^2)  
        = p_ Z ( z)^r [ψ( X_1, …,  X_r)| { Z_s =  z}_1 ≤ s ≤ r] + O(h_2^2) .
 
This completes the proof of (<ref>).  



To show (<ref>), note from the arguments in (<ref>) and (<ref>) and the assumption (<ref>) we get ∫ A_n^ z p_ Z( z) d z = O(h_2^2). The result in (<ref>) then follows by taking expectation with respect to Z on both sides of (<ref>). 



    Note that for Lemma <ref> to hold it is sufficient that 𝒞_K:= ∫ K( u)d u <∞. In this case, from the proof of Lemma <ref> it follows that
       
    lim_h_2→ 01/h_2^ r d_Z 𝔼[  ∏_s=1^r  K ( z -  Z_s /h_2) ψ( X_1, …,  X_r)  ] = 𝒞_K^r p_ Z ( z)^r [ψ( X_1, …,  X_r) | { Z_s =  z}_1 ≤ s ≤ r].
 


Using the above lemma we can derive the limit of the expectation of a homogeneous sum of r-fold kernel products. 




  
Suppose the assumptions the Lemma <ref> hold. Then 

    lim_n →∞1/(nh_2^ d_Z )^r 𝔼[ ∑_ i∈ [n]^r∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ] = p_ Z ( z)^r [ψ( X_1, …,  X_r) | { Z_s =  z}_1 ≤ s ≤ r],

where i = (i_1,i_2,…, i_r) and X_ i = ( X_i_1, X_i_2,…,  X_i_r).  
Moreover,  

    lim_n →∞1/(nh_2^ d_Z )^r ∫𝔼[ ∑_i_1,…, i_r =1^n  ∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_i_1, …,  X_i_r)  ] p_ Z( z) d z 
        = ∫ p_ Z ( z)^r+1[ψ( X_1, …,  X_r) | { Z_s =  z}_1 ≤ s ≤ r] d z.
 
 

 Denote by 
i the distinct elements of i ∈ [n]^r. Now,
    
    1/(nh_2^ d_Z )^r 𝔼[ ∑_ i∈ [n]^r∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ] 
        = 1/(nh_2^ d_Z )^r 𝔼[ ∑_k=1^r ∑_ i ∈ [n]^r: | i| =  k∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ]
        = 1/(nh_2^ d_Z )^r ∑_k=1^r ∑_ i ∈ [n]^r: | i| = k𝔼[ ∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ]
        = 1/(nh_2^ d_Z )^r ∑_k=1^r ∑_ i ∈ [n]^r: | i| = k𝔼[ ∏_ j ∈ iK_j ( z -  Z_j/h_2) ψ( X_ℓ_ i)  ] ,
   
       where for a given i∈ [n]^r, 
       
    K_j( z -  Z_j/h_2) := ∏_s: i_s = j K( z -  Z_j/h_2)
 
       and ℓ_ i=⋃_j=1^n{s: i_s = j}∈ [n]^r is the vector obtained after sorting i. Now, fix 1 ≤ k ≤ r and consider, 
      
    T_k, n    := ∑_ i ∈ [n]^r: | i| = k𝔼[ ∏_ j ∈ iK_j ( z -  Z_j/h_2) ψ( X_ℓ_ i)  ]    
        = h_2^k d_Z∑_ i ∈ [n]^r: | i| = k∫∏_j=1^kK_j (  u_j ) ψ(  x_ℓ_ i) ∏_j=1^k p_ X, Z( x_j,  z + h_2 u_j) ∏_j=1^kd x_j d u_j 
       * (after the substitution z_j =  z + h_2 u_j, for 1 ≤ j ≤ k) 
         = O(n^k h_2^k d_Z) ,

 by arguments as in Lemma <ref>. This implies, 
   
    1/(nh_2^ d_Z )^r 𝔼[ ∑_ i∈ [n]^r∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ]  
        =  T_r, n/(nh_2^ d_Z )^r  +  O(1/nh_2^ d_Z ) * (by (<ref>) and (<ref>)) 
        = 1/(nh_2^ d_Z )^r   (n)_r h_2^r d_Z∫∏_j=1^r  K(  u_j ) ψ( x_1, …,  x_r) ∏_j=1^r p_ X, Z( x_j, z + h_2 u_j) ∏_j=1^r d x_j d u_j + O(1/nh_2^ d_Z ) * (where (n)_r := n(n-1) ⋯ (n-r+1)) 
       →∫∏_j=1^r  K( u_j) ψ( x_1, …,  x_r) ∏_j=1^r p_ X, Z( x_j,  z) ∏_j=1^r d x_j d u_j
           
           *(by (<ref>) and (<ref>))
        = p_ Z( z)^r [ψ( X_1, …,  X_r)| { Z_s =  z }_1 ≤ s ≤ r ] .

    This proves (<ref>). 


The result in (<ref>) can be proved similarly, by repeating the previous arguments and using (<ref>). 





Note that the limits obtained in Lemma <ref> is zero, when the function ψ:(^d_X)^r→ is completely degenerate, that is, 

    [ψ( x_1,…, x_r-1, X_r)|  Z_r =  z] = 0,
 
almost surely for x_1,…, x_r-1∈^d_X and z ∈^d_Z. In this case one has to normalize the r-fold kernel product differently to obtain a non-trivial limit. 




 
Suppose the assumptions of Lemma <ref> hold. Further, assume that the function ψ:(^d_X)^r→ is completely degenerate as defined in (<ref>). Then the following hold: 
    
    lim_n →∞1/(nh_2^ d_Z )^r/2𝔼[ ∑_ i ∈ [n]^r∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ]
        = 
    {[ (∫ K^2( u)d u)^r/2(⌊r/2⌋) ! ∫ψ( x_1, x_1, …,  x_⌊ r/2⌋, x_⌊ r/2⌋) ∏_s=1^⌊ r/2⌋ p_ X, Z( x_s,  z) ∏_s=1^⌊ r/2 ⌋d x_s,                                                                                                      if  r  is even,;                                                                                                                    0                                                                                                       if  r  is odd. ].

        



As in the proof of Lemma <ref>, denote by 
i the distinct elements of i ∈ [n]^r. Then from (<ref>) we have, 
    
    𝔼[ ∑_ i∈ [n]^r∏_s=1^r  K( z -  Z_i_s/h_2) ψ( X_ i)  ]      =   ∑_k=1^r T_k, n,
  
   with T_k, n as defined in (<ref>). 
For any k < ⌊ r/2 ⌋, by (<ref>), 
    
    lim_n→∞T_k, n/(nh_2^ d_Z )^r/2≲lim_n→∞(nh_2^ d_Z )^k/(nh_2^ d_Z )^r/2 = 0.

Therefore, we only need to consider the case k≥⌊ r/2 ⌋. Towards this, suppose i ∈ [n]^r such that | i| = k >⌊ r/2 ⌋. Then by the pigeonhole principle, there exists an index  which occurs exactly once in i. Let ν_1( i) be the number of indices in i that repeats exactly once and ν_2( i) are those that repeats multiple times in i. Clearly, k = ν_1( i) + ν_2( i), for i ∈ [n]^r such that | i| = k. Now, recalling (<ref>), let  

    T_k, n = ∑_ i ∈ [n]^r: | i| = k𝔼[ ∏_ j ∈ iK_j ( z -  Z_j/h_2) ψ( X_ℓ_ i)  ] = ∑_0 ≤ a , b ≤ k 
      a + b = k T_a, b, n,
 
   where 
   
    T_a, b, n := ∑_ i ∈ [n]^r 
    ν_1( i) = a, ν_2( i) = b  𝔼[ ∏_ j ∈ iK_j ( z -  Z_j/h_2) ψ( X_ℓ_ i)  ] .
 Then, by the degeneracy of ψ all terms in the expansion (<ref>) with |S| ≤ a-1 are zero. Hence, T_a, b, n = O((nh_2^ d_Z )^a + b h_2^2 a). Also, note that r ≥ a+2b, hence a +b -r/2 ≤ a/2. 
   
  This implies, for k >⌊ r/2 ⌋, 
   
    T_a, b, n/(nh_2^ d_Z  )^r/2 = O((nh_2^ d_Z )^a +b -r/2h_2^2 a) = O((nh_2^d_Z+4)^a/2) → 0  ,
 
    since a ≥ 1 when k >⌊ r/2 ⌋ and nh_2^d_Z+4→ 0 (by Assumption <ref>). Hence, for k > ⌊ r/2 ⌋, by (<ref>), 
    
    lim_n→∞T_k, n/(nh_2^ d_Z )^r/2  = 0.
 

   
    Now, let us consider the case k=⌊ r/2 ⌋. If r is odd, then by arguments as in (<ref>), 
    
    lim_n→∞T_⌊ r/2 ⌋, n/(nh_2^ d_Z )^r/2 = lim_n →∞ O((nh_2^d_Z)^⌊ r/2⌋-r/2) = 0.

    Therefore, suppose r is even and k=⌊ r/2 ⌋. In this case, if a ≥ 1, then from (<ref>) we get, 
            
    lim_n →∞T_a, b, n/(nh_2^ d_Z )^r/2  =  lim_n →∞ O((nh_2^d_Z + 4)^a/2) = 0 ,

     Hence, recalling (<ref>)
     
    T_⌊ r/2 ⌋, n/(nh_2^ d_Z )^r/2 = T_0, ⌊ r/2 ⌋, n/(nh_2^ d_Z )^r/2 + o(1).

     Also, by the change of variable z_s =  z + h_2  u_s, 
            
    T_0, ⌊ r/2 ⌋, n/(nh_2^ d_Z )^r/2
        = n⌊ r/2⌋/n^⌊ r/2⌋( ⌊r/2⌋)! ∫∏_s=1^r/2 K ( u_s )^2 ψ( x_1, x_1, …,  x_⌊ r/2⌋, x_⌊ r/2⌋) ∏_s = 1^⌊ r/2 ⌋ p_ X, Z( x_s,  z + h_2 u_s) ∏_s = 1^⌊ r/2 ⌋d x_s d u_s 
       →( ⌊r/2⌋)! ( ∫ K ( u )^2 d u )^⌊ r/2 ⌋∫∏_s = 1^r/2ψ( x_1, x_1, …,  x_⌊ r/2⌋, x_⌊ r/2⌋) ∏_s = 1^⌊ r/2 ⌋ p_ X, Z( x_s,  z ) ∏_s = 1^⌊ r/2 ⌋d x_s  .
  
    Combining the above with (<ref>) and (<ref>), the proof of Lemma <ref> is completed. 
     


 
Note that if ψ is not symmetric, then Lemma <ref> holds with ψ' instead of ψ, where ψ' is defined as
    
    ψ'( x_1,…,  x_r) = 1/r!∑_τ∈ S_rψ( x_τ ) ,

        where S_r is the set of permutations of {1, 2, …, r} and x_τ = (  x_τ(1),  x_τ(2), … x_τ(r) ), for τ∈ S_r. 



To derive the asymptotic properties of the conditional 2-sample V-statistic (<ref>), we need the 2-sample analogous of the previous lemmas. We begin with the 2-sample version of Lemma <ref>. 
 

 Fix r_1, r_2≥ 1 and suppose ( X,  Y,  Z), ( X_1,  Y_1,  Z_1), …, ( X_r_1+r_2,  Y_r_1+r_2,  Z_r_1+r_2) are i.i.d. samples from p_ X, Y,  Z. Further, suppose the following conditions hold:   


  * p_ X,  Y,  Z satisfies assumption <ref>; 


  * the kernel K satisfies assumption <ref>; 


  * h_1 and h_2 are bandwidths satisfying the conditions in assumption <ref>. 

 
Consider a measurable function ψ: (ℝ^d_X)^r_1× (ℝ^d_X)^r_2→ℝ which is symmetric in its first r_1 and last r_2 coordinates and 

    sup_ y ∈ℝ^d_Y,  z ∈ℝ^d_Z𝔼[ψ^2(  X_[r_1+r_2]) |{ Y_s =  y }_ 1  ≤ s ≤ r_1, { Z_s =  z }_1 ≤ s ≤ r_1+r_2  ] < ∞,

where X_[r_1+r_2] = (  X_1, …,  X_r_1+r_2). Then the following holds, as h_1, h_2→ 0, 
   
    lim_h_1, h_2 → 01/h_1^r_1 (d_Y+d_Z)  h_2^ r_2 d_Z 𝔼[  ∏_s=1^r_1  K( ( y, z)  - ( Y_s, Z_s)  /h_1) ∏_s=r_1+1^r_1+r_2  K( z -  Z_s /h_2)    ψ(  X_[r_1+r_2] )  ] 
        =  p_ Y, Z ( y, z)^r_1 p_ Z ( z)^r_2[ψ(  X_[r_1+r_2] )  |{ Y_s =  y_s }_ 1  ≤ s ≤ r_1, { Z_s =  z_s }_1 ≤ s ≤ r_1+r_2] .
 
Further,  
   
    lim_h_1, h_2 → 0 1 /h_1^r_1 (d_Y+d_Z)  h_2^ r_2 d_Z [ ∏_s=1^r_1  K(  ( Y,  Z)   - ( Y_s,  Z_s)  /h_1)  ∏_s=r_1+1^r_1+r_2  K( Z -  Z_s /h_2) ψ(  X_[r_1+r_2] )  ] 
         
        = ∫  p_ Y, Z ( y, z)^r_1+1 p_ Z ( z)^r_2𝔼[ψ^2(  X_[r_1+r_2]) |{ Y_s =  y }_ 1  ≤ s ≤ r_1, { Z_s =  z }_1 ≤ s ≤ r_1+r_2  ] d y d z  .
 
    



    The proof follows similar to Lemma <ref>. The details are omitted. 
      



  Note that if the function ψ is conditionally degenerate of order 1 (as in Definition <ref>), then the limits in (<ref>) and (<ref>) are zero. In this case, by an expansion similar to (<ref>) one can show  that 

    1/h_1^r_1 (d_Y+d_Z)  h_2^ r_2 d_Z 𝔼[  ∏_s=1^r_1  K( ( y, z)  - ( Y_s, Z_s)  /h_1) ∏_s=r_1+1^r_1+r_2  K( z -  Z_s /h_2)    ψ(  X_[r_1+r_2] )  ]  
        = O(h_1^2 h_2^2) = O(h_1^4) ,

since h_2/h_1 → 0. Similarly, 

    1/h_1^r_1 (d_Y+d_Z)  h_2^ r_2 d_Z 𝔼[  ∏_s=1^r_1  K( ( Y,  Z)  - ( Y_s, Z_s)  /h_1) ∏_s=r_1+1^r_1+r_2  K( Z -  Z_s /h_2)    ψ(  X_[r_1+r_2] )  ] 
        = O(h_1^2 h_2^2) = O(h_1^4) .
 




Now, suppose the function ψ:(^d_X)^r_1× (^d_X)^r_2→, which is symmetric in its first r_1 and last r_2 arguments, is completely degenerate, as in Definition <ref>. Then we have following  2-sample version of Lemma <ref>. 


 
Suppose the assumptions of Lemma <ref> hold. Further, assume  that the function ψ:(^d_X)^r_1× (^d_X)^r_2→ is completely degenerate. Define 

    T_r_1, r_2, n^ y,  z : = 𝔼[ ∑_ i ∈ [n]^r_1∑_ j∈ [n]^r_2∏_s=1^r_1K( ( y, z)  - ( Y_i_s,  Z_i_s)  /h_1) ∏_s=r_1+1^r_1+r_2 K( z -  Z_j_s/h_2)  ψ(  X_( i, j))  ],

where  i = (i_1,i_2,…, i_r_1), j = (j_1, j_2,…, j_r_2), and 

    X_( i,  j) = ( X_i_1, X_i_2,…,  X_i_r_1;  X_j_1, X_j_2,…,  X_j_r_2).

Then the following hold:  



  * If r_1 = 2 ℓ_1 and r_2 = 2 ℓ_2 are both even,  
    
    lim_n →∞T_r_1, r_2, n^ y,  z/ (nh_1^ d_Y+d_Z )^r_1/2 (nh_2^ d_Z )^r_2/2
        =C_1^ℓ_1 C_2^ℓ_2ℓ_1 ! ℓ_2  ! ∫ψ^*( x_1,…, x_ℓ_1 + ℓ_2 ) ∏_i=1^ℓ_1 p_ X, Y, Z( x_i, y,  z)  ∏_i= ℓ_1+ 1^ℓ_1 + ℓ_2  p_ X, Z( x_i,  z)   ∏_i=1^ℓ_1 + ℓ_2 d x_i ,

    where ψ^*( x_1,…,  x_ℓ_1+ ℓ_2) := ψ( x_1, x_1, …, x_ℓ_1, x_ℓ_1;  x_ℓ_1+1, x_ℓ_1+1, …, x_ℓ_1+ℓ_2, x_ℓ_1+ℓ_2 ),  
    C_1 := ∫_^d_Y+d_Z K^2( ud u ), and C_2 := ∫_^d_Z K^2( v)d v. 
 
  * If either one of r_2 or r_1 is odd, 
 
    lim_n →∞T_r_1, r_2, n^ y,  z/ (nh_1^ d_Y+d_Z )^r_1/2 (nh_2^ d_Z )^r_2/2 = 0.








    




   The proof is similar to Lemma <ref>. We provide a sketch, omitting some details: Denote by 
i and j the distinct elements of i ∈ [n]^r_1 and j ∈ [n]^r_2. Then 
    
    T_r_1, r_2, n^ y,  z    =  ∑_0 ≤ a_1, b_1 ≤ r_1
     a_1 + b_1 ≤ r_1∑_0 ≤ a_2, b_2 ≤ r_2
     a_2 + b_2 ≤ r_2 T_a_1, b_1, a_2, b_2, n ,
 
       where 
       
    T_a_1, b_1, a_2, b_2, n
        := ∑_ i ∈ [n]^r_1
    ν_1( i) = a_1, ν_2( i) = b_1 ∑_ j ∈ [n]^r_2
    ν_1( j) = a_2, ν_2( j) = b_2 𝔼[ ∏_ s ∈ i K_s( ( y,  z)  - (  Y_s,  Z_s)  /h_1) ∏_ t ∈ j K_t( z -  Z_t/h_2)   ψ( X_(ℓ_ i, ℓ_ j))  ] ,

       with notation as in the proof of Lemma <ref>. 
       First note that when a_1+b_1  r_1/2 or a_2 + b_2  r_2/2, then as in the proof of Lemma <ref> we can show that 
        
    lim_n →∞ T_a_1, b_1, a_2, b_2, n/ (nh_1^ d_Y+d_Z )^r_1/2 (nh_2^ d_Z )^r_2/2 = 0 .
 
       Hence, suppose r_1 = 2 ℓ_1 and r_2 = 2 ℓ_2 are even and a_1+b_1 = ℓ_1 and a_2 + b_2 = ℓ_2. Then, by arguments similar to (<ref>) it can be shown that, when a_1 ≥ 1 or a_2 ≥ 1, 
           
    lim_n →∞ T_a_1, b_1, a_2, b_2, n/ (nh_1^ d_Y+d_Z )^r_1/2 (nh_2^ d_Z )^r_2/2 = 0 ,
 
      This implies, from (<ref>),            
      
    T_r_1, r_2, n^ y,  z/ (nh_1^ d_Y+d_Z )^r_1/2 (nh_2^ d_Z )^r_2/2  =  T_0, r_1/2, 0, r_2/2, n/ (nh_1^ d_Y+d_Z )^r_1/2 (nh_2^ d_Z )^r_2/2 + o(1) .
 
            The result in Lemma <ref> then now follows from Lemma <ref>. 
   


For a function ψ that is not necessarily symmetric, the result in Lemma <ref> holds with ψ' instead of ψ, where ψ' is the symmetrization of ψ as defined in (<ref>).   





 §.§ Proof of Proposition <ref>



We begin the following simple fact about kernel density estimation. 



    Suppose the assumptions of Proposition <ref> hold. If p_ Z( z)>0, then (recalling (<ref>))
    
    1/nh_2^ d_Z  w( z) = 
       1/nh_2^ d_Z ∑_i=1^n K( z- Z_i/h_2) L_2→ p_ Z( z) ,

    as n→∞. Similarly, if p_ Y,  Z( y,  z)>0, then 
    
    1/nh_1^ (d_Y+d_Z)  w(  y,  z) = 1/nh_1^ (d_Y+d_Z) ∑_i=1^n K(( y,  z) -( Y_i,  Z_i) /h_1) L_2→ p_ Y,  Z( y,  z).

    



    Note that by Lemma <ref>,
    
    [1/nh_2^ d_Z ∑_i=1^n K( z- Z_i/h_2)] = 1/h_2^d_Z[K( z- Z_1/h_2)] → p_ Z( z).
 
    Also, by Lemma <ref>,
    
    [1/nh_2^ d_Z ∑_i=1^n K( z- Z_i/h_2)]^2 =   n(n-1)/n^2h_2^2d_Z[K( z- Z_1/h_2) K( z- Z_2/h_2)] + n/n^2h^2d_Z[K^2( z- Z_1/h_2)]
       → p_ Z^2( z).

    Combining these we get [1/nh_2^ d_Z ∑_i=1^n K( z- Z_i/h_2)] → 0, as n→∞. This proves (<ref>). The result in (<ref>) can be proved similarly.  
    


Now, we proceed with the proof of Proposition <ref>. For any f∈ℱ_ z, we can write 
 
    S_n^ z(f)     :=  √(nh_2^ d_Z )(∫ f( x) dℙ_ X|  Z =  z( d x)-_ℙ_ X|  Z =  z [ f( X) ] ) 
        = √(nh_2^ d_Z )(∑_i=1^n w_ Z_i(  z ) f( X_i)/∑_i=1^n w_ Z_i(  z ) -_ℙ_ X|  Z = z [f( X)] ) 
        = √(nh_2^ d_Z )(1/nh_2^ d_Z ∑_i=1^n w_ Z_i(  z ) f( X_i)/1/nh_2^ d_Z ∑_i=1^n w_ Z_i(  z ) -_ℙ_ X|  Z = z [f( X)])
        = (1/√(nh_2^ d_Z )∑_i=1^n w_ Z_i(  z ) f( X_i) - 1/√(nh_2^ d_Z )∑_i=1^n w_ Z_i(  z ) _ℙ_ X|  Z = z [f( X)])/(1/nh_2^ d_Z ∑_i=1^n w_ Z_i(  z )  )  .

Similarly, for g ∈ℱ_ y,  z we have, 

    S_n^ y,  z(g)     := √(nh_1^ d_Y + d_Z )(∫ f( x) dℙ_ X|  Y= y,  Z =  z( d x)-_ℙ_ X|  Y= y,  Z =  z [ f( X) ] ) 
        = (1/√(nh_1^ d_Y + d_Z )∑_i=1^n w_ Y_i,  Z_i(  y,  z ) g( X_i) - 1/√(nh_1^ d_Y + d_Z )∑_i=1^n w_ Y_i,  Z_i(  y,  z ) _ℙ_ X|  Y =  y,  Z = z [g( X)])/(1/nh_1^ d_Y+d_Z ∑_i=1^n w_ Y_i,  Z_i(  y,  z )  )  .
 

Hence, given a finite collection of functions f_1, f_2, …, f_K_1∈ℱ_ z and a finite collection of functions g_1, g_2, …, g_K_2∈ℱ_ y,  z, 
to obtain the distribution of {{S_n^ z(f_s): 1 ≤ s ≤ K_1 }, {S_n^ y,  z(g_t): 1 ≤ t ≤ K_2 }}, 
we need to study the joint asymptotic behavior of 

    {(1/√(nh_2^ d_Z )∑_i=1^n w_ Z_i(  z ) , 1/√(nh_2^ d_Z )∑_i=1^n w_ Z_i(  z ) f_s( X_i) , 1 ≤ s ≤ K_1 ), 
               ( 1/√(nh_1^ d_Y+ d_Z )∑_i=1^n w_ Y_i,  Z_i(  y,  z ), 
    1/√(nh_1^ d_Y + d_Z )∑_i=1^n w_ Y_i,  Z_i(  y,  z ) g_t( X_i): 1 ≤ t ≤ K_2 ) } .
 
Due to the Cramer-Wold device, it is enough to derive the limiting distribution of 

    1/√(nh_2^ d_Z )∑_i=1^n w_ Z_i(  z ) ( ∑_s=1^K_1 a_s+ ∑_s=1^K_1 a_s' f_s( X_i) ) + 
    1/√(nh_1^ d_Y+ d_Z )∑_i=1^n w_ Y_i,  Z_i(  y,  z ) ( ∑_t=1^K_2 b_t  + ∑_t=1^K_2 b_t' g_t ( X_i) ),

for every a_1, a_2, …, a_K_1, a_1', a_2' …, a_K_1' ∈ and b_1, b_2, …, b_K_2, b_1', b_2' …, b_K_2' ∈. This means that it suffices to obtain the limiting distribution of 

    (1/√(nh_2^ d_Z )∑_i=1^n w_ Z_i(  z )  f( X_i), 1/√(nh_1^ d_Y+ d_Z )∑_i=1^n w_ Y_i,  Z_i(  y,  z ) g( X_i)) ,
 
for any f ∈^ z and g ∈ℱ^ y,  z. For this, consider, for α_1, α_2 ∈, 
    
    1/√(n)∑_i=1^n (α_1 1/√(h_2^d_Z) w_ Z_i( z) f( X_i) + α_2 1/√(h_1^d_Y+d_Z) w_ Y_i, Z_i( y, z) g( X_i))  = 1/√(n)∑_i=1^n V_n, i .

     Note that { V_n, i - [V_n, i]: 1≤ i≤ n} is a triangular array of random variables which are row-wise i.i.d. with variance σ_n^2 = [V_n, 1]. To compute the limit of σ_n^2, first note by that Lemma <ref>, as n →∞, 
     
    [ w_ Z_1( z) f( X_1)/√(h_2^d_Z)] → 0  and [w_ Y_1, Z_1( y, z) g( X_1)/√(h_1^d_Y+d_Z)] → 0.

     Also, by that Lemma <ref>, as n →∞, 
     
    [ w_ Z_1( z)^2 f( X_1)^2/h_2^d_Z] →(∫ K^2( u )d u) p_ Z( z) __ X|  Z = z[f( X)^2]
 and 
      
    [ w_ Y_1,  Z_1( y,  z)^2 g( X_1)^2/h_1^d_Y + d_Z] →(∫ K^2(( u,  v) )d u d v ) p_ Y,  Z( y,  z) __ X|  Y =  y,  Z = z[g( X)^2]  .
 
        Further, note that 
     
    [w_ Z_1( z) f( X_1)/√(h_2^d_Z)·w_ Y_1, Z_1( y, z) g( X_1)/√(h_1^d_Y+d_Z)] 
        = √(1/h_1^d_Y+d_Z h_2^d_Z)∫ K( z-  z_1/h_2) K ( ( y,  z) - ( y_1,  z_1) /h_1) f( x)g( x) p_ X, Y, Z( x, y_1,  z_1) d xd y_1 d z_1 
        = √(h_1^d_Y+d_Z h_2^d_Z)∫ K( u) K(( u,  v)) f( x)g( x) p_ X, Y, Z( x, y+h_1  u,  z + h_2  v) d xd u d v * (by the change of variables y_1 =  y +  h_1  u and z_1 =  z +  h_2  v)
       → 0 ,
 
    as n→∞, under Assumption <ref> and <ref>.  
    Hence, by Lemma <ref> we have,
     
    σ_n^2    →σ^2 
        := (∫ K^2( u)d u) α_1^2 p_ Z( z) __ X|  Z = z[f( X)^2] + (∫ K^2( (  u ,  v) )d u d v) α_2^2 p_ Y,  Z( y, z) _ℙ_ X |  Y= y,  Z =  z[g( X)^2] .

          Then by Lindeberg's central limit theorem, 
     
    1/√(n)∑_i=1^n ( V_n, i - [V_n, i]) D→ N(0,σ^2).

     Also, under assumption <ref> we have nh_1^d_Y+d_Z+4→ 0 and nh_2^d_Z+4→ 0, as n→∞. Therefore, by equation (<ref>)
     
    √(nh_2^d_Z)([1/h_2^d_Z w_ Z_1( z) f( X)] - ∫ f( x) p_ X, Z( x,  z)d x) → 0 ,  
       √(nh_1^d_Y+d_Z)([1/h_1^d_Y+d_Z w_ Y_1, Z_1( y,  z) g( X)] - ∫ g( x) p_ X, Y, Z( x, y,  z)d x) → 0.
 
Moreover, by Lemma <ref>, 
    1/nh_2^ d_Z ∑_i=1^n w_ Z_i(  z ) P→ p_ Z( z)  and 1/nh_1^ d_Y+d_Z ∑_i=1^n w_ Y_i,  Z_i(  y,  z ) P→ p_ Y,  Z(  y,  z).
  Hence, using the Cramer-Wold device and Slutsky's theorem we get the result in Proposition <ref>. 







 §.§ Proof of Proposition <ref>



We begin with the following result. 



 
Suppose the assumptions of Lemma <ref> hold. Further, assume  that the function ψ:(^d_X)^r_1× (^d_X)^r_2→ is completely degenerate as in Definition <ref>. If p_ Z( z)>0 and p_ Y, Z( y, z)>0, then  
    
    ∫ϕ( x_1, …,  x_r_1; x'_1,…, x'_r_2)∏_i=1^r_1d𝔾̂( x_i) ∏_j=1^r_2d𝔾̂'(x'_j) 
       D→∫ϕ( x_1, …,  x_r_1; x'_1…, x'_r_2)∏_i=1^r_1d𝔾( x_i) ∏_j=1^r_1d𝔾'(x'_j),

    as n →∞, where 
    
    𝔾̂ = √(nh_1^d_Y+d_Z)(ℙ_ X |  Y= y,  Z =  z - ℙ_ X|  Y=  y,  Z =  z)   and 𝔾̂'= √(nh_2^ d_Z )(ℙ_ X|  Z =  z - ℙ_ X|  Z =  z)  ,

    are the conditional empirical processes as defined in (<ref>) and (<ref>), respectively; and 
    
    𝔾 = 𝔾_ℙ_ X|  Y =  y,  Z =  z and 𝔾' =  𝔾_ℙ_ X|  Z =  z ,
 are as defined in Proposition <ref>.        



The proof of Lemma <ref> is given in Appendix <ref>. We will first use this result to prove Proposition <ref>. For this, let us define,

    V_n(ϕ)     = ∫ϕ( x_1, …,  x_r_1; x'_1,…, x'_r_2)  ∏_s=1^r_1d_ X| ( Y, Z)  = ( y, z) ( x _s) ∏_s=r_1+1^r_1+r_2d_ X|  Z =  z(x'_s)  ,
 
Now let us define,

    ψ(t,s) = ∫ϕ( x_1, …,  x_r_1; x'_1,…, x'_r_2) ∏_s=1^r_1dΔ_1( x_s) ∏_s=r_1+1^r_1+r_2Δ_2(x'_s),

where 
    Δ_1 = ℙ_ X |  Y= y,  Z =  z+s√(nh_1^d_Y+d_Z)(_ X|  Y =  y,  Z= z - ℙ_ X |  Y= y,  Z =  z)
 
and Δ_2 = _ X|  Z = z+t√(nh_2^d_Z)(_ X| Z= z-_ X|  Z = z). 
Note that ψ(t,s) is a bivariate polynomial in (t,s) with coefficient of t^is^j being 

    r_1 ir_2 j∫ϕ_i,j( x_1,…, x_i; x'_1,…, x'_j) ∏_s=1^id𝔾̂( x_s) ∏_s=1^jd𝔾̂'(x'_s) ,

where ϕ_i,j is the (i, j)-th Hoeffding's projection of the symmetrized function ϕ' (recall (<ref>)). 
Hence, by Lemma <ref>, 

    ∫ϕ_i,j( x_1,…, x_i; x'_1,…, x'_j) ∏_s=1^id𝔾̂( x_s) ∏_s=1^jd𝔾̂'(x'_s) →∫ϕ_i,j( x_1,…, x_i; x'_1,…, x'_j) ∏_s=1^id𝔾( x_s) ∏_s=1^jd𝔾'(x'_s) .

In particular, this implies the coefficients of t^is^j in ψ(s,t) are tight, for all i+j≥ 1. Hence, noting that ψ(0,0) = θ,  

    V_n(ϕ) = ψ( 1/(nh_1)^(d_Y+d_Z)/2 , 1/(nh_2)^d_Z/2) = ψ(0,0) + R_n = θ + R_n,
 
where R_n = O_P((nh_1^d_Y+d_Z)^-1/2). This completes the proof of Proposition <ref> (1). 


Next, suppose ϕ is conditionally degenerate of order k ≥ 0. (Note that k=0 means ϕ is conditionally non-degenerate). Then from Definition <ref>, the (i, j)-th Hoeffding's projection ϕ_i,j= 0, for all i+j≤ k. Then by a Taylor expansion,  

    V_n(ϕ) 
       = θ + ∑_(i, j) 
     i+j=k+1r_1 ir_2 j/n^(k+1)/2 h_1^ i (d_Y+d_Z)/2 h_2^ j d_Z/2∫ϕ_i,j ( x_1,…, x_i; x'_1,…, x'_j) ∏_s=1^id𝔾̂( x_s) ∏_s=1^jd𝔾̂'(x'_s) + R_n,

where 
    R_n=O_P( 1/(nh_1^d_Y+d_Z)^(k+2)/2 + 1/(nh_2^d_Z)^(k+2)/2),
 by Lemma <ref>. Since by Assumption <ref>, h_1^d_Y+d_Z/h_2^d_Z→0, we have,

    (nh_1^(d_Y+d_Z))^(k+1)/2 (V_n(ϕ) -θ) 
        =  ∑_i+j=k+1r_1 ir_2 j h_1^(k+1-i) (d_Y+d_Z)/2/ h_2^ (k+1-i) d_Z/2∫ϕ_i,j ( x_1,…, x_i; x'_1,…, x'_j) ∏_s=1^id𝔾̂( x_s) ∏_s=1^jd𝔾̂'(x'_s) + o_P(1) 
       D→r_1 k+1∫ϕ_k+1, 0 (x_1,…, x_k+1) ∏_i=1^k+1d𝔾(x_i).

The result in Proposition <ref> (2) follows by substituting k=0 in (<ref>). Also, Proposition <ref> (3) follows from (<ref>) by noting that θ=0, for k ≥ 1. 









  §.§.§ Proof of Lemma <ref> 



Let f_0=1, f_1,f_2,… be an orthogonal basis of L_2(ℝ^d_X, ℙ_ X|  Y= y,  Z= z) and g_0=1,g_1,g_2,… be the same for L_2(ℝ^d_X, ℙ_ X|  Z =  z). Since ϕ is completely degenerate, it can represented as 

    ϕ = ∑_i_1,…,i_r_1, j_1,…, j_r_2≥ 1⟨ϕ,f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2⟩ f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2,

where the equality holds in the L_2 sense. We define the truncated version of ϕ as follows: For L ≥ 1, 

    ϕ_L = ∑_1≤ i_1,…,i_r_1, j_1,…,j_r_2≤ L⟨ϕ,f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2⟩ f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2.
 
Note that by the orthogonality of {f_s}_s≥ 1 and {g_s}_s≥ 1 we have,
    
    ϕ-ϕ_L _2^2 
        = ∫ [ ϕ( x_1,…, x_r_1;x'_1,…,x'_r_2)-ϕ_L( x_1,…, x_r_1;x'_1,…, x'_r_2) ]^2 ∏_i=1^r_1dℙ_ X |  Y= y,  Z =  z(x_i) ∏_i=1^r_2d_ X|  Z = z(x'_i)  
       → 0,

    as L→∞. 
    
    The proof of Lemma <ref> proceeds in 3 steps: First we show that the integral of ϕ_L with respect to the conditional empirical processes 𝔾̂ and 𝔾̂' converges to its limiting counterpart with 𝔾 and 𝔾', as n →∞. To this end, using Proposition <ref> and the  continuous mapping theorem note that 
    
    {∏_s=1^r_1∫ f_i_s( u) d𝔾̂ ( u) ∏_s=1^r_2∫ g_j_s( u) d𝔾̂' ( u) }D→{∏_s=1^r_1∫ f_i_s( u) d𝔾 ( u) ∏_s=1^r_2∫ g_j_s( u) d𝔾' ( u) },

    where 𝔾̂ and 𝔾̂' are the conditional empirical processes and 𝔾 and 𝔾' the corresponding Gaussian processes as defined in the statement Lemma <ref>. Now, recalling (<ref>) and applying the continuous mapping theorem again gives, for every L ≥ 1, 
    
    ∫ϕ_L( x_1,…, x_r_1;x'_1,…, x'_r_2) ∏_i=1^r_1d𝔾̂ ( x_i) ∏_i=1^r_2d𝔾̂' (x'_i) 
        = ∑_1≤ i_1,…,i_r_1, j_1,…,j_r_2≤ L⟨ϕ,f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2⟩∏_s=1^r_1∫ f_i_s( u) dĜ( u)×∏_s=1^r_2∫ g_j_s( u)dĜ' ( u) 
       D→∑_1≤ i_1,…,i_r_1, j_1,…,j_r_2≤ L⟨ϕ,f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2⟩∏_s=1^r_1∫ f_i_s( u) d𝔾( u)×∏_s=1^r_2∫ g_j_s( u)d 𝔾' ( u) 
        = ∫ϕ_L( x_1,…, x_r_1;x'_1,…, x'_r_2) ∏_i=1^r_1d𝔾 ( x_i) ∏_i=1^r_2d𝔾' (x'_i),

    as n→∞. 
    
   The second step is to show that the difference ψ_L := ϕ-ϕ_L under 𝔾 and 𝔾' is negligible in L_2, as L →∞. Towards this, for a L_2 function f define,   
    
    V(f) := ∫ f( x_1,…, x_r_1;x'_1,…, x'_r_2) ∏_i=1^r_1d𝔾 ( x_i) ∏_i=1^r_2d𝔾' (x'_i).

    Since {∫ f_s ( u) d𝔾( u) }_1 ≤ s ≤ r_1 and {∫ g_s ( u) d𝔾'( u) }_1 ≤ s ≤ r_2 are two independent sequence of mean zero Gaussian random variables with variance c_K( z) and c_K( y, z), respectively, for any L ≥ 1, 
    
    [V(ψ_L)]^2    ≤ C_r_1, r_2, K∑_i_1,…,i_r_1, j_1,…,j_r_2≥ 1|⟨ψ_L, f_i_1×⋯× f_i_r_1× g_j_1×⋯× g_j_r_2⟩|^2 
       ≤ C_r_1, r_2, Kψ_L _2^2,

    for some constant C_r_1, r_2, K > 0 depending on the kernel K and r_1, r_2. Therefore, using (<ref>),  
    
    lim_L→∞[V(ψ_L)^2 ]=0.

    
 
    Finally, we show that ψ_L := ϕ-ϕ_L is negligible in L_2 under 𝔾̂ and 𝔾̂'. For this, similar to (<ref>),  define 
        
    V_n(f) := ∫ f( x_1,…, x_r_1;x'_1,…, x'_r_2) ∏_i=1^r_1d𝔾̂ ( x_i) ∏_i=1^r_2d𝔾̂' (x'_i).
 
       Since ϕ is completely conditionally degenerate, ψ_L is also  completely conditionally degenerate. Therefore, 
       recalling the definition of 𝔾̂ and 𝔾̂' from (<ref>) gives,     
       
    V_n( ψ_L ) 
        =  (nh_1)^r_1(d_Y+d_Z)/2 (nh_2)^ r_2 d_Z/2∫ψ_L( x_1,…, x_r_1;x'_1,…, x'_r_2) ∏_s=1^r_1dℙ_ X |  Y= y,  Z =  z (x_i)  ∏_s=r_1+1^r_1+r_2dℙ_ X|  Z =  z (x'_i) 
        = (nh_1)^r_1(d_Y+d_Z)/2 (nh_2)^ r_2 d_Z/2[   ∑_ i ∈ [n]^r_1∑_ j ∈ [n]^r_2∏_s=1^r_1 w_( Y_i_s,  Z_i_s)( y,  z) ∏_s=r_1+1^r_1+r_2 w_ Z_j_s( z)  ψ_L( X_ i ;  X_ j)   / w(  y,  z)^ r_1 w( z)^ r_2] .
  
By Lemma <ref>,  1/n h_2^d_Zw( z)  and  1/n h_1^d_Y+d_Zw( y,  z)  converges in L_2 to p_ Z( z) and p_ Y,  Z( y,  z). This implies, defining 

    T_n, L := 1/ (nh_1)^r_1(d_Y+d_Z)/2 (nh_2)^ r_2 d_Z/2∑_ i ∈ [n]^r_1∑_ j ∈ [n]^r_2∏_s=1^r_1 w_( Y_i_s,  Z_i_s)( y,  z) ∏_s=r_1+1^r_1+r_2 w_ Z_j_s( z)  ψ_L( X_ i ;  X_ j) ,

we have T_n, L/V_n(ψ_L) converges in L_2 to p_ Y,  Z( y,  z)^r_1 p_ Z( z)^r_2. Therefore, it suffices to shows that [T_n, L^2] → 0. Note that

    T_n, L^2     = ∑_ i ∈ [n]^2r_1∑_ j ∈ [n]^2r_2∏_s=1^2r_1 w_( Y_i_s,  Z_i_s)( y,  z)  ∏_s=1^2r_2 w_ Z_j_s( z)  ψ̃_L( X_ i;  X_ j) / (nh_1)^r_1(d_Y+d_Z) (nh_2)^ r_2 d_Z ,

where ψ̃_L is a function in 2 r_1 + 2 r_2 variables defined below: 
    
    ψ̃_L( x_1,…, x_2r_2,  x_2r_1+1',…, x_2(r_1+r_2)')
        = (ϕ-ϕ_L)( x_1,…,  x_r_2;  x_2r_1+1',…,  x_2r_1+r_2' )(ϕ-ϕ_L)( x_r_1+1,…,  x_2r_2;  x_2r_1+r_1+1',…,  x_2r_2+2r_1').
 
 Hence, using Lemma <ref>, 
    
    lim_n→∞[ T_n, L^2 ] 
        = (r_1)!(r_2) ! ∫ψ̃_L'( x_1, x_1,…, x_r_1+r_2, x_r_1+r_2) ∏_s= 1 ^r_1 p_ X, Y, Z( x_s, y,  z) ∏_s=r_1+1^r_1+r_2 p_ X, Z( x_s,  z)  ∏_s=1^ (r_1+r_2) d x_s 
        : = T_L ,

    where ψ_L' is the symmetrized version (as in (<ref>)). 
    Now, using the basis expansions (<ref>) and (<ref>) and the Cauchy-Schwarz inequality it follows that 
    
    T_L    ≤ (r_1)!(r_2)! p_ Y, Z( y, z)^r_1 p_ Z( z)^r_2𝔼[ ψ_L^2(  X_[r_1+r_2]) |{ Y_s =  y }_ 1  ≤ s ≤ r_1, { Z_s =  z }_1 ≤ s ≤ r_1+r_2  ] 
       → 0 ,

as L →∞, by (<ref>). Hence, from (<ref>) and recalling that T_n, L/V_n(ψ_L) converges in L_2 to p_ Y,  Z( y,  z)^r_1 p_ Z( z)^r_2, we have 

    lim_L→∞lim_n→∞[V_n(ψ_L)^2]→ 0.

Combining (<ref>), (<ref>), and (<ref>), the result in Lemma <ref> follows.   





§ ADDITIONAL RESULTS
 





 §.§ A Sufficient Condition for (<ref>)
 


Here, we derive a sufficient condition for a  density f( x,  y) to be nice (recall Definition <ref>). 


Suppose ( X,  Y) is a random variable in ^d_X+d_Y with joint density f( x,  y) and marginal densities f_ X( x) and f_ Y( y). Moreover, suppose an appropriate interchange of integrals and derivatives is allowed (see (<ref>)) and for every y, u ∈^d_Y, |R_n( x, y; u)|≤ M( x, y; u) where ∫ M^2( x, y;  u) f( x, y)d x is finite. Then (<ref>) holds if f_ Y( y), 

    sup_ u:  u=1[( u ∂/∂ ylog f_ Y| X( Y| X)^⊤)^2| Y= y],  and sup_ u: u=1[ u ∂^2/∂ y^2log f_ Y| X( Y| X) u^⊤| Y= y]
 are all bounded in y. 


 
By Taylor's theorem, 

    f( x, y+δ_n u) = f( x, y) + δ_n  u ∂/∂ y f( x, y)^⊤ + δ_n^2/2 u ∂^2/∂ y^2 f( x,  y_n^')  u^⊤,

for some intermediate y_n^' such that y -  y_n^'≤δ_n. Then, as n→∞, we have,

    R_n( x, y; u)  = 1/2 u ∂^2/∂ y^2 f( x,  y_n^')  u^⊤/f( x, y)   →1/2 u ∂^2/∂ y^2 f( x,  y)  u^⊤/f( x, y)
        = 1/2 u ∂^2/∂ y^2 f_ Y| X( y| x)  u^⊤/f_ Y| X( y| x)
        = 1/2{( u ∂/∂ ylog f_ Y| X( y| x)^⊤)^2 +  u ∂^2/∂ y^2log f_ Y| X( y| x) u^⊤}.

If the interchange of integrals and derivatives are allowed, it is easy to see that,

    ∫( u ∂/∂ ylog f_ Y| X( y| x)^⊤)^2 f( x, y)d x = -∫ u ∂^2/∂ y^2log f_ Y| X( y| x) u^⊤ f( x, y) d x.

This implies, 

    ∫{( u ∂/∂ ylog f_ Y| X( y| x)^⊤)^2 +  u ∂^2/∂ y^2log f_ Y| X( y| x) u^⊤} f( x, y) d x  = 0.

Now, using |R_n( x,  y; u)| ≤ M( x, y; u) where ∫ M^2( x,  y; u) f( x, y)d x is finite and the Reverse Fatou's Lemma we get 

    lim sup_n→∞∫(R_n( x, y; u))^2 f( x, y)d x 
       ≤∫lim sup_n→∞(R_n( x, y; u))^2 f( x, y)d x 
        = ∫1/4{( u ∂/∂ ylog f_ Y| X( y| x)^⊤)^2 +  u ∂^2/∂ y^2log f_ Y| X( y| x) u^⊤}^2 f( x, y)d x 
        = 1/4[( u ∂/∂ ylog f_ Y| X( Y| X)^⊤)^2 +  u ∂^2/∂ y^2log f_ Y| X( Y| X) u^⊤|  Y= y] f_ Y( y) .

Hence, (<ref>) holds if f_ Y( y), 

    sup_ u:  u=1[( u ∂/∂ ylog f_ Y| X( Y| X)^⊤)^2| Y= y],  and sup_ u: u=1[ u ∂^2/∂ y^2log f_ Y| X( Y| X) u^⊤| Y= y]
 are bounded in y. 






 §.§ Normalized Conditional Ball Divergence
 



In this section we prove the following result about the normalized conditional Ball Divergence (recall (<ref>)): 
    
    ℛ( X,  Y|  Z) :=   [Θ^2(_ X|  Y,  Z, _ X|  Z) ) ]/𝔼 [∫Θ^2(_δ_ x, _ X|  Z) d_ X| Y, Z ( x)   ]  .






   The normalized conditional Ball Divergence has the following properties: 
     
         
  (1) ℛ( X,  Y|  Z) ∈ [0, 1], 
         
  (2) ℛ( X,  Y|  Z) = 0 if and only if X Y |  Z, and
         
  (3) ℛ( X,  Y|  Z) = 1 if and only if X is a measurable function of Y and Z almost surely. 
     


 
    Recalling (<ref>) note that 
    
    Θ^2(F,G)     = A + B ,
 
    where 

    A     := ∫ (F-G)^2(B̅( u,ρ( v, u)))d F( u)d F( v) , 
     
        B     := ∫ (F-G)^2(B̅( u,ρ( v, u)))d G( u)d G( v) .
 
    Now, for any fixed u, v∈^d, define Q( w) =  1{ w ∈B̅( u,ρ( v, u)) } - G(B̅( u,ρ( v, u))). Note that, for W ∼ F, 
    
    𝔼[Q( W)] = 𝔼[  1{ W ∈B̅( u,ρ( v, u)) } - G(B̅( u,ρ( v, u))) ]  = (F-G)(B̅( u,ρ( v, u)).
 
    Hence, by Jensen's inequality, 
    
    (F-G)^2(B̅( u,ρ( v, u)) =  𝔼[Q( W)]^2    ≤𝔼 [Q( W)^2] 
        = 𝔼[ (  1{ W ∈B̅( u,ρ( v, u)) } - G(B̅( u,ρ( v, u))) )^2 ] 
        = ∫(  1{ w ∈B̅( u,ρ( v, u)) } - G(B̅( u,ρ( v, u))) )^2 d F( w) 
        = ∫ (ℙ_δ_ w - G)^2(B̅( u,ρ( v, u))) d F( w) .
     
  Now, combining (<ref>) and (<ref>) and applying Fubini's theorem gives, 
    
    A    ≤∫(∫ (_δ_ w - G)^2 (B̅( u,ρ( v, u))) d F( u)d F( v) ) d F( w) .

        Similarly, we get 
    
    B    ≤∫(∫ (_δ_ w - G)^2 (B̅( u,ρ( v, u))) d G( u)d G( v) ) d F( w).

   Combining (<ref>), (<ref>), and recalling the definition of Θ^2(F,G) from (<ref>) then gives, 
    
    Θ^2(F,G) ≤∫Θ^2(_δ_ w, G) d F( w).

    Setting F = _ X| Y, Z and G = _ X| Z in (<ref>) gives, 
    
    Θ^2(_ X| Y, Z,_ X| Z) ≤∫Θ^2(_δ_ x, _ X| Z) d_ X| Y, Z( x).

    Therefore, taking expectations on both sides with respect to ( Y,  Z) shows ℛ( X, Y| Z) ∈ [0,1]. This proves Proposition <ref> (1). 
    
    It is easy to see that ℛ( X, Y| Z) = 0 if and only if Θ^2(_ X| Y, Z, _ X| Z) = 0 almost everywhere, that is, if and only if X Y| Z (by Proposition <ref>). This shows Proposition <ref> (2). 
    
    Next, note that ℛ( X, Y| Z) = 1 if and only if  (<ref>) holds with an equality. This implies, by the equality condition of Jensen's inequality, for almost every u,  v, 
    
    Q( x) =  1{ x ∈B̅( u,ρ( v, u)) } - _ X |  Z(B̅( u,ρ( v, u)))  is constant almost everywhere _ X| Y, Z.
 
  This means _ X| Y, Z is a degenerate distribution or, in other words, X is a measurable function of ( Y, Z). This proves Proposition <ref> (3).  






 Paper Title: Estimating quantum Markov chains using coherent absorber post-processing and pattern counting estimator.










School of Mathematical Sciences, University of Nottingham, United Kingdom
Centre for the Mathematics and Theoretical Physics of Quantum Non-Equilibrium Systems,
University of Nottingham, Nottingham, NG7 2RD, UK
Department of Mathematics, Polytechnic University of Milan, Milan, Piazza L. da Vinci 32, 20133, Italy



School of Mathematical Sciences, University of Nottingham, United Kingdom
Centre for the Mathematics and Theoretical Physics of Quantum Non-Equilibrium Systems,
University of Nottingham, Nottingham, NG7 2RD, UK





School of Mathematical Sciences, University of Nottingham, United Kingdom
Centre for the Mathematics and Theoretical Physics of Quantum Non-Equilibrium Systems,
University of Nottingham, Nottingham, NG7 2RD, UK

———————————————————————————————————–
§ ABSTRACT 
We propose a two step strategy for 
estimating one-dimensional dynamical parameters of a quantum Markov chain, which involves quantum post-processing the output using a coherent quantum absorber and a “pattern counting” estimator computed as a simple additive functional of the  outcomes trajectory produced by sequential, identical measurements on the output units. We provide strong theoretical and numerical evidence that the estimator achieves the quantum Cramé-Rao bound in the limit of large output size.




Our estimation method is underpinned by an asymptotic theory of translationally invariant modes (TIMs) built as averages of shifted tensor products of output operators, labelled by binary patterns. For large times, the TIMs form a bosonic algebra and the output state approaches a joint coherent state of the TIMs whose amplitude depends linearly on the mismatch between system and absorber parameters. Moreover, in the asymptotic regime the TIMs capture the full quantum Fisher information of the output state. While directly probing the TIMs' quadratures seems impractical, we show that the standard sequential measurement is an effective joint measurement of all the TIMs number operators; 
indeed, we show that counts of different binary patterns extracted from the measurement trajectory have the expected joint Poisson distribution. Together with the displaced-null methodology of <cit.> this provides a computationally efficient estimator which only depends on the total number of patterns. This opens the way for similar estimation strategies in continuous-time dynamics, expanding the results of <cit.>.  




 


   





Estimating quantum Markov chains using coherent absorber post-processing and pattern counting estimators
    Mădălin Guţă
    
=========================================================================================================




§ INTRODUCTION



Quantum statistical inference <cit.> provides the mathematical framework for enhanced metrology 
<cit.>, imaging <cit.>,  waveform 

and noise 
estimation <cit.>, 
and quantum sensing applications <cit.> including  time keeping <cit.>, magnetometry <cit.>, biomedical sensing <cit.>, 
thermometry <cit.>, gravitational wave detection <cit.>.







The cornerstone of quantum estimation is the quantum Cramér-Rao bound (QCRB)  <cit.> 
which places a fundamental restriction on the precision in estimating unknown  parameters of a quantum state. 

For one-dimensional parameters the bound is attainable in the limit of many copies, by measuring a specific observable called the symmetric logarithmic derivative, which is the quantum analogue of the classical score function. However, when dealing with complex models involving correlated states of many-body systems,

the optimal measurement may be hard to compute and implement in practice, and in the case of pure states (and more generally, rank deficient states) such a measurement is highly non-unique. Therefore, it is particularly important 
to devise realistic measurement schemes which allow the estimation of unknown parameters with close to optimal precision, by means of computationally efficient estimators. 


In this paper we provide a general measurement and data processing protocol for estimating an arbitrary  dynamical parameter of a quantum Markov chain (QMC), which satisfies the above criteria. A QMC is a discrete time model of an open quantum system, in which the system interacts successively with a sequence of identically prepared “noise units” representing the environment, cf. Figure <ref> a). The setup is similar to Haroche's photon-box one-atom maser <cit.> and to that used in quantum collision models <cit.>, and provides a physical mechanism for generating versatile many-body states such  as matrix product states <cit.> and finitely correlated states <cit.>. By discretising time, QMCs can  be used to model continuous-time dynamics of a Markovian open system coupled with Bosonic input-output channels <cit.>.
   




For clarity, it is useful to distinguish between two mainstream approaches to parameter estimation in quantum open systems. In the setting of <cit.>, the quantum system undergoes a noisy evolution depending on an unknown parameter, and the experimenter tries to extract information about the parameter by repeatedly applying instantaneous direct measurements and control operations while the system is evolving. In contrast, the setting adopted in this paper is commonly used in quantum optics and input-output theory 
<cit.> where the experimenter does not have direct access to the system but can measure the output field of an environment channel coupled to the system. This allows the experimenter to track the conditional state of the system by means of stochastic filtering equations <cit.> and control it using feedback.
As these techniques require full knowledge of the system's dynamical parameters, it is important to devise tools for estimating such parameters from the stochastic trajectory of the measurement record. Since the early works <cit.>, many aspects of continuous-time estimation have been investigated, including adaptive estimation <cit.> filtering methods <cit.>, Heisenberg scaling <cit.>, sensing with error correction <cit.>, Bayesian estimation <cit.>, quantum smoothing <cit.>, 
estimation of linear systems <cit.>, central limit and large deviations theory for trajectories <cit.>, concentration bounds for time averaged observables <cit.>, estimation with feedback control <cit.>.


However, a major problem in this area has been that standard  measurement protocols such as counting and homodyne do not achieve the ultimate precision limit prescribed by the QCRB in terms of the quantum Fisher information (QFI) of the output state <cit.>.


Two recent papers <cit.> have addressed this problem by introducing the idea of quantum post-processing of the output state using a quantum coherent absorber <cit.>. For a given QMC reference dynamics, the absorber takes the system's output as its own input and is characterised by the property that it “reverts” the action of the system, so that its output is a trivial product state (vacuum in continuous-time dynamics), cf. Figure <ref> b). In the statistical estimation framework, the absorber is set to a particular reference parameter of the QMC dynamics, so that a small deviation of the true parameter from this value will lead to non-trivial output statistics which can be used to estimate the deviation as illustrated in Figure <ref> c). In <cit.>, two of the present authors proved that the QCRB can be achieved by performing sequential, adaptive measurements on the output units, after the interaction with the coherent absorber. In addition, the adaptive measurement can be implemented efficiently in a Markovian fashion. The work <cit.> deals with the same problem but in the continuous-time setting, and proposes to perform a standard counting measurement (instead of an adaptive one) in conjunction with post-processing using the coherent absorber. In both papers the final estimator was computed from the measurement trajectory using the maximum likelihood method, which becomes inefficient for long trajectories. 



The strategy proposed here is similar to that of <cit.>, albeit in discrete rather than continuous time, but strengthens it in several important aspects. Firstly, we employ the technique of displaced-null measurements <cit.> to 
provide a precise recipe for choosing the coherent absorber parameter. As we explain below, this is an important technical detail, as the intuitive choice of parameters fails to achieve the QCRB in the limit of long trajectories. Secondly, our strategy employs a two step adaptive procedure which allows us to compute the final estimator as a simple linear transformation of the total number of “pattern counts” which can be easily extracted from the measurement trajectory, cf. Figure <ref> d). This circumvents the computational issues associated with the maximum likelihood estimator. Thirdly, we provide strong theoretical evidence that the final estimator achieves the QCRB in the limit of large times.

This is based on a novel representation of the output in terms of translationally invariant modes which are shown to satisfy the quantum local asymptotic normality property <cit.>. 



We now give a brief summary of the two steps estimation strategy proposed here and the related mathematical results. We consider a QMS whose dynamics depends on a one-dimensional parameter 
θ which we aim to estimate by measuring the output state produced after n time steps. In the first stage we run the QMC dynamics with the unknown parameter θ for 
1≪ñ≪ n time steps and measure the noise units in a fixed basis, cf. Figure <ref> a). From the total counts statistics we construct a rough estimator θ̃_n by matching the empirical frequency to its expected value. In general this estimator is not optimal but its mean square error has the standard 1/ñ scaling (<cit.>). In the second stage we run the system and absorber QMC for the remaining n-ñ time steps and measure the output in the standard basis. If the absorber parameter matched the true system parameter θ, this measurement would produce a string of 0s (corresponding to no counts in continuous-time) cf. Figure <ref> b). Therefore, it would seem natural to choose the absorber parameter to be θ̃_n, our best guess at the unknown parameter θ. However, this choice is unsuitable since for small deviations Δ_n = θ-θ̃_n, the counting statistics depends quadratically on Δ_n, which prevents the estimation of θ at standard 1/n rate. This non-identifiability issue is explained in detail in <cit.>, which also provides the solution to this problem. We deliberately set the absorber parameter at θ_ abs= θ̃_n -δ_n, which is away from the best guess by a small “displacement” δ_n↓ 0 chosen to be larger than the uncertainty |Δ_n|. This allows us to unambiguously identify θ from counts statistics. Stage two of the estimation procedure is illustrated in Figure <ref> c).


We turn now to the question of estimating θ from the counts trajectory ω= (ω_1,ω_2,…, ω_n) of the second stage measurement (setting n-ñ to n for simplicity). Since θ-θ_ abs is vanishingly small (even with the extra displacement), ω will typically contain a small number of 1s while most of the outcomes will be 0, cf. Figure <ref> d). This allows us split the trajectory into long sequences of 0s and in between them, binary “excitation patterns” starting and ending with a 1. For each pattern 
α (e.g. 1, 11, 101 etc.) we count the number of occurrences N_α,n. In Theorem <ref> we show that in the limit of large n the counts N_α,n become independent Poisson variables whose intensities are λ_α u^2 where u= √(n)(θ-θ̃_ abs) is the “local parameter” and λ_α is a model dependent coefficient. Moreover the total Fisher information of the Poisson variables is equal to the output QFI, which shows that the pattern counts statistics capture the full information of the output state. Using this asymptotic behaviour, we construct a simple estimator θ̂_n (cf. equations (<ref>) and (<ref>)) which is linear in the total pattern count, and we argue why it should achieve the QCRB in the limit of large n. To summarise, the two step procedure provides a computationally and statistically efficient estimation method which involves only standard basis measurements and a minimal amount of “quantum post-processing” implemented by the coherent absorber.

For a more in-depth understanding of why the  excitation pattern counts have asymptotically Poisson distributions, we refer to sections <ref> and <ref> where we develop a theory of translationally invariant modes (TIMs) of the output. These modes turn out to capture all statistical information about the unknown parameter, and can be measured simultaneously and optimally by performing the sequential standard output measurement. For each excitation pattern α=(α_1,… ,α_k)∈{0,1}^k we define the creation operator A^*_α(n) on the output chain of length n. This consists of a running average 

    A^*_α(n)=1/√(n)∑_i=1^n-k+1σ^α_i

where σ^α_i is the tensor product of the 
the type 
σ^α = σ^α_1⊗…⊗σ^α_k where σ^0= 1 and σ^1 = σ^+ = |1⟩⟨ 0|, with first tensor acting on position i of the output chain. In Proposition <ref> and Corollary  <ref> we show that asymptotically with n, by applying the creation operators to the reference (vacuum) state |0⟩^⊗ n, we obtain Fock-type states with different excitation pattern numbers. The creation and annihilation operators satisfy the Bosonic commutation relations with each excitation pattern being an independent mode. For large n, a Fock state is a superposition of basis states consisting of long sequences of 0s interspersed with the corresponding patterns appearing in any possible order. One of our key results, Theorem <ref> shows that when the gap between system and absorber parameters scales as θ - θ_ abs=u/√(n), the quadratures of the excitation pattern modes satisfy the  Central Limit Theorem and the corresponding joint state is a product of coherent states whose amplitudes are linear in the local parameter u. The total QFI of this multimode coherent state is equal to the output QFI, showing that the TIMs contain all statistical information about the dynamics. We also prove separately, that the number operators of the TIMs have asymptotic Poisson distributions, as expected for a coherent state. Together with the result of Theorem <ref>, this completes a circle of ideas, which played a crucial role in formulating our estimation strategy. In a nutshell, when looking at the output from the perspective of the TIMs, one deals with a simple Gaussian estimation problem. Using the displaced-null method, we can achieve the QCRB by measuring the number operators of the TIMs and such a measurement can be implemented by simple sequential counting measurements followed by the extraction of pattern counts from the measurement trajectory ω.



 

The paper is organised as follows. In section <ref> we give a brief review of quantum estimation theory and the displaced-null measurement technique developed in <cit.>. In section <ref> we introduce the notion of QMC and the estimation problem, together with the idea of quantum post-processing using a coherent absorber. In section <ref> we define the translationally invariant modes of the output and establish their Fock space properties. In section <ref> we show that the restriction of the output state to the TIMs is a coherent state whose amplitude is linear in the local parameter and whose QFI is equal to the output QFI (cf. Theorem <ref> and Corollary <ref>). 
In section <ref> we establish that the excitation pattern counts obtained from the sequential output measurement have asymptotically Poisson distribution (cf. Theorem <ref>). In section <ref> we formulate our measurement and 
estimation strategy and define the "pattern counts" estimator. Finally in section <ref> we present results of a simulation study confirming the earlier theoretical results. 










§ QUANTUM ESTIMATION AND THE DISPLACED NULL MEASUREMENT TECHNIQUE




In this section we give a brief overview  of the quantum parameter estimation theory <cit.> used in this paper, with an emphasis on asymptotic theory and the displaced-null measurement technique developed in our previous work <cit.>. In particular, we explain  why this method is asymptotically optimal, by employing the Gaussian approximation technique called local asymptotic normality 
<cit.>. Later on, this picture will guide our intuition when dealing with the Markov estimation problem. For our purposes it suffices to discuss the case of one-dimensional parameters, and we refer to <cit.> for the multi-dimensional setting.

Let ρ_θ∈ M(ℂ^d) be a family of quantum states depending smoothly on a one-dimensional parameter 
θ. Consider a measurement described by a positive operator valued measure 
{M_1,…, M_k} and let X be the measurement outcome with probability distribution p_θ(X=i) =  Tr(ρ_θ M_i). The quantum Cramér-Rao bound (QCRB) <cit.> states that the variance of any unbiased estimator θ̂ =θ̂(X) is lower bounded as

    Var(θ̂) =𝔼_θ(θ̂-θ)^2 ≥ 
    F_θ^-1

where F_θ is the quantum Fisher information (QFI) defined as 
F_θ =  Tr (ρ_θℒ_θ^2) with ℒ_θ the symmetric logarithmic derivative (SLD) operator which satisfies 

    dρ_θ/dθ =1/2(ℒ_θρ_θ + ρ_θℒ_θ).

In general, the QCRB is not achievable when only a single copy of ρ_θ is available. However, the bound is attainable in the asymptotic limit of large number of samples by the following two step adaptive procedure <cit.>. Given n copies of ρ_θ one can use a small proportion of the samples (e.g. ñ = n^1-ϵ for a small ϵ >0) to compute a preliminary (non-optimal) estimator 
θ̃_n of θ; reasonable estimators will concentrate around θ such that 
|θ̃_n-θ|=O (n^-1/2+ϵ) with high probability, which will be assumed throughout. In the second step, one measures the SLD operator 
ℒ_θ̃_n on each of the remaining copies. If X_1,… , X_n^' are the outcomes of these measurements (with n^' = n-ñ) then the estimator

    θ̂_n:= θ̃_n+ 1/F_θ̃_nn^'(∑_i=1^n^' X_i)

is asymptotically optimal in the sense that 

    n𝔼_θ(θ̂_n -θ)^2 → F_θ^-1
 
in the limit of  large n and in addition θ̂_n is asymptotically normal, i.e. 
√(n)(θ̂_n -θ) converges in distribution to the normal N(0, F_θ^-1). 

However, for certain models including that considered in this paper,  measuring the SLD may not be feasible experimentally. Instead, we will  use a different method called displaced-null measurement <cit.>, which aims to estimate the parameter of a pure state models 
ρ_θ = |ψ_θ⟩⟨ψ_θ| 
by measuring each copy in a basis that contains the vector |ψ_θ̃⟩ with θ̃ close to the true parameter 
θ. The Fisher information of such a measurement is known to converge to the QFI as θ̃ approaches θ <cit.>. This suggests that the QCRB can be achieved asymptotically by using a two-step strategy similar to the SLD case: one first obtains a preliminary estimator θ̃_n and then measures each copy in a basis containing the vector |ψ_θ̃_n⟩. However, it turns out that this “null measurement" strategy fails due to the fact that for small deviations from θ̃_n, the outcome probabilities depend on (θ-θ̃_n)^2 and one cannot distinguish between left and right deviations from θ̃_n, cf. <cit.> for the precise mathematical statement. This non-identifiability issue can be sidestepped by deliberately changing the reference parameter from θ̃_n to 
θ_0:=θ̃_n - δ_n where δ_n =n^-1/2+3ϵ, so that θ= θ_0 +(u+τ_n)/√(n) with 
τ_n=n^3ϵ. The choice of τ_n is not unique but we refer to <cit.> for the general requirements. Since 
|θ -θ̃_n| =O(n^-1/2+ϵ) ≪δ_n, it means that 
θ lies on the right side of θ_0 and can be unambiguously identified from the outcomes of a measurement in a basis {|e_0⟩, … , |e_d-1⟩} such that 
|e_0⟩≡ 
|ψ_θ_0⟩. In addition, as θ_0 approaches θ in the limit of large n, the displaced-null measurement exhibits the optimality properties of the “null measurements" without sharing their non-identifiability issues. 


Let X_1,… , X_n'∈{0, 1,… , d-1} 
be the independent outcomes of basis {|e_0⟩, … , |e_d-1⟩} measurements performed on the remaining n^' = n-ñ systems, and let N_j,n denote the counts of the outcome j=0,… d-1. The displaced-null estimator based on the two-stage measurement strategy is defined as follows

    θ̂_n := θ̃_n +û_n/√(n)

with local parameter estimator

    û_n= 
    
    2/τ_n f∑_j=1^d-1N_j,n - τ_n/2

where f= 4 ψ̇_θ̃_n^2 is the QFI at θ̃_n. The estimator θ̂_n is asymptotically optimal in the sense of equation (<ref>) and asymptotically normal.



In appendix <ref> we give more insight into this method by analysing its  properties using the theory of local asymptotic normality <cit.>. In a nutshell, for large n the original model becomes equivalent to a Gaussian one consisting of a multi-mode coherent state whose amplitude depends linearly on the parameter, while the SLD and displaced-null strategies translated into measuring a quadrature and respectively the modes number operators of the shifted state. While this i.i.d. setup is different from the Markovian one studied in this paper, the overall asymptotic picture is similar and reader may find the i.i.d. case useful in guiding the intuition.    








§ QUANTUM MARKOV CHAINS AND POST-PROCESSING  USING COHERENT ABSORBERS







We start this section by reviewing the problem of  estimating dynamical parameters of quantum Markov chains (QMC). We then introduce the notion of quantum coherent absorber, which will play a key role in designing an optimal sequential measurement strategy.


A quantum Markov chain consists of a system interacting successively with a chain of independent "noise units" (the input) modelling the environment. In this paper the system's space is taken to be ℂ^d while the "noise units" are two dimensional systems prepared in the state |0⟩ where {|0⟩, |1⟩} is the standard basis in ℂ^2. We expect that the theory developed here works for general finite dimensional inputs, but we restrict here to this minimal setup which can be used to represent a discretised version of a continuous-time Markovian model with a single Bosonic field <cit.>.

At each time step  the system interacts with the input unit via a unitary
U on ℂ^d⊗ℂ^2. If the system is initially prepared in a state |φ⟩, the joint state of system and noise units (output) after n times steps is 

    |Ψ_n⟩   =    U_n |φ⊗ 0^⊗ n⟩
       =    
    U^(n)·…· U^(2)· U^(1)|φ⊗ 0^⊗ n⟩∈ℂ^d⊗(ℂ^2)^⊗ n

where U^(i) is the unitary acting on the system and the i-th noise unit. By expanding the state (<ref>) with respect to the standard product basis in the output we have

    |Ψ_n⟩ = 
    ∑_i_1,… , i_n ∈{0,1} 
    K_i_n… K_i_1 |φ⟩⊗ |i_1⟩⊗…⊗ |i_n⟩

where K_i= ⟨ i|U|0⟩ are Kraus operators acting on ℂ^d.



From equation (<ref>) it follows that the reduced system state of the system at time n is given by 

    ρ^ sys_n := 
    
     Tr_ out
    (|Ψ_n⟩⟨Ψ_n|) = 
    𝒯_*^n(ρ^ sys_ in),   ρ^ sys_ in = |φ⟩⟨φ|,

where the partial trace is taken over the output noise units, and 
𝒯_*:M(ℂ^d) → M(ℂ^d) 
is the Markov transition operator 
(Schrödinger picture)

    𝒯_*: ρ↦∑_i∈{0,1} 
    K_i ρ K_i^*

whose dual (Heisenberg picture) will be denoted by 𝒯.  

On the other hand, the reduced state of the output is 

    ρ^ out_n    :=    Tr_ sys (|Ψ_n⟩⟨Ψ_n|)
    
       =   ∑_ i, j∈{0,1}^n⟨φ |
    K^*_ jK_ i 
    |φ⟩·
    | i⟩⟨ j|

where K_ i:= K_i_n… K_i_1 for i= (i_1,… i_n).

 
Throughout the paper we will assume that the dynamics is primitive in the sense that 𝒯_* has a unique stationary state ρ_ ss>0 so that 𝒯_*(ρ_ ss)= ρ_ ss and it is aperiodic, i.e. the only eigenvalue of 𝒯_* with unit absolute value is 1.



Estimation of dynamical parameters


We investigate the following quantum estimation problem: assuming that the dynamics depends smoothly on an unknown parameter 
θ∈ℝ, we would like to estimate θ by performing measurements on the output state 
ρ_n^ out generated after a number n of interaction steps. In particular, we are interested in designing measurement strategies which achieve the highest possible precision, at least in the limit of large times. 

Let θ↦ U_θ be a smooth map describing how the dynamics depends on an unknown parameter θ, which is assumed to belong to an open bounded interval Θ of ℝ. We use similar notations |Ψ_θ,n⟩, K_θ,i, 𝒯_θ to denote the dependence on θ of the system-output state, Kraus operators, transition operator, etc. Two sequences of quantum statistical models indexed by time are of interest here: the system-output state 
𝒮𝒪_n:= {|Ψ_θ,n⟩ :θ∈Θ} defined in equation (<ref>) and the output state 
𝒪_n:= {ρ^ out_θ,n : θ∈Θ} defined in equation (<ref>). While the former is more informative than the latter and easier to analyse, we are particularly interested in estimation strategies which involve only measurements on the output, hence the importance of the model 𝒪_n. The following Theorem <cit.> shows that for primitive dynamics the QFI of both models scale linearly with n with the same rate, so having access to the system does not change the asymptotic theory. To simplify the expression of the QFI rate (<ref>) we assume the following "gauge condition"

    ∑_j  Tr 
    (ρ_θ^ ssK̇_θ,j^*
    K_θ,j) =0.

The condition (<ref>) means 
that ∑_j 
K̇_θ,j^*
K_θ,j belongs to the subspace {X:  Tr(ρ_θ^ ssX) =0}⊆ M(ℂ^d) on which 
the resolvent ℛ_θ:= ( Id - 𝒯_θ)^-1 is well defined as the Moore-Penrose inverse. The condition can be satisfied by choosing the complex phase of the Kraus operators appropriately, or equivalently the phase of the standard basis in the noise unit space ℂ^2.



Consider a primitive discrete time Markov chain whose unitary U_θ depends smoothly on θ∈Θ⊂ℝ, and assume that condition (<ref>) holds true. The QFI F^ s+o_n(θ) of the system and output  state |Ψ_θ,n⟩ and the QFI F^ out_n(θ) of the output state ρ^ out_θ, n scale linearly with n with the same rate: 

    lim_n→∞1/n F^ s+o_θ(n) =lim_n→∞1/n F^ out_θ(n) = f_θ
          =4∑_i=1^k   Tr[ρ^ ss_θK̇_θ, i^*K̇_θ,i] 
    
          
    +8∑_i=1^k  Tr[ Im ( K_θ, iρ^ ss_θK̇_θ,i^* ) ·ℛ_θ(  Im∑_jK̇_θ,j^*
    K_θ,j)]

where 


ℛ_θ is the Moore-Penrose inverse of Id- 𝒯_θ.


In the following, we will always assume that f_θ>0 for every θ∈Θ. In general, the classical Fisher information associated to simple repeated measurements (measuring the same observable on each output unit) does not achieve the QFI rate f_θ. However, the class of available measurements can be enlarged by unitarily "post-processing" the output before performing a standard measurement, so that  effectively one measures the original output in a rotated basis. While this shifts the difficulty from measurement to "quantum computation", it turns out that the post-processing can be implemented with minimal computational cost by employing  the concept of a coherent absorber introduced in <cit.>. Indeed <cit.> demonstrated that the QFI rate is achievable by combining post-processing by a coherent absorber with a simple adaptive sequential measurement scheme. Furthermore, <cit.>, argued that one can also achieve the QFI by performing  simple counting measurements in the output, without the need for adaptive measurements. Our goal is to revisit this scheme and to provide a new, computationally and statistically effective estimation strategy.




 §.§ Quantum postprocessing with a coherent absorber




The working of the coherent absorber 
is illustrated in Figure <ref> b).  Consider a QMC with a fixed and known  unitary U. After interacting with the system, each output noise unit interacts with a separate d dimensional system (the coherent absorber), via a unitary V. The system and absorber can now be regarded as a single doubled-up system which interacts with the input via the unitary W:= V U on ℂ^d⊗ℂ^d ⊗ℂ^2, where U acts on first and third tensors and V on second and third. The defining feature is that the system plus absorber have a pure stationary state. One can arrange this by requiring 

    VU: 
    |χ^ ss⟩⊗ |0⟩↦ 
    |χ^ ss⟩⊗ |0⟩

where |χ^ ss⟩∈ℂ^d⊗ℂ^d is a purification of the system stationary state, i.e. ρ^ ss =   Tr_ abs ( |χ^ ss⟩⟨χ^ ss|). This 
implies that in the stationary regime the output is decoupled from system and absorber, and is in the "vacuum" state |0⟩^⊗ n. 
We briefly recall a few expressions related to the construction of V that will be useful later on, and refer to Lemma 4.1 in <cit.> for more details; for clarity, in the following we will use the labels S,A,N to indicate system, absorber and noise unit.

Let us consider a spectral representation of ρ^ ss and the corresponding purification:

    ρ^ ss=∑_i=1^dλ_i |i_S⟩⟨i_S|,   |χ^ ss⟩=∑_i=1^d√(λ_i)|i_S⟩⊗|i_A⟩.

For simplicity we assume that the eigenvalues λ_i are strictly positive and are ordered in decreasing order;  

one can check that the following vectors are orthonormal:

    |v_i⟩=∑_k=0^1∑_j=1^d√(λ_j/λ_i)⟨i_S|K_k |j_S⟩|j_A⟩⊗|k_N⟩,    i=1,…, d.


For any choice of v_d+1,…, v_2d such that {v_1,…, v_2d} is an orthonormal basis for ^d ⊗^2 (the Hilbert space corresponding to the absorber and the ancilla), a suitable choice for V is given by

    V=
    1_S ⊗ (
    ∑_i=1^d |i_A ⊗ 0_N⟩⊗⟨v_i|+ 
    ∑_i=d+1^2d 
    |i_A ⊗ 1_N⟩⊗⟨v_i| ) .


Note that V is not uniquely defined: there is freedom in the spectral resolution of ρ^ ss if there are degenerate eigenvalues and in picking v_d+1,… , v_2d. 

The Kraus operators corresponding to the reduced dynamics W= VU of the system and the absorber together are given by the following expression:

    K̃_k:    =   ⟨ k| W|0⟩ =
    ∑_l=0^1⟨k_N| V  |l_N⟩⟨l_N| U |0_N⟩
       =   ∑_l=0^1  K_l ⊗ V_kl


where V_kl and K_l are the Kraus operators of V and 
U considered without ampliation. 
The following Lemma prescribes the structure of the blocks V_kl. 
We first define the “recovery” channel <cit.> with Kraus operators K^'_i= √(ρ) K_i^*√(ρ^-1) and note that they satisfy the normalisation condition and the recovery channel has ρ as invariant state.


The absorber operators V_kl are of the following form. The blocks V_0l are determined as 
V_0l = K_l^' T
where the transpose is taken with respect to the eigenbasis of ρ. Assuming that 
1 - |V_00|^2 and 1 - |V_01|^2 are strictly positive, then the V_1l blocks are determined up to an overall arbitrary unitary u

    V_10 = u |V_10| , V_11= uw |V_11|

where   
|V_10| = √(1 - 
|V_00|^2), 
|V_11| = √(1 - |V_01|^2 ) are fixed, as well as the unitary 
w= -|V_10|^-1 V_00^* V_01 
|V|_11^-1.




Proof. 
From the definition of |v_i⟩ we have


    V_0k   =
     
     ∑_i,j=1^d√(λ_j/λ_i)⟨j_S| K^*_k |i_S⟩|i_A⟩⟨j_A| = K_k^' T.

which proves the first statement.
From the fact that V is unitary we obtain 

    V_00^*V_00 +V_10^*V_10   =   1
    
    V_01^*V_01 +V_11^*V_11   =   1

from which we get 

    |V_10| = √(1 - |V_00|^2),    
    |V_11| = √(1 - |V_01|^2)

which means that the absolute values of V_10, V_11 are fixed. 
Let V_10= U_0 |V_10| and V_11= U_1 |V_11| be their polar decompositions. Then from

    V_00^*V_01 + V_10^*V_11 =0

we get V_10^*V_11= -V_00^*V_01 and 

    U_0^* U_1= |V_10|^-1 V_10^*V_11 |V_11|^-1 = 
    -|V_10|^-1 V_00^*V_01 |V_11|^-1

which is a fixed unitary w. This proves the claim.



Later on we will require that the system and absorber transition operator 𝒯̃(·) = ∑_k K̃^*_k ·K̃_k is primitive, in addition to the 
system's transition operator 𝒯 satisfying the same property. At the moment we are not able to establish what is the connection between the two properties. However we performed extensive numerical simulations with randomly chosen QMC dynamics and corresponding absorbers which indicate that “generically” with respect to the original dynamics, for every primitive 𝒯 there exists a corresponding absorber such that 
𝒯̃ is primitive. In fact, a stronger statement seems to hold, which is that the spectral gap of 𝒯̃ is smaller or equal to that of 𝒯 and one can always choose the absorber such that the two are equal. For more details on the absorber theory we refer to the recent paper <cit.>.


Returning to the parameter estimation setting where U= U_θ depends on the unknown parameter θ, we note that one cannot implement a coherent absorber which precisely matches 
the system dynamics. Instead, one can implement the absorber for an approximate value θ_0 of θ and try to estimate the offset 
θ-θ_0 by measuring the output. This setting is closely related to that of displaced null measurements discussed in section <ref>. Indeed, the joint state of system, absorber and output is pure for all θ and at θ=θ_0 it is of the product form 
|χ^ ss_θ_0⟩⊗|0⟩^⊗ n, assuming that system and absorber are initially in the stationary state. Therefore, repeated standard basis measurements on the output units constitute a null measurement (in conjunction with a final appropriate measurement on system and absorber). 


The exact procedure for determining θ_0 will be described in section <ref> and follows the important displacement prescription outlined in section <ref>. For the moment it suffices to say that θ_0 will be informed by the outcome of a preliminary estimation stage involving simple (non-optimal) measurements on the output (without post-processing), and it will converge to θ in the limit of large n.  
While for θ_0=θ the output state is the "vacuum", for θ≠θ_0 the output could be seen as carrying a certain amount of "excitations" which increases with the parameter mismatch |θ-θ_0|. 

In section <ref> we show how these "excitations" can be given a precise meaning by fashioning the output Hilbert space into a Fock space carrying modes labelled by certain "excitation patterns". In section 
<ref> we show that from this perspective, the output state converges to a joint coherent state of the excitation pattern modes whose displacement depends linearly on θ-θ_0. This will allow us to devise a simple "pattern counting" algorithm for estimating θ in section <ref>.  









 







§ TRANSLATIONALLY INVARIANT MODES IN THE OUTPUT







In this section we introduce the concept of translationally invariant modes (TIMs) of a spin chain. We show that in the limit of large chain size, certain translationally invariant states acquire the characteristic Fock space structure, and that the corresponding creation and annihilation operators satisfy the bosonic commutation relations. This construction will then be used in analysing the stationary Markov output state in section <ref>.

Let (ℂ^2)^⊗ n be a spin chain of length n and let |Ω_n⟩: = |0⟩^⊗ n be the reference “vacuum” state. For every pair of integers (k,l) with 1≤ l≤ k we define an excitation pattern of length 
k and number of excitations l to be an ordered sequence
α := (α_1, …α_k) ∈{0,1}^k such that α_1=α_k =1 and ∑_i=1^k α_i =l. 
For instance, for k=1,2 the only patterns are 1 and respectively 11 while for k=3 the possible patterns are 
111 and 101.

For each pattern α of length k we define `creation and annihilation' operators

    A^*_α(n) = 1/√(n)∑_i=1^n-k+1σ^α_i, 
       A_α(n)=1/√(n)∑_i=1^n-k+1σ_i^α*

where σ_i^α=∏_j=0^k-1σ_i+j^α_j+1, with σ^0 := 1, 
σ^1:= σ^+ := |1⟩⟨ 0| and the index i denotes the position in the chain.

In particular for α=1 we have

    A^*_1(n)=1/√(n)∑_i=1^nσ_i^+,    A_1(n)=1/√(n)∑_i=1^nσ_i^-.



We further define the “canonical coordinates” and “number operator” of the “mode” α as

    Q_α(n)   =   A_α(n)+A^*_α(n)/√(2),
    
    P_α(n)    =   A_α(n)-A^*_α(n)/√(2)i,
     
    N_α(n)   =   A^*_α(n)A_α(n).

We now introduce “Fock states” obtained by applying 
creation operators to the vacuum. 

Let 𝒫 denote the ordered set of all patterns, where the order is the usual one when regarding patterns as integer numbers in binary representation. Let n:𝒫→ℕ be pattern counts 
n = (n_α)_α∈𝒫 such 
that all but a finite number of counts are zero, and let 
n!:= ∏_α∈𝒫n_α! and | n|:=∑_αn_α the total number of patterns.



We define the approximate Fock state associated to the set of 
counts n as

    | n
    ; n⟩:= 
    1/√( n)!∏_α∈𝒫 
    A^*_α(n)^n_α
    
    |Ω_n⟩ ,    n ≥ 1,

where the product is ordered according to the order on 𝒫. For any fixed n these vectors are not normalised or orthogonal to each other, and indeed they are not linearly independent since the Hilbert space is finite dimensional; however, Proposition <ref> shows 
that the expected Fock structure emerges in the limit of large 
n. Let us first illustrate this with a simple example. 
The state containing 2 patterns α =1 is given by 

    |n_1= 2;n⟩   :=   1/√(2)(A_1^*(n) )^2 |Ω_n ⟩
       =   1/√(2)n∑_i≠ j=1^n 
    |0… 0
    10 … 01 0… 0⟩

where i≠ j indicate the positions of the excitations, while the state containing a single 11 pattern is 

    |n_11=1;n⟩   :=    
    A_11^*(n) |Ω_n ⟩
       =   1/√(n)∑_i=1^n-1 
    |0… 0
    11 0… 0⟩

Now it is easy to check that as n→∞

    ⟨ n_1= 2;n| n_1= 2;n ⟩   =    1/2n^24n(n-1)/2→ 1,
    ⟨ n_11=1;n| n_11=1;n ⟩   =    n-1/n→ 1,
    ⟨ n_1= 2;n| n_11=1;n ⟩   =   2(n-1)/2√(n)n= O(1/√(n)).

This is generalised in the following Proposition which establishes the familiar structure of the bosonic Fock space in the limit of large n.



Let | n;n ⟩ be the "Fock states" defined in equation (<ref>). In the limit of large n the "Fock states" become normalised and are orthogonal to each other

    lim_n→∞⟨ n;n|  m ;n⟩ =
    δ_ n,  m .

Moreover, the order of the creation operators in (<ref>) becomes irrelevant in the limit of large n.


The proof of Proposition <ref> can be found in Appendix <ref>. From Proposition <ref> we obtain the following Corollary which shows that the action of the creation operators on the "Fock states" converges to that of a bosonic creation operator in the limit of large n.



Let β be a pattern and let δ^(β) be the counts set with δ^(β)_α = δ_α, β. Let | n; n⟩ be a "Fock state" as defined in equation (<ref>).
In the limit of large n the action of creation and annihilation operators A^*_β(n) and A_β(n) satisfy

    A^*_β(n)| n ;n⟩   =   √(n_β +1) | n + δ^β; n⟩+o(1)
    
    A_β(n)| n ;n⟩   =   √(n_β) | n - δ^β; n⟩+o(1)


The proof of Corollary <ref> can be found in Appendix <ref>.




§ LIMIT DISTRIBUTION OF QUADRATURES AND NUMBER OPERATORS.





 
In this section we analyse the structure of the output state obtained by post-processing the output of a QMC with a coherent absorber, as described in section <ref>. Motivated by the fact that the statistical uncertainty scales as 
1/√(n)

we choose the absorber parameter 
θ_0 to be fixed and known, and write the system parameter as
θ= θ_0+u/√(n),
where u is an unknown local parameter. For a more in depth motivation we refer to local asymptotic normality theory in appendix <ref>.

Since the output state becomes stationary for long times, it is natural to focus on the state of the translationally invariant modes introduced in section <ref>. In Theorem <ref>
we show that asymptotically with n, the restricted state of these modes is a joint coherent state whose amplitude depends linearly on u, i.e. a Gaussian shift model. Moreover, Corollary <ref> shows that the QFI of this model is equal to the QFI rate f_θ_0 of the output state characterised in Theorem <ref>. This means that the TIMs capture the entire QFI of the output. Together with Theorem <ref> of section <ref>, these  results will be the theoretical underpinning the  estimator proposed in section <ref>.



We consider the system and absorber together as an open system with space ℂ^D= ℂ^d⊗ℂ^d with D= d^2, interacting with the noise units. The corresponding unitary is W_θ= V_θ_0U_θ where the absorber parameter is a fixed value θ_0 (which later will be determined based on a preliminary estimation procedure) and θ is the unknown parameter to be estimated. We distinguish between the system Kraus operators K_θ,i =
⟨ i|U_θ |0⟩∈ M(ℂ^d) and the system and absorber Kraus operators 
K̃_θ,i = ⟨ i| W_θ|0⟩∈ M(ℂ^D), and similarly between the system transition operator 𝒯_θ and the system and absorber transition operator 𝒯̃_θ (X) =∑_i=0^1 K̃_θ,i^*XK̃_θ,i. 
 
We will be interested in the probabilistic and statistical properties of the output state ρ̃^ out_θ of the system and absorber dynamics, for parameters θ in the neighbourhood of a given θ_0. For clarity we state the precise mathematical properties we assume throughout.


The following properties of the system-absorber QMC are assumed to be true:


  * 𝒯̃_θ has a unique faithful invariant state ρ̃^ ss_θ and is aperiodic for θ in a neighborhood of θ_0;

  * The Kraus operators K̃_θ,i and the stationary state ρ̃^ ss_θ are analytic functions of θ around θ_0;

  * At θ_0 the stationary state is pure 
ρ̃^ ss_θ_0=
|χ^ ss_θ_0⟩⟨χ^ ss_θ_0| and 
K̃_θ_0,i |χ^ ss_θ_0⟩=(1-i)|χ^ ss_θ_0⟩ for i=0,1.



To formulate the result we use the 
local parametrisation θ= θ_0 + u/√(n)
where u is to be seen as a local parameter to be estimated from the output of length n. To simplify the notation, we denote the derivatives at θ_0 as 

    K̇_i := . dK_θ,i/dθ|_θ_0,  K̇̃̇_i := . dK̃_θ,i/dθ|_θ_0,
 and drop the subscript θ_0 in K_θ_0,i =: K_i, K̃_θ_0,i =: K̃_i, etc.; we also use the local parameter instead of θ e.g. 
K_θ_0+u/√(n),i =: K_u,i. 

Note that properties 1. and 2. in Hyothesis <ref> imply that ρ̇̃̇^ ss=ℛ̃_*𝒯̇̃̇_*(ρ̃^ ss) where ℛ̃ is the Moore-Penrose inverse of Id - 𝒯̃. In addition, K̃_0|χ^ ss⟩= K̃^*_0|χ^ ss⟩ = |χ^ ss⟩.
Indeed by construction K̃_0|χ^ ss⟩ =|χ^ ss⟩ and 
K̃_1|χ^ ss⟩ =0, and by applying  
K̃_0^*K̃_0+ K̃_1^*K̃_1=1 to |χ^ ss⟩ we get K̃_0^*|χ^ ss⟩ = |χ^ ss⟩. 


Since for large n the dynamics reaches stationarity, we consider the output state corresponding to the system starting in the stationary state ρ̃^ ss_u

    ρ̃^ out_u,n=
    ∑_ i,  j∈{0,1}^n Tr[ 
    ρ̃^ ss_u 
    K̃^*_u, jK̃_u, i]| i⟩⟨ j|


To formulate our results below we need to introduce several superoperators acting on the system and absorber space. For  x∈ M(ℂ^D), we define 

    A_i(x)=𝒯̃(x)     i=0 
    K̃^*_1xK̃_0     i=1 
    K̃^*_0xK̃_1     i=-1,

and

    Ȧ_1(x)=K̇̃̇_1^*xK̃_0 + K̃_1^*xK̇̃̇_0


Furthermore, for every pattern  α of length l, we denote

    A^α= A_α_1⋯ A_α_l and Ã^α=Ȧ_1 A_α_2⋯ A_α_l.


With a slight abuse of notation we will denote the expectation with respect to a density matrix ρ as 
ρ(X) =  Tr(ρ X).


 

Let θ_0 be a fixed parameter and assume that the dynamics satisfies the assumptions in Hypothesis (<ref>). Let θ=θ_0 +u/√(n) be the system parameter with fixed local parameter u. Let α be a fixed pattern and let z=β+iγ∈ℂ with |z|=1. Then the following convergence in distribution hold in the limit of large n with respect to the output state ρ̃^ out_u,n as defined in equation (<ref>).

    
  i)
    The quadratures (<ref>) and (<ref>) of the TIM mode α satisfy the joint Central Limit Theorem

    β Q_α(n)+γ P_α(n)  N(u μ_α,z, 1/2),

where N(μ, V) denotes the normal distribution with mean μ and variance V and  μ_α,z= √(2) (z̅μ_α) with

    μ_α= ρ̃^ ss((𝒯̇̃̇ℛ̃ A^α+Ã^α)(1)).


  ii) The number operator (<ref>) of the TIM mode α satisfies the Poisson limit:

    N_α(n)  Poisson(u^2λ_α),

where
λ_α:=|μ_α|^2.





The proof of Theorem <ref> can be found in Appendix <ref>. 


We now provide a simpler expression for the parameters μ_α in terms of Kraus operators and their first derivatives. As a by-product we show that the sum of all (limiting) QFIs of the Gaussian modes 
(Q_α(n), P_α(n)) is the QFI rate of the output state. This means that the TIMs capture all the QFI of the output state.

 
    Let α be any excitation pattern and let μ_α 
    
    be the constant defined in (<ref>).
    Then μ_α can also be expressed as

    μ_α   =   ⟨K̃_α_|α|⋯K̃_α_1(1-K̃_0)^-1K̇̃̇_0 χ^ ss | χ^ ss⟩
       +   ⟨K̃_α_|α|⋯K̃_α_2K̇̃̇_1 χ^ ss|χ^ ss⟩.
    
    Moreover with
    λ_α=|μ_α|^2, one has
    
    ∑_αλ_α=(K̃_1(1-K̃_0)^-1K̇̃̇_0+K̇̃̇_1)χ^ ss^2.



The proof of Lemma <ref> can be found in Appendix <ref>.

 
Asymptotically with n, the total QFI of the TIMs is equal to the QFI rate of the output state (<ref>), that is

    4∑_αλ_α=4 (K̃_1(1-K̃_0)^-1K̇̃̇_0+K̇̃̇_1)χ^ ss^2 =f_θ_0 .


The proof of Corollary <ref> can be found in Appendix <ref>. 

The upshot of this section is that (asymptotically in n) the statistical information of the output state is concentrated in the TIMs and the state's restriction to the TIM Bosonic algebra is a coherent state. Formally, in order to optimally estimate the parameter, one would only need to measure the appropriate quadrature of the Gaussian shift model, as explained in section <ref>. However, it is not obvious how to perform such a measurement, and the theoretical insight does not seem to help on the practical side. Surprisingly, it turns out that the standard sequential counting measurement is an effective joint measurement of all the TIMs' number operators! This will be the main result of the next section, which in conjunction with the displaced null strategy discussed in section <ref>, provides the ingredients of a counting-based estimation strategy.




§ LIMIT THEOREM FOR COUNTING TRAJECTORIES




In this section we continue to investigate the probabilistic properties of the output state and consider the distribution of the stochastic process obtained measuring the output units sequentially in the canonical basis {|0⟩, |1⟩}. We consider the system and absorber dynamics with 
fixed absorber parameter θ_0 and system parameter 
θ= θ_0+u/√(n) for a fixed local parameter u. The  output state is given by equation (<ref>). The probability of observing a sequence ω =(ω_1,… , ω_n)∈{0,1}^n as the outcome of the first n measurements is given by

    ν_u,n(ω):=ρ^ ss_u ( B_u,ω_1⋯ B_u,ω_n(1) )

where 

    B_u,j(x)=K̃^*_u,1 xK̃_u,1    j=1 
    K̃^*_u,0xK̃_u,0    j=0 
    .

In order to state the main result of this section, we need to introduce a collection of events: first of all we define

    B_ 0(n)={(0,…,0)}⊂{0,1}^n.

Let us fix any real number 0<γ<1. Let m = {m_α}_α∈𝒫 be a set of pattern excitation counts where all  occupation numbers are zero except (m_α^(1), …,m_α^(k)), and define B_ m(n) as the empty set for every n strictly smaller than 
∑_i=1^km_α^(i) |α^(i)|+(k-1)n^γ (|α| is the length of the pattern α), otherwise B_ m(n) is the set of all binary sequences obtained concatenating m_α^(1) copies of α^(1)s, up to m_α^(k) copies of α^(k)s in any order, and inserting 0s before, between and after them in order to reach a total length of n and making sure that between two consecutive patterns there are at least n^γ 0s.


The following Theorem shows that the distribution of the 
pattern counts m converges to a product of Poisson distributions with the same intensities as those of the number operators of the TIM in Theorem <ref>. This means that performing a standard output measurement and extracting the pattern counts provides an effective joint measurement of the number operators of the TIMs. This finding is essential in constructing an optimal estimator in section <ref>.

 
For every positive constant C>0 and finite collection of excitation patterns counts m=(m_α^(1),… ,m_α^(k)),

the following limit holds true:

    lim_n → +∞sup_|u|<C |ν_u,n (B_ m(n) )-e^-λ_totu^2∏_i=1^k(λ_α^(i)u^2)^m_α^(i)
    /m_α^(i)! |=0.

where 
    
    λ_tot    :=   -1/2⟨χ^ ss|(2Ḃ_0* R_0*Ḃ_0*+ B_0*)(|χ^ ss⟩⟨χ^ ss|)|χ^ ss⟩
       =   -(⟨χ^ ss, (2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0+K̃_0) χ^ ss⟩ ).

Moreover,

    ∑_αλ_α = λ_tot.



The Proof of Theorem <ref> can be found in Appendix <ref>.

The previous result has some relevant consequences. Let us define the “pattern extraction” function 



which associates to each trajectory ω∈{0,1}^n a set of pattern counts { N_α,n(ω): α∈𝒫}∈ℕ^𝒫, which is uniquely determined by the condition that ω is a maximal union of contiguous patterns separated by sequences of 0s of length at least n^γ with a fixed 0<γ<1; moreover, let us consider the stochastic process given by the infinite collection of independent random variables {N_α: α∈𝒫} where N_α is a Poisson random variable with parameter λ_α u^2.
 
  For every u ∈ℝ the law of the stochastic process { N_α,n: α∈𝒫} under the measure ν_u,n converges to the one of {N_α:α∈𝒫}. Moreover, for every α∈𝒫, p≥ 1 one has
  
    lim_n→ +∞𝔼[N_α,n^p]=𝔼[N_α^p].


The Proof of Corollary <ref> can be found in Appendix <ref>.

However, in the following we will be interested in local parameters with growing size, i.e. |u| ≤ n^ϵ^' for some 0 <ϵ^' <1/2. In this case we can show the following result.
 
 For 0 <ϵ^' <1/6 and for every finite collection of excitation patterns counts m the following holds true:

    lim_n → +∞sup_|u| ≤ n^ϵ^' | ν_u,n (B_𝐦(n) )/ e^-λ_totu^2∏_i=1^kλ_α^(i)^m_iu^2m_i/m_i!-1  |=0.
  

The Proof of Proposition <ref> can be found in Appendix <ref> as well. Upgrading this result to a  weak convergence one similar to Theorem <ref> remains the subject of future research.



§ PATTERN COUNTING ESTIMATOR



In this section we describe our adaptive estimation scheme which exploits the asymptotic results presented in sections <ref> and <ref>. The scheme involves four key ingredients: 

    
  i) 
    perform a simple output measurement (no absorber) to compute a preliminary estimator;
    
    
    
  ii) set the absorber parameter by using the displaced-null measurement technique developed in <cit.> and run the system-absorber dynamics for the remainder of the time;
    
  iii) perform a sequential counting measurement in the output and extract counts for the TIMs from the outcomes trajectory;
    
  iv)
    construct a simple estimator expressed in terms of total counts of patterns for different TIM modes.
    


The first step of the adaptive protocol is to use ñ := n^1-ϵ≪ n output units to produce a rough preliminary estimator 
θ̃_n. This can be done by performing a repeated standard basis measurement on the output (without using an absorber). 
Typically, the estimator
θ̃_n will have variance scaling with the standard rate ñ^-1= n^-1+2ϵ, and ñ^1/2(θ̃_n-θ) will satisfy the central limit theorem and a concentration bound ensuring that |θ̃_n-θ| =O(n^-1/2+ϵ) with high probability. 

For instance, one can define 
θ̃_n by equating the empirical counting rate with the theoretical rate n_τ:=  Tr(ρ^ ss_τ K_τ, 1^*K_τ,1), see <cit.>. 

In the second step we set the absorber at a parameter value θ_0 and run the system-absorber quantum Markov chain for the reminder of the time n^' = n-ñ. 
The naive choice for θ_0 is our best guess θ̃_n about θ, based on the first stage measurement. However, with this choice, the counting measurement suffers from the non-identifiability issue described in section <ref>. This can be resolved by further displacing the absorber parameter by an amount 
δ_n:= τ_n/√(n) where τ_n= n^3ϵ so that 
θ_ abs:= 

θ̃_n - δ_n. As usual we write 
θ = θ̃_n+u_n/√(n) where u_n is a local parameter satisfying |u_n|≤ n^ϵ, so that 
θ= 

θ_ abs
+(u_n+τ_n)/√(n).

In the third step we perform standard basis measurements in the output of the modified dynamics which includes the absorber. For simplicity, in the discussion below we ignore the fact that we have n^' = n-ñ rather than n output units, which does not affect the error scaling of the estimator. Let ω= (ω_1,… , ω_n) be the measurement outcome with distribution (<ref>), where the local parameter u_n is replaced by 
u_n+τ_n to take into account the displacement. 


We now describe the construction of the estimator from the outcomes of this last stage measurement. The estimator will be built using the “pattern extraction” function { N_α,n(ω): α∈𝒫}∈ℕ^𝒫 that we defined in the previous section; we recall that it
associates to each trajectory ω∈{0,1}^n a set of pattern counts, which is uniquely determined by the condition that ω is a maximal union of contiguous patterns separated by sequences of 0s of length at least n^γ with a fixed 0<γ<1, cf. Figure <ref> d) for illustration. This means that the algorithm will not detect any pattern which contains a sequence of zeros of length larger than n^γ, since this would be seen as being made up of several identified patterns.
We now introduce the final estimator using an intuitive argument based on extrapolating the results of Theorem <ref> from fixed to slowly growing local parameters u_n+τ_n. This means that N_α,n(ω) is approximately distributed as Poisson((u_n+τ_n)^2 |μ_α|^2), for large n; since τ_n= n^3ϵ is larger than u_n= O(n^ϵ), the intensity of the Poisson distribution diverges with n, and the distribution can be approximated further by the normal  
N((u_n+τ_n)^2|μ_α|^2, (u_n+τ_n)^2|μ_α|^2 ) with the same mean and variance. Using 

    1/τ_n(u_n+τ_n)^2    =   
    2u_n + τ_n + o(1) 
    1/τ^2_n(u_n+τ_n)^2    =    1+ o(1)

we obtain that 

    Y_α,n := 1/|μ_α|(N_α,n/τ_n -τ_n |μ_α|^2 )

has approximate distribution N(2u_n|μ_α| , 1). A simple computation shows that the optimal estimator of u_n based on the (approximately) normal variables Y_α,n is the linear combination

    û_n:= Y_n: = 2/ f_θ_ abs∑_α |μ_α| Y_α,n =2/f_θ_ absτ_n∑_α N_α,n - τ_n/2

where f_θ_ abs= 4∑_α |μ_α|^2 is the quantum Fisher information rate of the output by Corollary <ref>. Note that Y_n depends only on the total number of patterns of the trajectory ω, not to be confused with the total number of 1s. 

Since Y_α,n is approximately normal with distribution N(2u_n|μ_α| , 1), we obtain that û_n has approximate distribution N(u_n, f_θ_ abs^-1). 
The final estimator of θ is 

    θ̂_n:=θ̃_n+û_n/√(n).

and it attains the QCRB in the sense that 
√(n)(θ̂_n-θ) converges in distribution to N(0,f_  θ^-1).

At the moment we only have a rigorous proof of this statement assuming a stronger version of Proposition <ref>, which we were not able to obtain; however, we point out that Theorem 2 in <cit.> establishes a similar optimality result in the case of multi-parameter estimation with independent, identical copies.

Before proceeding, we need to introduce some more notations. Let us consider the following collection of random variables:

    N_n,θ,θ̃∼Poisson(λ_ tot(θ̃)(√(n)(θ-θ̃)+τ_n)^2),

where n ∈ℕ, and θ̃, θ∈Θ. We recall that τ_n=n^3ϵ is the displacement size. Let us define

    Y_n,θ,θ̃:=1/2τ_n λ_ tot(θ̃)
    N_n,θ,θ̃
    - τ_n/2.

 
Let fix θ∈Θ at which f_θ is continuous and let θ̃_n be a preliminary estimator which uses ñ:=n^1-ϵ samples with ϵ small enough, such that it satisfies the concentration bound

    ℙ_θ(| θ̃_n-θ |>n^-1/2 +ϵ)
    ≤ C e^-n^ϵ r

for some constants C,r>0. Let 

    θ̂_n:=θ̃_n+Y_n/√(n)  ,

be the final estimator as defined in (<ref>) and (<ref>).

If for every a ∈ℝ one has

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵ |𝔼_θ[e^iaY_n|θ̃_n=θ̃]-𝔼[e^iaY_n,θ, θ̃]|=0,

then θ̂ is asymptotically optimal and asymptotically normal, i.e. the following convergence in law holds for large n

    √(n)(θ̂_n-θ)  N (0, 1/f_θ ).



The proof of the following theorem and the relationship between the extra hypothesis in Eq. (<ref>) and Proposition <ref> can be found in Appendix <ref>.




























In the following section we illustrate our method with results of numerical simulations on a qubit model.











§ NUMERICAL EXPERIMENTS




In this section we illustrate the estimation protocol through numerical simulations, using a qubit model inspired by the previous work <cit.>.
The simulations are implemented in Python using the QuTiP package <cit.>. 


The quantum Markov chain model consists of a two-dimensional system coupled to two-dimensional noise units by a unitary U_θ with unknown parameter θ∈ℝ, where the noise units are all prepared in the same initial state | 0⟩. Since the system interacts with a fresh noise unit at each step, it suffices to specify the action of  U_θ on the states |00⟩ and |10⟩, and we define

    U_θ : |00⟩   →   cos(θ)  √(1-θ^2)|00⟩
          + i sin(θ) √(1-θ^2)|10⟩ + θ|11⟩, 
    
        U_θ : |10⟩   →    i sin(θ)  √(1-λ)|00⟩
          + cos(θ) √(1-λ)|10⟩ + √(λ) e^i ϕ|01⟩,

where λ, ϕ are known parameters. In  simulations we  used ϕ=π/4, λ=0.8 and θ=0.2 for the true values of the parameter. 


In a first simulation study we verify the predictions made in Corollary <ref>. We run the dynamics for a total of n=6× 10^5 time steps, with absorber parameter 
θ_ abs= θ_0 (cf. section <ref>) and system parameter θ= θ_0+ u/√(n) such that θ=0.2 and the local parameter is u =2.

For each run we perform repeated measurements in the standard basis of the output units, to produce a measurement record 
ω =(ω_1, …, ω_n). For each such trajectory, the pattern counts N_α, n(ω) are obtained by identifying patterns (sequences starting and ending with a 1) which are separated by at least n^γ 0s and no pattern contains more than n^γ 0s, where γ>0 is a small parameter. This can be done by combing through the sequence and identifying occurrences of a given such pattern padded by n^γ 0s to the left and right (taking care of the special case of the first and last patterns). Note that for any given n this procedure will not count patterns with more than  n^γ successive 0s. Since the mean counts for each pattern α is |μ_α|^2 u^2 and |μ_α|^2 decays exponentially with |α|, we find that patterns of such length are unlikely to occur for large n. The results of N=2000 independent repetitions of the experiment are illustrated in Figure <ref> which shows a good match between the counts histograms corresponding to several patterns (in blue) and the theoretical Poisson distributions 
Poisson(|μ_α|^2 u^2) (in orange) , as predicted by Corollary <ref>.  








In a second simulation study we use system parameter
θ=0.2 and set the absorber at θ_ abs = θ-δ_n, with displacement δ_n= n^-1/2τ_n for n=6× 10^5 and 
τ_n = 7. We perform the same measurement as above and extract the pattern counts N_α,n(ω) for each trajectory ω. We then use the pattern counts estimator (<ref>) to estimate θ, taking 
θ̃_n=θ. This amounts to assuming that the first stage of the general estimation procedure outlined in section <ref> gives a perfect estimator, which is then used in setting the absorber parameter in the second step. While this procedure cannot be used in a practical situation, the study has theoretical value in that it allows us to study the performance of the pattern counts estimator in its own right, rather than in conjunction with the first step estimator. Figure <ref> shows that the final estimator 
θ̂_n has Gaussian distribution with variance closely matching 1/(n f_θ), thus achieving the QCRB in this idealised setup. This can also be seen by comparing the “effective” Fisher information 
F_ eff:= (n (θ̂_n-θ)^2)^-1 where the mean square error is estimated from the data, with the QFI rate f_θ; the former is equal to F_ eff = 13.8 while the latter is f_θ = 13.5.





In the third simulations study we implement the full estimation procedure described in section <ref> including the first stage estimator. In the first step we fix θ =0.2, and run the Markov chain (without the absorber) for ñ= 4× 10^5. We then perform sequential measurements in the standard basis on the output noise units to obtain a measurement trajectory 
from which we compute θ̃_n by equating the empirical mean (the average number of 1 counts)

with the stationary expected value 
 c_τ: =  Tr (ρ^ ss_τ K_τ,1^* K_τ,1), where K_τ,i are the system's Kraus operators. The effective Fisher information of this estimator is 
 F_ eff = 4.06, significantly lower than the QFI f_θ =13.5.
 

We then set the absorber to 
θ_ abs = θ̃_n - n^-1/2τ_n where τ_n = 25.5 and n=6.6× 10^6 and we run the system and absorber chain for n^' = n-ñ steps. The system and absorber  is initialised in the pure stationary state corresponding to θ̃_n, but since the system and absorber dynamics is assumed to be primitive, any other initial state will 
result in equivalent asymptotic results. We then perform the counting measurement as in the previous simulation studies to obtain a trajectory ω∈{0,1}^n' and extract the pattern counts N_α,n^'(ω). 


From these average counts we compute the estimator (<ref>), where the Fisher information f is computed at θ̃_n and multipled by n^'/n to account for the smaller number of samples used in the last step. The results are illustrated in Figure <ref> which compares the histogram of θ̂_n (in blue) with the density of a normal distribution with mean θ and variance 1/(n f_θ). We find that there is a good fit with the normal distribution but less accurate that that of the second simulation study, cf. Figure <ref>. This is expected, since the final estimator is based on a two stage estimation process and does not use any prior information about θ. A more accurate measure of the protocol's performance is given by the  effective Fisher information which works out as F_ eff = 10.8 compared to the QFI rate f_θ=13.5, while the effective Fisher information of the first stage was only 4.06. These simulation results are in agreement with the theoretical arguments put forward in section <ref> which indicate that the two stage estimator attains the QCRB asymptotically.















 












































































    

      





 §.§ Construction of the estimator





To construct a suitable estimator, we start by considering the local parameters of our model. We assume we've performed some generally sub-optimal preliminary estimation on n^1 - ϵ (ϵ>0) of our total samples. With some reasonable assumptions on the initial estimator θ̃, we can assume that θ∈ I = {θ̃ - n^-1/2n^ϵ, θ̃ + n^-1/2n^ϵ} as n→∞ [previous paper]. Therefore, we write θ = θ̃ + u.n^-1/2, where u is a local parameter satisfying  |u|<n^ϵ. We also introduce a local parameter for the absorber offset τ_n=n^aϵ such that the parameter used by the absorber θ_0 is

    θ_0 = θ̃ + τ_n/√(n),

where a>2. This requirement on a ensures that the absorber is offset sufficiently (justify need for 2?) outside this localized region I. The difference between these parameters is then

    θ - θ_0 = 1/√(n)(u - τ_n)

producing a local parameter u-τ_n. 

Need to add in new factor involving proportion used in final estimation


We now want to relate this local parameter to the rate of occurrence of a specific pattern 
α. From section <ref> we known that the counts N_α(n) of a pattern α are distributed as 

    N_α(n) ∼ Poisson(|μ_α|^2 (u - τ_n)^2 )

in the limit of large n.

Furthermore, by Theorem <ref> we also known that in this limit the quadratures are normally distributed with mean μ_α (u - τ_n) and variance 1/2.

Needs the most work. Need to justify link to this view of measuring the number operator wrt a coherent state


To construct an estimator, we consider the number operator in the Fock space setup introduced in sections <ref> and <ref>. which corresponds to measuring a displaced number operator N_μ_ατ_n = (a - μ_ατ_n)^* (a - μ_ατ_n) with respect to the coherent state |√(2)μ_α u⟩:

    N_α(n)    = N_μ_ατ_n (|√(2)μ_α u⟩) 
       = (a - μ_ατ_n)^* (a - μ_ατ_n) 
       = a^* a - μ_α^* τ_n a - μ_ατ_n a^* + |μ_α|^2τ_n^2,

where the creation/annihilation operators are defined such that a|√(2) u⟩ = u |√(2) u⟩. We then calculate

    ⟨ a^* a ⟩_|√(2)μ_α u⟩   = |μ_α|^2 u^2 = O(n^2ϵ), 
        
    ⟨μ_α^* τ_n a + μ_ατ_n a^* ⟩_|√(2)μ_α u⟩   = 
        2 |μ_α|^2 τ_n u = O(n^(1+a)ϵ),
        
    ⟨ |μ_α|^2 τ_n^2 ⟩_|√(2)μ_α u⟩   = 
        |μ_α|^2 τ_n^2 = O(n^2aϵ)
        .


Note: we can safely neglect the first term in the limit of large n as a>2 ensures (<ref>) ≪ (<ref>) ≪ (<ref>). We then have

    N_α(n) - |μ_α|^2 τ_n^2 ∼ -√(2)|μ_α|τ_n 𝒩( √(2)|μ_α|u, 1/2)


To construct an estimator from the random variable N_α(n) we want to find a function of the random variable with the correct mean √(2) u and minimum variance 1/2. To this end, we consider 

    Y_α = 
        
        -N_α(n) + |μ_α|^2 τ_n^2
        /√(2) |μ_α| τ_n
        ∼𝒩(√(2)|μ_α|u, 1/2).



To include multiple patterns in the final estimator û, we then take a linear combination of these Y_α terms: û = ∑_α c_α Y_α. We then want to find coefficients c_α such that û has the correct mean and minimal variance. We find

    ⟨û⟩   = ∑_α c_α√(2) |μ_α| u 
         c_α = 1/√(2)
        |μ_α|
        /
        ||μ||^2

where ||μ||^2 = ∑_α |μ_α|^2. Checking the variance, we find

    Var(û)    = ∑_α c_α^2 . 1/2
       = 1/4 ||μ||^2 = 1/F

where F=4||μ||^2 is the QFI. Therefore, combining these together we have constructed an estimator with the correct mean and minimal variance in the limit of large n:

    û   = ∑_α
        |μ_α|
        /√(2) ||μ||^2
        
        .
        
        -N_α(n) + |μ_α|^2 τ_n^2
        /√(2) |μ_α| τ_n
        
       =
        2/F τ_n∑_α (-N_α(n)) + τ_n/2



Finally, we restrict the patterns α to the patterns counted in our simulations α^*, which results in the final estimator 

    û = 2/F_rτ_n∑_α^* (-N_α(n)^*) 
        + τ_n/2

where F_r = ∑_α^* |μ_α^*|^2 is the restricted Fisher information.












§ CONCLUSIONS AND OUTLOOK



In this paper we developed a computationally efficient strategy to estimate  dynamical parameters of a quantum Markov chain and provided strong theoretical evidence that the estimator achieves the quantum Cramér-Rao bound in the large time limit. In addition, we established asymptotic results pertaining to the mathematical structure of the output state which are of more general interest.


The estimation strategy consisted of two estimation stages. In the first stage a rough  estimator is computed from outcomes of simple output measurements by using a fast but non-optimal procedure, e.g. equating the empirical counts with the expected value at the estimated parameter. In the second stage, we used a coherent quantum absorber <cit.> to post-process the output <cit.>. When tuned to the true value of the system parameter, the absorber “reverts” the evolution such that, in the stationary regime, the post-processed output is identical to the input “vacuum” product state. On the other hand, small  mismatches between system and absorber parameters lead to slight rotations away from this product state, which can be detected by simple sequential measurements on the output “noise units”.
To achieve the perfect “null” setup it would seem natural to use the first stage estimator as absorber parameter, but as shown in <cit.>, this leads to non-identifiability issues and sub-optimal final estimators. Instead, we applied the displaced-null technique <cit.> which prescribes an extra parameter shift, calibrated to 
remove the non-identifiability issue while preserving the optimality of the sequential measurement.

The key theoretical contributions of this work are related to the understanding of the output state and the stochastic measurement process. We introduced the concept of translationally invariant modes (TIMs) of the output and showed how they generate a Bosonic algebra in the asymptotic limit. Each mode is labelled by a binary sequence called a “pattern”
and its creation operator is an average of shifted blocks consisting of tensor products operators. 
We then showed that when the mismatch between system and absorber parameter scales at the estimation rate n^-1/2, the restriction of the output state to the TIMs becomes a multi-mode coherent state whose displacement depends linearly on the mismatch (quantum Gaussian shift model). Moreover, we showed that the TIMs carry all the quantum Fisher information of the output state and are therefore the relevant quantum statistics of the problem. While homodyne is the standard optimal measurement for such models, 
in the presence of the additional parameter displacement the 
modes amplitudes become large and counting measurements become effectively equivalent to homodyne. Surprisingly, we discovered that the sequential counting measurement acts as a joint measurement of all TIMs number operators. Due to the proximity to the vacuum state, typical trajectories consist of a relatively small number of patterns separated by long sequences of 0s. We showed that for large times the patterns counts distribution converges to the Poisson distribution of the TIMs coherent state. These insights allowed us to devise a simple “pattern counting” estimator for estimating the unknown parameter.


Our discrete-time results open the way for fast and optimal continuous-time estimation strategies based on coherent absorber post-processing <cit.> and simple pattern counting estimation, as opposed to expensive maximum likelihood methods. Interesting and important topics for future investigations concerns the robustness of the pattern counting method with respect to various types of noises, improving the estimation accuracy for short times, extensions to multi-parameter models and the relationship between the general theory of quantum absorbers <cit.> and quantum estimation.  




Acknowledgements. This work was supported by the EPSRC grant EP/T022140/1. MG thanks Juan Garrahan, Mankei Tsang, Dayou Yang, Martin Plenio and Susana Huelga for fruitful discussions. F.G. has been partially supported by the MUR grant Dipartimento di Eccellenza 2023–2027 of Dipartimento di Matematica, Politecnico di Milano and by the INDAM GNAMPA project 2024 “Probabilità quantistica e applicazioni”.







§ DISPLACED-NULL MEASUREMENTS AND ASYMPTOTIC NORMALITY




In this section we review the concept of quantum local asymptotic normality (QLAN) in its simplest form involving a one-parameter family of pure states, and use it to better understand the asymptotic theory of SLD and displaced-null measurements. For the general QLAN theory we refer to <cit.> and <cit.>. For our purposes, QLAN expresses the fact that the statistical model |ψ_θ⟩^⊗ n describing an ensemble of n identically prepared systems can be approximated by a simpler quantum Gaussian model consisting of a coherent state of a continuous variables (CV) system, whose mean is related to θ by a linear transformation. Below, we will show that the SLD measurement and the null measurement in the multi-copy model correspond to measuring canonical variables and respectively displaced number operators in the Gaussian picture.


More precisely, let  
θ_0 be a fixed parameter value and write θ= θ_0 + u/√(n) where u is a local parameter describing the deviation from θ_0. The latter can be thought of as being the first stage estimator θ̃_n, and will be used as such later on, but for the moment is considered to be fixed and known. 
To simplify the presentation we assume that ⟨ψ_θ_0 |ψ̇_θ_0⟩= 0, which can always be satisfied by appropriately choosing the phase of |ψ_θ⟩ around θ_0. Let {|0⟩, … , |d-1⟩} be an orthonormal basis such that 
|0⟩ = |ψ_θ_0⟩. 
Then in the first order of approximation we can write the state as 

    |ψ_θ⟩   =   
    |0⟩  + u/√(n) |ψ̇_θ_0⟩
    +O(n^-1)
       =   
    |0⟩  + u/√(n)∑_j=1^d-1 c_j |j⟩ + O(n^-1),   
    c_j = ⟨ j |
    ψ̇_θ_0⟩ .

The SLD is given by 

    ℒ_θ_0  = 
    2(|ψ_θ_0⟩⟨ψ̇_θ_0 | + 
    |ψ̇_θ_0⟩⟨ψ_θ_0 |)

and the QFI is 
F_θ_0= 4ψ̇_θ_0^2 = 4 c^2, where c= 
(c_1,…, c_d-1)∈ℂ^d-1.

We now construct a Gaussian model which approximates the multiple copies model in the neighbourhood of θ_0. Let 
ℱ_d-1=ℱ^⊗(d-1) be the Fock space of d-1 CV modes with annihilation operators 
(a_1,… ,a_d-1), and let | n⟩ = ⊗_j=1^d-1 |n_j⟩ be the Fock basis, where n= (n_1, … , n_d-1)∈ℕ^d-1. We further denote by 
| z⟩ = ⊗_j=1^d-1 |z_i⟩ a coherent state with amplitude z= (z_1,… , z_d-1)∈ℂ^d-1 such that 
⟨ z |a_j| z⟩ =z_j. On the Fock space we consider the quantum statistical model with parameter 
u∈ℝ consisting of coherent states

    | cu⟩ = ⊗_j=1^d-1 |c_j u⟩.

We now show (in three different ways) that the multiple copies model  
|ψ^n_u⟩ := |ψ_θ_0+u/√(n)^⊗ n⟩, which capture the local properties of the original model around θ_0 in terms of the rescaled parameter u, is approximated by the Gaussian model | cu⟩. Firstly, one can show that the collective variables 

    Q^n_j   : =   1/√(2n)∑_i=1^n(|0⟩⟨ j| + |j⟩⟨ 0|)_j
    
    P^n_j   : =   1/√(2n)∑_i=1^n( -i|0⟩⟨ j| +i |j⟩⟨ 0|)_j

 converge in distribution to the canonical variables Q_j, P_j of the CV Gaussian model, with respect to the state | cu⟩. Secondly, the convergence holds on the geometric level, i.e. it can be expressed in terms of the overlaps 

    lim_n→∞⟨ψ^n_u| ψ^n_v⟩ =
    ⟨ cu |  cv⟩.

where u,v are fixed local parameters. The reader will recognise that for d=2 these statements are statistical reformulations of results from coherent spin states <cit.>. 




The third type of QLAN convergence is akin to the Le Cam strong convergence of statistical models in classical statistics <cit.>; it states that the multi-copy and Gaussian models can be mapped into each other operationally by means of quantum channels. Let 
𝒮_n be the symmetric subspace of 
(ℂ^d)^⊗ n and let V_n: 𝒮_n→ℱ_d-1 be the isometry 

    | n; n⟩↦ | n⟩

where | n; n⟩ is the normalised projection onto S_n of the basis vector (⊗_j=1^d-1 |j⟩^⊗ n_j )⊗|0⟩^n-| n|∈(ℂ^d )^⊗ n. Then the following holds for any 0<γ<1 <cit.>

    lim_n→∞sup_|u|≤ n^1-γ V_n |ψ^n_u⟩  - | cu⟩ =0

which means that the original model can be mapped isometrically (with no loss of information) into the Gaussian model with vanishing asymptotic error, uniformly over all the relevant local parameters. 


From the estimation perspective, QLAN can be used to devise an asymptotically optimal, two steps measurement strategy for the multi-copy model. 
We first compute a preliminary estimator θ̃_n by using a small sub-sample and set θ_0 =θ̃_n. Subsequently, we map the ensemble state into the (approximately) Gaussian one by applying the isometry V_n, and then measure the SLD a c^† +  c a^† of the Gaussian shift model, which is the limit of 1/√(n)∑_i=1^nℒ^(i)_θ_0 where ℒ_θ_0 is the SLD (<ref>). Joining the two estimation stages, we compute the final estimator as
θ̂_n = θ̃_n +û_n/√(n) where 
û_n = X/F_θ̃_n and X is the result of the second stage measurement. Similarly to  (<ref>), the estimator
θ̂_n defined above is optimal in the sense that it achieves the QCRB asymptotically, cf. equation (<ref>).




We now move away from SLD measurements and consider the QLAN perspective on displaced-null measurements. An important point here is that the parameter u of the Gaussian shift model |  cu⟩ is not completely arbitrary but can be considered to be bounded by n^ϵ, where n is the sample size of the qudit ensemble. Indeed, recall that 
the reference parameter θ_0 should be thought of as a preliminary estimator θ̃_n obtained by measuring a sub-sample of ñ= n^1-ϵ qudits. 
Assuming this has standard concentration properties i.e. |θ -θ̃_n|< n^-1/2 +ϵ with high probability, and writing θ= θ̃_n+u/√(n) we obtain |u|≤ n^ϵ <cit.>. 



While the collective SLD observable in the qudit model maps onto a quadrature of the limit model, a measurement in the null basis 
{ |e_0 ⟩≡|ψ_θ_0⟩, … ,|e_d-1⟩} corresponds to a simultaneous measurement of the number operators N_j = a_j^*a_j, for all the CV modes a_j. More precisely, in the limit of large n the joint distribution of the counts {N_j,n}_j=1^d-1 for outcomes corresponding to vectors {|e_j⟩}_j=1^d-1, converges to the joint distribution of the number operators {N_j = a_j^* a_j}_j=1^d-1 with respect to the coherent state | cu⟩, which is the product of independent Poisson distributions 
Poisson (|c_j|^2 u^2). Since the latter depends on u^2, the local parameter is non-identifiable so the measurement does not distinguish the parameters θ_±:= θ_0± u/√(n). Consequently, any final estimator has an error of the same order as the statistical error of the initial estimator, and is therefore far from optimal. Let us now change the reference point to 
θ_0 = θ̃_n - δ_n, with δ_n = τ_n/√(n) and τ_n = n^3ϵ. From this vantage point the limit model is 
 | c(u+ τ_n)⟩, and measuring in the displaced-null basis { |e_0 ⟩≡|ψ_θ_0⟩, … |e_d-1⟩} is again asymptotically equivalent to measuring the number operators {N_j}_j=1^d-1, with the difference that the limiting distribution is the product of Poisson(|c_j|^2(u+τ_n)^2). Since |u|<n^ϵ (with high probability) due to the preliminary estimation step, and τ_n≫ n^ϵ this means that u is uniquely determined by the measurement distribution.
 Moreover, for large n the Poisson distribution can be approximated by the Gaussian N(λ_i^n, λ_i^n ) where 
 λ_i^n= |c_i|^2(u+τ_n)^2. By expanding (u+τ_n)^2/τ_n and neglecting u^2/τ_n= O(n^-ϵ) we obtain that the rescaled variable
 
    Y_i,n = 1/|c_i|(N_i,n/τ_n -τ_n |c_i|^2)

converges in  distribution to N(2u|c_i|, 1). A simple signal to noise analysis shows that the best estimator of u is the linear combination

    û_n= 1/2 c^2∑_j=1^d-1 |c_j| Y_j,n = 1/2τ_n c^2∑_j=1^d-1N_j,n - τ_n/2

Its limiting distribution is N(u, F_θ^-1) where F_θ= 4 c^2 is the quantum Fisher information of the original model. In particular
the final estimator θ̂_n =θ̃_n+ û/√(n) achieves the QCRB in the sense of (<ref>).









§ PROOFS OF PROPOSITION <REF> 
AND COROLLARY <REF>
 




Proof of Proposition <ref>. Let α = (α^(1),…, α^(p)), β=(β^(1), … ,β^(q)) be the patterns with non-zero counts for n and respectively m; let (n_1, …, n_p) and   (m_1,…, m_q) be their counts, and let M be the maximum length of all patterns in α and β. 

Since pattern creation operators A_α^*(n) involve sums of σ^α_i for different positions i, the Fock state 
| n ;n⟩ is a superposition of vectors obtained by applying the following type of ordered products to the vacuum

    ∏_k=1^n_pσ^α^(p)_i_p,k…∏_k=1^n_1σ^α^(1)_i_1,k

where i_j,k is the index marking the location of the left end of the k-th pattern α^(j), with j∈{1,…, p} and k∈{1,… , n_j}. In general, some of these operators may `overlap' (act on the same spin) and the computation of the superposition becomes cumbersome.
However, since the total length of the patterns is finite, for large n, the main contribution in the superposition comes from arrangements in which there are no overlapping patterns. Even more, we can restrict to arrangements where the patterns are separated by at least M zeros, in which case each pattern  in the sequence of zeros and ones can be identified unambiguously. 








Therefore

    | n;n⟩
          =
    1/√(n^| n|)1/√( n!)∑_ i∈ℐ( n;n) ∏_j=1^p 
     ∏_k=1^n_j σ^α^(j)_i_j,k 
    |Ω_n  ⟩ + o(1) 
    
          =
    1/√(n^| n|)1/√( n!)∑_ i∈ℐ( n;n)  
    |ω( i,  n)⟩
    + o(1)

where n!:=n_1!… n_p! and  ℐ( n;n) is the subset of locations i={ i_j,k} leading to non-overlapping patterns with counts set n, such that all patterns are at distance of at least M from each other, and ω( i,  n)∈{0,1}^n is the basis vector (trajectory) obtained by placing the all the patterns corresponding to the counts set n at locations prescribed by i. More precisely i∈ℐ( n;n) if for any two pairs (j,k) and (j̃, k̃) with 
i_j,k≤ i_j̃,k̃ one has 
    i_j,k+ |α^(j)|+M-1 <i_j̃, k̃. The basis vector |ω( i,  n)⟩ consist of zeros except patterns α^(j) written at positions i_j,k for j∈{1,… ,p} and k∈{1,… , n_j}. 
    
    
    To show that the remainder term is o(1) note that any term in the superposition | n  ,n ⟩ corresponding to a specific product of σs is a vector of the standard basis, and any such vector 
    |ω⟩ has at most a fixed number M· | n| of 1s. Note also that the action of applying a pattern 
    σ^α_i to the vacuum cannot be reversed by applying subsequent σs, since these contain only creation operators. This means that for a given 
    |ω⟩, the locations i ={i_j,k} of the σs  producing this vector are limited to an area of size 2 M around each 1 in the sequence, or in other words, the coefficient 
    of |ω⟩ is bounded by (2 M)^M| n|. On the other hand, the number of basis vectors 
    |ω⟩ obtained by applying patterns such that at least two overlap or are at distance smaller than M from each other is o(n^| n|) since the number of possible locations for two such σs is O(n). Therefore, the remainder term in (<ref>) is o(1).
     



Using equation (<ref>) we obtain

    ⟨ m;n | n;n⟩=
          1/√(n^| n|· n^| m|· n! · m!)∑_ i , ĩ⟨ω( i,  n)|ω(ĩ,  m) ⟩ +o(1)

where the sum runs over 
i∈ℐ( n;n) and 
ĩ∈ℐ( m;n). Since each i and ĩ uniquely determines the patterns it contains, the basis vectors |ω( i,  n)⟩  and 
|ω(ĩ,  m) ⟩ have non-zero overlap (coincide) only if their sets of patterns coincide, 
i.e. n=  m. Therefore, if n≠ m  hold then

    lim_n→∞⟨ m;n | n;n⟩ =0.


On the other hand,
if m =  n then

    lim_n→∞⟨ n;n | n;n⟩ = 
    lim_n→∞1/n^| n| n! ( n!)^2
    |ℐ( n,n)|/ n! =1

where we took into account that each basis 
vector |ω ( i,  n)⟩ appears n! times in the sum over i∈ℐ( n;n), and that

    lim_n→∞|ℐ( n , n)|/n^| n| =1 .





Proof of Corollary <ref>. As shown in the proof of Proposition <ref> the "Fock state" | n; n⟩ can be approximated by a superposition of basis states in which the patterns in n are non-overlapping and are situated at least at a certain distance from each other. Moreover the remainder term contains o(n^| n|) basis vectors with bounded coefficients. After applying A^*_β(n), the multiplicity of each basis vector |ω⟩ is finite and the number of possible basis vectors is 
o(n^| n|+1). Taking account of the factor n^-(| n|+1)/2 we find that the action of A^*_β(n) on the 
o(1) remainder in (<ref>) is still o(1). On the other hand, the action on the main term in (<ref>) is to add a pattern β separated by the other patterns by max(M, |β|), with a negligible term coming from locations in which β overlaps with or is too close to one of the existing patterns. The factor √(n_β +1) comes from the definition of the "Fock state" | n+ δ^(β);n⟩.

The action of A_β(n) on one of the vectors |ω( i,  n)⟩ is to produce a superposition of basis vectors in which the pattern 
β has been removed from the set 
n of patters, at all possible locations. This may include removing part of a existing pattern α^(j) which coincides with β. These two case will produce orthogonal vectors and can be evaluated separately. In the first case, the pattern β is removed from one of the locations where such pattern existed. 

In the Fourier decomposition of 
√(n_β)| n-δ^β; n⟩, 
a basis vector 
|ω⟩ given by non-overlapping patterns has coefficient 

    c(ω) = 
    √(n_β) n!/n_β√(n· n_β)/√(n^| n| n!) =
     n! √(n)/√(n^| n| n!)

Such a vector can be obtained in approximately n ways by removing a pattern β from a basis vector which appears in the decomposition of 
| n; n⟩. Therefore its coefficient in A_β | n, n⟩ is approximately

    n!/√(n)· n·1/√(n^| n| n!)

where we took into account a factor 1/√(n) from the definition of A_β. We therefore obtain that the coefficients of the non-overlapping terms in A_β | n; n⟩ and 
√(n_β) | n -δ^β; n⟩ agree asymptotically. The fact that the o(1) terms remain small after applying A_β can be shown similarly to the above.












§ PROOFS OF THEOREM <REF> 
AND LEMMA <REF>
 



In order to prove the theorem, we will make use of the method of moments; this can be done since the moment problem corresponding to the moments of Poisson and Gaussian random variables admits a unique solution (this can be seen for instance using Cramér condition, see <cit.>).

Number operators. We will show the convergence of the moments of N_α(n) in the state ρ^ out_u,n to those of a Poisson random variable of intensity λ(u):=u^2 λ_α. We recall the the r-th moment of a Poisson random variable of intensity λ(u) is equal to ∑_m=1^r S(r,m) λ(u)^m where S(r,m) are the Stirling numbers of the second type: a combinatorial interpretation of S(r,m) is the number of partitions in m non-empty subsets of a set of cardinality r. 





Let us focus on the expression of the r-th moment of N_α(n). For simplicity we 
denote ⟨ X⟩_u := ρ̃^ out_u,n(X). From (<ref>) we have

    ⟨ N_α(n)^r ⟩_u=1/n^r∑_i_1,…,i_r=1 
     j_1,…,j_r=1^n-|α|+1⟨σ^α_i_1σ^α*_j_1⋯σ^α_i_rσ^α*_j_r⟩_u.


Splitting of the sum based on non-overlapping groups of σs. Let us consider a term σ^α_i_1σ^α*_j_1⋯σ^α_i_rσ^α*_j_r and represent each σ^α_i or σ^α*_i  as a block of length |α| covering positions {i, i+1, … i+|α|-1} of the string {1,2,… n}.

Depending on the overlapping pattern of the blocks, the indices can be split (uniquely) in a number s= s(i_1, j_1,… ,i_r, j_r) of groups (1 ≤ s ≤ 2r) such that blocks in different groups do not overlap, and each group cannot be split into further non-overlapping sub-groups. Among these groups we identify  
g =g(i_1, j_1,… ,i_r, j_r) special groups characterised by the fact that they are made up of one or more pairs of blocks of the type (σ_z^α*σ^α_z) for some z. 

We call P_0 the set of such groups. Note that not all groups associated to a product of σs can be in 𝒫_0 because the order of σ_z^α* and σ_z^α in the latter is opposite to that in which such terms appear in N_α(n).

For example, consider the pattern α="11" for n=10 and the term

    σ^α_8σ_1^α *σ^α_6 σ^α*_8 σ^α_1 σ_2^α^*σ_2^ασ_5^α*

This has 3 non-overlapping groups of operators (that commute with each others): the first one

    σ^α*_1σ_1^ασ^α*_2  σ_2^α

belongs to P_0, while the second and  third ones 

    σ^α_6σ^α*_5
    
        and    σ^α_8 σ_8^α*

are not in P_0. Therefore s=3 and g=1.


Let us now look at the expected value of a given product of σs. We have

    ⟨σ^α_i_1σ^α*_j_1⋯σ^α_i_rσ^α*_j_r⟩_u=
    ρ^ ss_u ( C_u,1𝒯̃^x_1_u⋯𝒯̃^x_s-1_u C_u,s (1) )

where x_i's are the distances between the non-overlapping groups and C_u,i is the map corresponding to the i-th group, which is computed according to the following rule. In  every group we multiply the operators on a given position (we may have several σ^+, σ^- or σ^0 on one position) and the result will be an element of the set 𝒪:= {σ^+, σ^-, σ^0, |0⟩⟨0|, |1⟩⟨1|,  0}. For example the first group above gives

    σ^α*_1σ_1^ασ^α*_2  σ_2^α 
    =(|0⟩⟨0|)_1(|0⟩⟨0|)_2(|0⟩⟨0|)_3

while

    σ^α_6σ^α*_5
    =σ^-_5(|1⟩⟨1|)_6 σ^+_7

and 

    σ^α_8 σ_8^α*
    =(|1⟩⟨1|)_8(|1⟩⟨1|)_9.

Suppose the result of this computation is 
O^1_i⋯ O^k_i+k-1, with O^k∈𝒪; then in the expectation, this translates into a  superoperator 
C_u =  C_u[O^1]∘…∘ C_u[O^k] obtained by composing in the same order basic maps C_u[O] defined as follows

    C_u [σ^+]:=  A_u,1,    C_u [σ^0]:=  A_u,0,    C_u [σ^-]:=  A_u,-1,
        C_u [0]:= 0,
     C_u [|j⟩⟨j|]:=  B_u,j,    j∈{0,1}.

where A_u,i are defined as in equation (<ref>) by replacing K̃_i with K̃_u,i
and

    B_u,j(x)=K̃^*_u,1 xK̃_u,1    j=1 
    K̃^*_u,0xK̃_u,0    j=0 
    .

In what follows we will drop the label u when u=0.










We can now compute the expectation of our example product of σs as

    ⟨σ^α_8σ_1^α *σ^α_6 σ^α*_8 σ^α_1 σ_2^α^*σ_2^ασ_5^α*⟩_u=
       ρ^ ss_u (  B_u,0^3𝒯̃_u A_u,-1 B_u,1 A_u,1 B^2_u,1(1) ).



Let us define the space

    1^⊥:={x ∈ M(ℂ^D): ρ̃^ ss(x)=0},

and note that the following properties hold true:

    
  * A_0 and B_0 leave 1^⊥ invariant;
    
  * the image of A_i and B_j  for i,j ≠ 0 is contained in 1^⊥;
    
  * for every m ≥ 1, the norm of A^m_0=𝒯̃^m restricted to 1^⊥ is less or equal than cλ^m for some c>0 and 1>λ>0. 
    


The reason why we singled out groups in P_0 is given by the following key observation: such groups are the only ones for which  the corresponding map C_u,i is only composed by A_u,0s and B_u,0s. Any other group will contain at least one 𝒜_u,± 1 or ℬ_u,1 factor.  From this and the properties above it follows that if 𝒞 is not in 𝒫_0 then 

    𝒯̃^m ∘𝒞≤ cλ^m.



We will often identify the maps with the corresponding patterns saying, for instance, that the map is in P_0 when the corresponding group is. 




We will prove the convergence of moments by expanding in Taylor series in u and showing the convergence at each order. 
The moment of order r of the Poisson distribution with intensity 
u^2λ_α is 

    m_r = ∑_k=1^r S(r, k) u^2kλ_α^k

where S(r, k) are the Stirling numbers of second kind; the mth derivative with respect to u at u=0 is 
S(r, m/2)m!λ_α^m/2 for m even, and zero otherwise. 

Recall that each term in (<ref>) gives rise to 
a a certain set of groups of indices made up of overlapping blocks, and each group corresponds to a map  C_u,i such that the expectation is expressed as in equation in (<ref>). We will show that in limit of large n, the only terms which contribute to the m-th derivative are those coming from certain configurations with s-g=m where s is the number of groups and g is the number of groups in 𝒫_0. 

 
Taylor approximation for a given set ( C_u,1, … C_u,s) up to order m=s-g.
 Let us consider the sum of all the terms in Eq. (<ref>) coming from all the correlations corresponding to a given sequence of maps 𝒞:=
 ( C_u,1, … , C_u,s). This contribution is given by the product between a combinatorial factor (independent of n) counting how many products of σs in (<ref>) lead to the same set of maps 
 𝒞 and the following sum

    1/n^r∑_x_0+… +x_s = n-Kρ̃^ ss_u
     ( C_u,1𝒯̃^x_1_u⋯𝒯̃^x_s-1_u C_u,s (1) ),

where K is the total length of all the s blocks in  𝒞, which is smaller than 2|α| r. 
Note that the factors 𝒯̃^x_0_u and 
𝒯̃^x_s_u have been suppressed due to stationarity but the indices x_0, x_s are still present in the sum. We denote by g the number of maps in ( C_u,1, …,  C_u,s) that belong to P_0. 

Let us now consider the Taylor expansion of the correlations in (<ref>). The 0-th order term is 0 because at least one C_i is not in P_0 and hence ρ̃^ ss annihilates the result. This is because on one hand, C_i*(ρ̃^ ss)=ρ̃^ ss for all C_i∈𝒫_0, but on the other hand at least one  C_i is not in 𝒫_0 and hence it contains a term 𝒜_±1 or ℬ_1 for which 𝒜_± 1 *(ρ̃^ ss)=ℬ_1* (ρ̃^ ss) =0 since K̃_1|χ^ ss⟩ =0. 

Before addressing in detail the derivatives of (<ref>) we make some remarks concerning the magnitude of the sum. Note that this contains O(n^s) terms which are uniformly bounded and the largest possible value of s is 2r (all patterns are non-overlapping). On the other hand the sum is preceded by the factor n^-r and any derivative further multiplies it by n^-1/2 since all operators depend on u via u/√(n). These arguments alone are not sufficient to deduce the convergence of (low order) derivatives, at least for configurations 𝒞 with large s. 

The key additional ingredient is the fact that for 𝒞_i∉𝒫_0, the factors 𝒯̃^x_i𝒞_i are exponentially decreasing, cf. equation (<ref>). This will provide more conservative upper bounds for the derivatives of the sum (<ref>), as detailed below. Before considering that, note that the exponential bound can also be used to get an alternative proof of the convergence to 0 of

    1/n^r∑_x_0+… +x_s = n-K C_u,1𝒯̃^x_1_u⋯𝒯̃^x_s-1_u C_u,s (1).

Indeed, by summing over x_is for groups not in 𝒫_0,  we get that the sum (<ref>) is O(n^g/n^r) rather than O(n^s/n^r). Since in any configuration 𝒞 , at least one group is not in 𝒫_0, and each group in 𝒫_0 has at least two of the original 2r blocks, we see that g<r so that the whole sum is O(n^-1). The upshot for derivatives will be that in order to create contributions that do not decay, the derivatives have to be applied in an "efficient" way, namely to factors of the type 𝒯^x_i∘𝒞_i with 
𝒞_i not in 𝒫_0. 
This will break the exponential decay and allow for the balancing of the terms in the derivative with the pre-factor n^-r.


 



Consider now the case of the first order derivative of the sum (<ref>). This will split into a sum of sub-sums, one for each of the terms 𝒞_u,i or 
𝒯^x_i_u that are differentiated. Each sub-sum will be shown to have a decaying contribution to the derivative.



  i) 
If a term 𝒞_u,i in 𝒫_0 is differentiated, then the sub-sum is similar to the original sum with the difference that 𝒞_u,i is replaced by another bounded term 𝒞̇_i). The overall contribution is then of the order n^-r× n^g× n^-1/2 which decays since g<r.


  ii) 
If a term 𝒯̃^x_i-1_u is differentiated, for which 𝒞_u,i is in 𝒫_0 then the product 𝒯̃^x_i-1_u 𝒞_u,i becomes the sum 

    S_i = ∑_l=1^x_i-1𝒯̃
    ^x_i-1-l𝒯̇̃̇𝒯̃^l-1 C_i(·)

By ergodicity 𝒯̃^l-1(𝒞_i(·)) converges exponentially fast to ρ̃^ ss(𝒞_i(·))1 for large l so 

    S_i = 
    ∑_l^'=1^x_i-1𝒯̃^l^'(𝒯̇̃̇(1)) ·ρ̃^ ss(𝒞_i(·)) + O(1)

By the same argument 𝒯̃^l^'𝒯̇̃̇(1) converges to ρ̃^ ss(𝒯̇̃̇(1)) = 0 for large values of l^'. The latter follows from differentiating ρ̃^ ss_u (𝒯̃_u(·)) = ρ̃^ ss_u(·). Therefore S_i is O(1) which does not change the magnitude of the overall sum before differentiation, so this contribution decays as well, as argued in point i).


  iii) 
If a term 𝒯̃^x_i-1_u is differentiated, for which 𝒞_u,i is not in 𝒫_0, one obtains a sum S_i as in equation (<ref>). Since 𝒞_i leaves 1^⊥ invariant we have 
𝒯̃^l ∘𝒞_i≤ c λ^l so the terms with large l become negligible while for small l (large x_i-1-l) 
𝒯̃^x_i-1-l(·) converges to 
ρ̃^ ss(·)1. Therefore, using 
∑_l^' =0^∞𝒯̃^l^'𝒞_i = ℛ̃𝒞_i where ℛ is the Moore-Penrose inverse of Id- 𝒯̃, we obtain

    S_i=ρ^ ss(𝒯̇̃̇ℛ̃ C_i(·)) 1+o(1).

Hence differentiating 𝒯̃^x_i-1_u for which 𝒞_u,i is not in 𝒫_0 removes the exponential decay associated to un-differentiated  term 𝒯̃^x_i-1_u𝒞_u,i so that the upper bound for this contribution to the derivative is 
n^g+1n^-1/2n^-r. Since g<r we have 
g+1 -1/2-r<0 so the contribution to the derivative 
decays (even though it decays slower than the previous sub-sums, which will be important when considering higher derivatives).






  iv)
If a term 𝒞_u,i not in 𝒫_0 is differentiated then for large x_i-1

    𝒯̃^x_i-1Ċ_i(·)=ρ̃^ ss(Ċ_i(·))1 + o(1)

This means that such a derivative breaks the exponential decay of the product 𝒯̃^x_i-1 C_i but the contribution to the sum is still bounded by  
n^g+1n^-1/2n^-r.




The upshot of the argument for the first order derivative is that differentiating a 𝒞_u,i in 𝒫_0 or the factor 𝒯^x_i-1 in front of it, does not increase the magnitude of the sum and the contribution to the overall sum decays. On the other hand, differentiating a 𝒞_u,i which is not in 𝒫_0 or the factor 𝒯^x_i-1 in front of it, breaks the overall exponential decay of the product and contributes with an additional factor n^1/2 compared to the un-differentiated term.



We now consider higher order derivatives. 
One can again consider the possible positions where derivatives are applied and evaluate their separate contributions to the derivative. Following the same argument as in point i) above, one can see that differentiating a term 𝒞_u,i in 𝒫_0 once or multiple times does not bring any change compared to the term before differentiation. Similarly, as in point ii) one finds that differentiating 𝒯^x_i-1_u in front of 𝒞_u,i which is in 𝒫_0, once or multiple times does not bring any change (this is due to the fact that d^k/du^k𝒯_u^x(1)=0 for every k ≥ 1). Now we move our attention to terms 𝒯^x_i-1_u𝒞_u,i for which 𝒞_u,i is not in 𝒫_0. As in cases iii) and iv) above, differentiating either one of the two factors will break the exponential decay and bring an extra overall multiplicative factor n^1/2 compared to the un-differentiated terms. However if any of the two factors is differentiated further (more than one derivative in the product 𝒯^x_i-1_u𝒞_u,i) then the additional derivative does not change the bound except for the multiplicative factor n^-1/2 due to differentiation. This shows that in order to obtain contributions that do not vanish asymptotically, the derivatives have to be placed 
on the products 𝒯^x_i-1_u𝒞_u,i for which 𝒞_u,i is not in 𝒫_0, with at most one derivative for each product.


Recall that we are considering derivatives up to order 
m=s - g which is equal to the number of 𝒞_u,i not in 𝒫_0. According to the argument above, the most favourable positions for the derivatives is on different terms 𝒯^x_i-1_u𝒞_u,i. Therefore, for a derivative of order k≤ m the entire derivative can be upper bounded as 

    n^-r× n^-k/2× n^g× n^k =
    n^-r+g+k/2

Since k≤ m = s-g the exponent is smaller than 
t:= -r+g+ (s-g)/2. We will show that t≤ 0 with equality if and only if m= s-g and the configuration 𝒞 is such that all groups in 𝒫_0 consists of 2 perfectly overlapping σs and the groups not in 𝒫_0 consists of single blocks. Indeed for given total number 2r of σ 
blocks, s+g is maximum if all groups in 𝒫_0 have just two σ blocks and all others have a single block, so 2g+ s-g = s+g ≤ 2r, which implies t ≤ 0. In conclusion, for any configuration 𝒞, the derivatives below order m=s-g decay asymptotically, and that of order m does as well unless 𝒞 is of the special type described above. Derivatives of order above m will be treated below. Note that for the special configurations, m is even so all odd derivatives decay, as is expected from the fact that the intensity of the limit Poisson distribution is proportional to u^2. We will now compute the 
limit of the mth derivative for the special configuration. 

If 𝒞_i∈𝒫_0 consists of two blocks of σs then the corresponding product is

    σ^α *σ^α= O_α_1… O_α_|α|

where 
α=(α_1, …, 
α_|α|) and O_0 := 1,  
O_1= |0⟩⟨ 0|. Therefore 

    𝒞_i = C^(α_1)… 
    C^(α_|α|)

with C^(0) = 𝒜_0 = 𝒯 and C^(1)= ℬ_0.
Therefore when x_1,…, x_s are large the following holds

    𝒯^x_i-1 C_i(·) = 
    ρ^ ss( C_i (·)) 1
     + o(1) =  ρ^ ss(· ) 1 + o(1)

since 
ρ^ ss∘𝒜_0 =ρ^ ss and ρ^ ss∘ℬ_0 =ρ^ ss
This means that for large x_1,…, x_s the derivative of order 
m=2(m/2) factorises as 

    |ρ^ ss(𝒯̇ℛ A^α(1)+ρ^ ss(Ã^α(1))|^2(m/2)

where Ã^α is obtained from A^α differentiating the first A_1. This formula follows from the contributions obtained at points iii) and iv) above and the fact that the groups that are differentiated contain a single σ block.

We remark that the term for which we computed the limit appears in the m-th derivative with a factor m! in front.

Finally, we need to compute the combinatorial factor counting the number of terms in the expectation which produce the desired set 𝒞 consisting of g= r-m/2 groups of two σs in 𝒫_0 plus 
m groups of single σs (with s=r+m/2 total number of groups). 

We will show that this is exactly S(r,m/2) (Stirling number of second type) i.e. the number of partitions of a set of r elements into m/2 non-empty subsets. For this it is enough to show that the numbers of ways in which we can pair the σs is in a bijection with the partitions in m/2 classes of r elements. Consider the collection of σs

    σ_i_1^ασ_j_1^α*⋯σ_i_r^ασ_j_r^α*,

which, for our purposes, we can identify with the set of "dipoles"

    (α, α*)_1, ⋯, (α, α*)_r

In order to create a C in P_0, we need to pair a σ_j_z^α* with a σ_i_w^α such that z<w. This induces an equivalence relation on the collection of dipoles where we identify two dipoles (α, α*)_z, (α, α*)_w if j_z=i_w and we impose transitivity of this relation. By construction,  in each equivalence class 
{ (α, α*)_z_1,(α, α*)_z_k} we have j_z_l = i_z_l+1 so all positions of α*s are equal to the position of α in the next dipole. In other words, an equivalence class uniquely determines a set of pairs of equal indices, and altogether the set of equivalence classes uniquely determine the splitting into 𝒫_0 groups. Since i_z_1 and j_z_k are the only indices that are not paired in the chosen equivalence class, the number of classes is m/2, i.e. half of the number of groups not in 𝒫_0. The number of ways to group the r dipoles in m/2 equivalence classes is the Stirling number of second type S(r,m/2) which provided the combinatorial factor for our mth order derivative.

With this we conclude that the mth order derivative of (<ref>) converges to the corresponding derivative of (<ref>). 



Reminder of the Taylor approximation (order s-g+1). The last thing we need to check is that the remainder corresponding to the sum of all the terms in Eq. (<ref>) with s non-overlapping blocks is negligible; the reminder is given by

    u^m^'/m^'!n^r+m^'/2∑_x_1+… +x_s = n-K .d^m^'/du^m^'f_x_1,…,x_s(u) |_u=η_x_1,…,x_s-1

for some |η_x_1,…,x_s-1|≤ |u|/√(n), where m^'=s-g+1 and

    f_x_1,…,x_s(u)=ρ^ ss_u ( C_1,u𝒯^x_1_u⋯𝒯^x_s-1_u C_s,u (1) ).

Notice that r+m^'/2>s: indeed,

    r+s-g+1/2>s ⇔ 2r+1>g+s.

Therefore, it is enough to show that

    .d^l+1/du^l+1f_x_1,…,x_s(u) |_u=η_x_1,…,x_s-1,    2 ≤ l ≤ 2r

are uniformly bounded: the only terms that requires some care are the derivatives of 𝒯^x_u, which are of the type

    ∑_0≤ l_1 ≤⋯≤ l_k ≤ x𝒯_η^x-l_k𝒯_η^(m_k)⋯𝒯_η^(m_1)𝒯_η^l_1-1,

where 𝒯^(m)_η stays for the m-th derivative of 𝒯_u evaluated at η. Using

    
  * the spectral decomposition of 𝒯_η(·)=ρ^ ss_η(·)1+ℛ_η, with ρ^ ss_η(ℛ_η(·))=0 and ℛ_η≤λ<1 (for n big enough) and
    
  * the fact that 𝒯^(m)(1)=0 (m ≥ 1),

we have that

    ∑_0≤ l_1 ≤⋯≤ l_k ≤ x𝒯_η^x-l_k𝒯_η^(m_k)⋯𝒯_η^(m_1)𝒯_η^l_1-1= 
       ∑_0≤ l_1 ≤⋯≤ l_k ≤ x𝒯_η^x-l_k𝒯_η^(m_k)ℛ_η^l_k-l_k-1-1⋯𝒯_η^(m_1)ℛ_η^l_1-1≤
       C∑_0≤ l_1 ≤⋯≤ l_k ≤ xλ^l_k,

which is bounded.




















  
  
  



 
    



























Quadratures. For the sake of keeping notation simple, we will show the proof in the case of z=1, but the same reasoning applies to the general case inserting z and z where needed. We recall that k=|α|.

First of all, notice that the mean of Q_α(n) converges to uμ_α,1: indeed, its first moment is given by

    1/√(n)∑_i=1^n-k+1ρ̃^ ss(u/√(n)) (   √(2)( A^α) (1) )  → u μ_α,1.


In order to simplify the proof, we consider the standardised random variable

    √(2) (Q_α(n) - n-k+1/nu μ_α, 1 )=
       A^*_α(n)+A_α(n)-2(μ_α)u=
       1/√(n)∑_i=1^n-k+1σ̃^α_i(u) + σ̃^α*_i(u),

where

    σ̃^α(u)=σ^α-u/√(n)ρ((𝒯̇ℛ A^α+Ã^α)(1))1.
 In order to prove the statement, we need to show that the sequence of standardized quadratures converges in law to a standard Gaussian random variable.


For r ≥ 2, the r-th moment of the standardised random process at time n has the following expression:

    ∑_j_1,…,j_r ∈{α,α*}1/n^r/2∑_i_1,…,i_r =1^n-k+1⟨σ̃^j_1_i_1⋯σ̃^j_r_i_r⟩_u/√(n).


Consider the correlation corresponding to a choice of indices i_1,…,i_r such that there are exactly s non-overlapping groups and let g be the number of groups with overlapping terms of the form σ̃_z^α*(u)σ̃^α_z(u) (as before, we denote the set of such groups as P_0). As before, these are the only overlapping groups that at u=0 will produce an operator which is not in 1^⊥.

First suppose that s-g>0. We can prevent the maps which do not belong to P_0 to be annihilated by ρ and to cause an exponential decay differentiating and repeating the same computations as in the case of number operators. If we consider the m-th term in the Taylor expansion up to m=s-g, we can see that it grows at most as n to the power g+m/2-r/2; if we want the exponent to be bigger or equal than 0, we need that m≥ r-2g; since m≤ s-g, one realises that g+m/2-r/2 can at most be equal to 0 and this is true when m=s-g and s+g=r, which means that every group in P_0 is of the form σ̃_z^α*(u)σ̃^α_z(u) and all the other groups are singletons. However, if a group is composed by a single element, since we centered the random process, the first derivative will not be enough to cancel the exponential decay. Therefore, if s-g>0, the corresponding terms will decrease to 0.

On the other hand, if r is even and s=g=r/2, the 0-th order term is equal to 1, hence the leading term comes from the case when r is even and g=r/2. In this case one can see that the limit quantity is equal to 1 times the way we can pair the σ̃(u)'s in groups of the type σ̃_z^α*(u)σ̃^α_z(u) and this is given by (r-1)!! (which is the number of partition into pairs of a set of r elements).

The reminder can be controlled as in the case of number operators.





Alternative expression for μ_α. 
Using that K̃_0|χ^ ss⟩=|χ^ ss⟩ and K̃_1|χ^ ss⟩=0, one can write

    𝒯̇̃̇_*(ρ̃^ ss)=   ∑_i=0^1K̇̃̇_i|χ^ ss⟩⟨χ^ ss|K̃_i^* + K̃_i|χ^ ss⟩⟨χ^ ss|K̇̃̇^*_i
    
    =   |K̇̃̇_0χ^ ss⟩⟨χ^ ss| + |χ^ ss⟩⟨K̇̃̇_0χ^ ss|.


Under the "gauge condition (<ref>) 

and using the explicit expression of K̃_θ,i's in Eq. (<ref>) we obtain ⟨χ^ ss|K̇̃̇_0 χ^ ss⟩=0.


 



From 
K̃_0^*K̃_0 + K̃_1^*K̃_1 = 1 we obtain K̃_0^*|χ^ ss⟩ = |χ^ ss⟩.
Therefore 

    K̃_0 = |χ^ ss⟩⟨χ^ ss| + P^ ss_⊥K̃_0 P^ ss_⊥
 
where P^ ss_⊥ = 1- |χ^ ss⟩⟨χ^ ss|, which implies

    ∑_k ≥ 0K̃_0^k|K̇̃̇_0 χ^ ss⟩= |(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩

and

    ℛ̃_*
    𝒯̇̃̇_*
    (ρ̃^ ss)
       =   |(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨χ^ ss|
       +   |χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|.

When we evaluate it against
A^α(1), the first term in the previous equation gets killed, while the second one produces the term

    ⟨K̃_α_|α|⋯K̃_α_1(1-K̃_0)^-1K̇̃̇_0 χ^ ss | χ^ ss⟩.

The rest of the proof is just a trivial check.

Expression for the total intensity. Since λ_α≥ 0, one has that ∑_αλ_α =C ∈ [0,+∞] and the limit is always the same irrespectively of the choice of partial sums. Notice that

    λ_(1)   =   |⟨ (K̃_1(1-K̃_0)^-1K̇̃̇_0 + K̇̃̇_1)χ^ ss|χ^ ss⟩|^2
       =   (|χ^ ss⟩⟨χ^ ss|Y),

where

    Y   :=   |(K̃_1(1-K̃_0)^-1K̇̃̇_0 +K̇̃̇_1)χ^ ss⟩
          ⟨(K̃_1(1-K̃_0)^-1K̇̃̇_0 +K̇̃̇_1)χ^ ss|.


For any α such that |α|≥ 2, λ_α is equal to:

    | ⟨K̃_α_|α|⋯K̃_α_2 (K̃_α_1(1-K̃_0)^-1K̇̃̇_0 +K̇̃̇_1)χ^ ss | χ^ ss⟩|^2=
       | ⟨K̃_1K̃_α_|α|-1⋯K̃_α_2 (K̃_1(1-K̃_0)^-1K̇̃̇_0 +K̇̃̇_1)χ^ ss | χ^ ss⟩|^2=
       (X K̃_α_|α|-1⋯K̃_α_2YK̃^*_α_2⋯K̃^*_α_|α|-1 ),

where X:=|K̃_1^*χ^ ss⟩⟨K̃_1^*χ^ ss| and Y is the same as in Eq. (<ref>).

Therefore

    ∑_ 2 ≤ |α| ≤ Nλ_α= (∑_k=0^N-1𝒯̃^k(X)Y  )

and

    lim_N→ +∞∑_ 2 ≤ |α| ≤ Nλ_α=∑_ 2 ≤ |α|λ_α=(ℛ̃(X)Y),

therefore

    ∑_αλ_α =(( Id+ℛ̃ B_1)(|χ^ ss⟩⟨χ^ ss|)Y).

Let us massage a little the expression we obtained:

    (( Id+ℛ̃ B_1)(|χ^ ss⟩⟨χ^ ss|)Y)=
       ⟨χ^ ss|Yχ^ ss⟩ + ⟨χ^ ss| B_1*ℛ̃_*(Y-(Y)|χ^ ss⟩⟨χ^ ss|) χ^ ss⟩=
       ⟨χ^ ss|Yχ^ ss⟩ + ⟨χ^ ss|𝒯̃_*ℛ̃_*(Y-(Y)|χ^ ss⟩⟨χ^ ss|)χ^ ss⟩-
       ⟨χ^ ss|ℛ_*(Y-(Y)|χ^ ss⟩⟨χ^ ss|)χ^ ss⟩=
       ⟨χ^ ss|Yχ^ ss⟩ -⟨χ^ ss|(Y-(Y)|χ^ ss⟩⟨χ^ ss|) χ^ ss⟩=(Y).

We used the fact that B_0(|χ^ ss⟩⟨χ^ ss|)=|χ^ ss⟩⟨χ^ ss| and that 𝒯̃_*ℛ̃_*=ℛ̃_*- Id. Finally,

    (Y)=(K̃_1(1-K̃_0)^-1K̇̃̇_0+K̇̃̇_1)χ^ ss^2






§ PROOF OF COROLLARY <REF>
 





We will use the expression of ∑_αλ_α given by

    -(⟨χ^ ss, 2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0+K̃_0 χ^ ss⟩ )

as states in equation (<ref>) in Theorem <ref>. The proof of this identity can be found in Appendix <ref>; the expression we use here has the advantage that it immediately shows that ∑_αλ_α does not change for different choices of the postprocessing. Before proceeding, we recall the expression of all the terms appearing in the previous equation using quantities of the dynamics of the system alone.



  * |χ^ ss⟩=∑_i=1^d√(λ_i)|φ_i⟩_S ⊗|φ_i⟩_A, where ρ^ ss=∑_i=1^dλ_i |φ_i⟩⟨φ_i| is the spectral resolution of the stationary state of the system dynamics;

  * The Kraus operator K̃_0 is uniquely determined as follows (left tensor is the system)

    K̃_0=∑_k=0^1∑_i,j=1^d√(λ_j/λ_i)⟨φ_j,K_k^*φ_i ⟩ K_k ⊗|φ_i⟩⟨φ_j|.


  * The derivative of K̇̃̇_0 is (where we keep in mind than only the system unitary depends on θ)

    K̇̃̇_0=∑_k=0^1∑_i,j=1^d√(λ_j/λ_i)⟨φ_j,K_k^*φ_i ⟩K̇_k ⊗|φ_i⟩⟨φ_j|,


  * The second derivative is

    K̃_0=∑_k=0^1∑_i,j=1^d√(λ_j/λ_i)⟨φ_j,K_k^*φ_i ⟩K_k ⊗|φ_i⟩⟨φ_j|.



Note that

    ⟨χ^ ss, K̃_0 χ^ ss⟩   = ∑_k=0^1∑_i,j=1^dλ_j ⟨φ_i, K_k φ_j ⟩_S ⟨φ_j,K_k^* φ_i ⟩_S
       =∑_k=0^1(K_k ρ^ ssK^*_k).

Hence, using that ∑_k=0^1K_k^*K_k +K_k^*K_k +2K̇^*_kK̇_k=0, one has

    -(⟨χ^ ss, K̃_0 χ^ ss⟩)=∑_k=0(ρ^ ssK̇^*_kK̇_k).

Let us consider the rest of the total intensity: using that (1-K̃_0)^-1K̇̃̇_0|χ^ ss⟩=∑_l=0^+∞K̃_0^lK̇̃̇_0|χ^ ss⟩, one gets


    ⟨χ^ ss, K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩
          =∑_i,j=1^d∑_a,b=0^1∑_l=0^+∞∑_k_1,… k_l=0λ_j ⟨φ_i,K̇_a K_k_l⋯ K_k_1K̇_b φ_j ⟩·
          ·⟨φ_j,K_b^* K^*_k_1⋯ K_k_l^* K_a^* φ_i ⟩
          
        =∑_a,b=0^1∑_l=0^+∞∑_k_1,… k_l=0^1(K̇_a K_k_l⋯ K_k_1K̇_b ρ^ ssK_b^* K^*_k_1⋯ K_k_l^* K_a^* )
          = ( ∑_a=0^1K^*_a K̇_a ℛ ( ∑_b=0^1K̇_b ρ^ ss K_b^* ) ).

Therefore,

    -2(⟨χ^ ss, K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩)
          =2  ( (∑_a=0^1K^*_aK̇_a  ) ℛ ( ( ∑_b=0^1K̇_b ρ^ ss K_b^* )  )  ).





§ PROOF OF THEOREM <REF> AND PROPOSITION <REF>
 



First of all, let us show that λ_ tot given by Eq. (<ref>) is equal to ∑_αλ_α, whose expression is given by equation (<ref>). We have

    (K̃_1(1-K̃_0)^-1K̇̃̇_0+K̇̃̇_1)χ^ ss^2=
       ⟨ (1-K̃_0)^-1K̇̃̇_0χ^ ss,K̃_1^*K̃_1 (1-K̃_0)^-1K̇̃̇_0χ^ ss⟩
       +⟨ (1-K̃_0)^-1K̇̃̇_0χ^ ss,K̃_1^*K̇̃̇_1 χ^ ss⟩
       +⟨χ^ ss,K̇̃̇^*_1K̃_1 (1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩
       +⟨χ^ ss,K̇̃̇_1^*K̇̃̇_1 χ^ ss⟩.

Note that

    ⟨χ^ ss,K̇̃̇_1^*K̇̃̇_1 χ^ ss⟩=1/2⟨χ^ ss,  B_1(1) χ^ ss⟩=-1/2⟨χ^ ss,  B_0(1) χ^ ss⟩
       =-(⟨χ^ ss,K̃_0 χ^ ss⟩ )-K̇̃̇_0 χ^ ss^2.

Moreover, since K̃_0^*K̃_0 +K̃_1^*K̃_1 = 1,

    ⟨ (1-K̃_0)^-1K̇̃̇_0χ^ ss,K̃_1^*K̃_1 (1-K̃_0)^-1K̇̃̇_0χ^ ss⟩
       =(1-K̃_0)^-1K̇̃̇_0χ^ ss^2-K̃_0(1-K̃_0)^-1K̇̃̇_0χ^ ss^2.

As for the other terms, one has

    ⟨ (1-K̃_0)^-1K̇̃̇_0χ^ ss,K̃_1^*K̇̃̇_1 χ^ ss⟩
       =⟨ (1-K̃_0)^-1K̇̃̇_0χ^ ss,Ḃ_1(1) χ^ ss⟩
       =-⟨ (1-K̃_0)^-1K̇̃̇_0χ^ ss,Ḃ_0(1) χ^ ss⟩
       =-⟨K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0χ^ ss,χ^ ss⟩
       -⟨K̃_0(1-K̃_0)^-1K̇̃̇_0χ^ ss , K̇̃̇_0 χ^ ss⟩

and analogously

    ⟨χ^ ss,K̇̃̇^*_1K̃_1 (1-K̃_0)^-1K̇̃̇_0χ^ ss⟩
       =-⟨χ^ ss,K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0χ^ ss⟩
       -⟨K̇̃̇_0 χ^ ss,K̃_0(1-K̃_0)^-1K̇̃̇_0χ^ ss⟩.

Putting everything together, one gets

    ∑_αλ_α=(K̃_1(1-K̃_0)^-1K̇̃̇_0+K̇̃̇_1)χ^ ss^2
       =(1-K̃_0)^-1K̇̃̇_0χ^ ss^2-K̃_0(1-K̃_0)^-1K̇̃̇_0χ^ ss^2
       -2(⟨K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0χ^ ss,χ^ ss⟩)
       -2(⟨K̃_0(1-K̃_0)^-1K̇̃̇_0χ^ ss , K̇̃̇_0 χ^ ss⟩)
       -(⟨χ^ ss,K̃_0 χ^ ss⟩ )-K̇̃̇_0 χ^ ss^2
       =λ_ tot+(1-K̃_0)^-1K̇̃̇_0χ^ ss^2
       -(K̃_0(1-K̃_0)^-1+1)K̇̃̇_0χ^ ss^2=λ_ tot.












Let us now prove the first part of the theorem. What we are actually going to show is that for n → +∞ and |u| ≤ n^ϵ^', one has

    e^λ_totu^2ν_u,n (B_𝐦(n) )≍∏_i=1^k(λ_α^(i)u^2)^m_α^(i)/m_α^(i)!.

Let us consider a fixed ordered sequence of excitation patterns α^(1),…,α^(k) (here we do not require them to be distinct). For an observation time n big enough, the probability of observing such a sequence of patterns separated one from the other by more than n^γ consecutive 0s is given by


    ∑_x_1 +… +x_k+1 = n-K
    x_2,…,x_k≥ n^γ      ρ̃^ ss_u  ( B^x_1_u,0 B_u,α^(1) B^x_2_u,0⋯.
    
           .⋯ B^x_k_u,0 B_u,α^(k) B^x_k+1_u,0(1)) ).


where K=∑_i=1^k|α_i| and B_u,α(x):= B_u,α_1⋯ B_u,α_|α|(x).

The rest of the proof follows a similar line as the proof of Theorem <ref>: we study the Taylor expansion of the series and identify the leading terms in the limit n → +∞. We will often use the spectral decomposition of B_0, i.e.

    B_0(x)=ρ̃^ ss(x) |χ^ ss⟩⟨χ^ ss| + 
     E_0(x)

such that

    
  * for any k ≥ 1, for  some constants C>0, 0 <λ<1, one has E_0^k≤ Cλ^k,
    
  * ρ̃^ ss( E_0(·))=0 and 
    E_0(|χ^ ss⟩⟨χ^ ss|)=0.

Notice that, in general, ρ̃^ ss(u) and the eigenvector of B_u,0* corresponding to the spectral radius only coincide for u=0.

We will also use the following identities


  * ρ̃^ ss (𝒯̇̃̇ℛ̃ B_0^x B_α(· )) =0 follows from (<ref>) and the fact that K̃_1|χ^ ss⟩=0.

  * ρ̃^ ss(Ḃ_α(·))=0 follows from K̃_1|χ^ ss⟩=0

  * ρ̃^ ss(ℬ̇_0(·)) =0 follows from the "gauge condition" (<ref>) 

and using the explicit expression (<ref>).




Significant terms in the Taylor approximation (up to the 2k-th term).
We will show that up to derivatives of order 2k the only contribution in the Taylor expansion of (<ref>) which does not decay with n is that coming from the part of the order 2k derivative in which one takes the second order derivative to each of the blocks B^x_i_u,0 B_u,α^(i), for i=1,… k. This follows from the observations below:






1) the first derivative of (<ref>) at u=0 is zero since 
ρ̃^ ss( B_α(· ))=0 and the first derivative of ρ̃^ ss_u( B_u,0^x  B_u,α(·)) at u=0 is equal to

    ρ̃^ ss (𝒯̇̃̇ℛ̃ B_0^x B_α(· ))+∑_l=0^x-1ρ̃^ ss(Ḃ_0 B_0^l  B_α(·))+ρ̃^ ss(Ḃ_α(·))

and the three terms have been shown to be equal to zero above.





















2) Each block of the type 
B^x_u,0 B_u,α needs to be differentiated twice. Indeed if it is not differentiated at all, then it will bring an exponential decaying contribution since 

    ∑_n^γ≤ x ≤ n B^x_0 B_α≤ C ∑_n^γ≤ x ≤ nλ^x=C(λ^n^γ-λ^n+1)/1-λ→ 0 ;

This follows from the fact that since α contains at least one 1, we have ρ̃^ ss( B_α (·)) =0 which means that B_α(·) belongs to the subspace on which 
ℬ_0 acts as the strict contraction ℛ_0. 

Alternatively, if we take a first order derivative we get a vanishing contribution since 

    ∑_n^γ≤ x ≤ n B_0^x Ḃ_α≤ C∑_n^γ≤ x ≤ nλ^x
 
which decays as the term in the previous equation and

    ∑_n^γ≤ k < x ≤ n B_0^x-kḂ_0  B_0^k-1 B_α
       ≤ C∑_n^γ≤ k < x ≤ nλ^x≍ Cn^γλ^n^γ→_n → +∞ 0.


On the other hand, by differentiating a block of the form B^x_u,0 B_u,α twice, one stops the exponential decay and obtains a linear growth in n; however, we need to remember that every time we use a derivative, everything gets multiplied by u/√(n), which is of the order n^-1/2+ϵ^'. Therefore, if we consider the terms in the Taylor expansion up to order 2k, the only one that does not decay to 0 is the one with 2k derivatives where we use two of them on the block ρ̃^ ss( B^x_1_u,0 B_u,α^(1)(·)) at the beginning and other two on each following block of the form B_u,0^x_i B_u,α^(i)(·). Any other term where we have less derivatives involved or where we spend them in a different way is either 0 or decays at least as (for k ≥ 2)

    (n^1/2+γλ^n^γ)n^2kϵ^'

uniformly in 
u for |u|≤ n^ϵ^'; indeed, for any derivative used in a different way (they cannot be more than 2(k-1) because two of them need to be used for the first block), one gains a growth of n^1/2, but suffers a decay of at least n^γλ^n^γ.







We now focus on the leading (order 2k) term of the Taylor expansion. The second derivative of a block of the type 
ℬ^x_u,0ℬ_u,α looks like

    ℬ^x_0ℬ̈_α + 
    ∑_k=1^x 
    ℬ^k-1_0ℬ̈_0ℬ^x-kℬ_α
          +
    2∑_k=1^xℬ^k-1_0ℬ̇_0ℬ^x-k_0ℬ̇_α
          +
    2∑_1≤ k< s ≤ xℬ^k-1_0ℬ̇_0ℬ^s-k-1_0ℬ̇_0ℬ^x-s_0 ℬ_α

For large n this becomes



    ρ̃^ ss[
    (
    ℬ̈_α   + 
    2 ℬ̇_0ℛ_0ℬ̇_α )(·)]
     |χ^ ss⟩⟨χ^ ss|
     
          
    +ρ̃^ ss[(ℬ̈_0 + 2 ℬ̇_0ℛ_0ℬ̇_0 )
     ℛ_0 ℬ_α )(·)]
     |χ^ ss⟩⟨χ^ ss|

where 
ℛ_0 
is the Moore-Penrose inverse of Id - ℬ_0, and we have used the spectral decomposition of ℬ_0. Moreover, the rightmost term 
B_0^x_k+1(1) (which is not differentiated) converges to 
|χ^ ss⟩⟨χ^ ss|. This means that the full 2k derivative of (<ref>) becomes

    u^2k/k!∏_i=1^k1/2ρ̃^ ss[ B^(2)_α^(i) (|χ^ ss⟩⟨χ^ ss|) ]

where 

    B^(2)_α :=
     B_α+2 Ḃ_0  R_0Ḃ_α 
    +( B_0+
        2Ḃ_0  R_0 Ḃ_0) R_0  B_α^.



 
 





The number of terms with two derivatives in each block is equal to (2k)!/2^k (it is the same as the number of terms of the form 2^k= (d^2/dx^2(x^2)  )^k in the 2k-th derivative of (x^2)^k): (2k)! simplifies with the one coming from the Taylor expansion, while the factor 2^-k can be distributed to each factor. The expression of the factors and the upper bound on the error can be obtained differentiating and using the spectral decomposition of
 B_0; the 1/k! in front comes from the sum: indeed one can see
 that
 
    ∑_x_1 +… +x_k+1 = n-K
    x_2,…,x_k≥ n^γ≍∑_x_1 +… +x_k+1 = n≍n^k/k!,

 which is the number of ways one can choose k numbers out of n. 





The first term of the form

    1/2ρ̃^ ss( B_α(|χ^ ss⟩⟨χ^ ss|))= |⟨χ^ ss|K̃_α_|α|⋯K̃_α_2K̇̃̇_1 χ^ ss⟩|^2.

This follows from the fact that K_1|χ^ ss⟩ =0, so the derivatives need to be applied the the first K_1 terms of ℬ_α.
For the second term, one gets

    ρ̃^ ss(Ḃ_0  R_0Ḃ_α(|χ^ ss⟩⟨χ^ ss|))=
       2 (⟨K̃_α_|α|⋯K̃_α_2K̇̃̇_1 χ^ ss| χ^ ss⟩×.
       .
     ⟨χ^ ss|K̃_α_|α|⋯K̃_α_1(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩).

Finally, for the last term, below we  will show that

    1/2ρ̃^ ss(( B_0+2Ḃ_0  R_0 Ḃ_0) R_0  B_α(|χ^ ss⟩⟨χ^ ss|))=
       |⟨χ^ ss| K̃_α_|α|⋯K̃_α_1(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩|^2.

Note that the sum of the three terms is equal to |μ_α|^2 
where μ_α is given in equation (<ref>) in Lemma <ref>.

In conclusion, we obtained that for large n, the probability of any sequence of n outcomes showing the ordered sequence of excitation patterns given by α^(1), …, α^(k) is asymptotically equivalent to

    1/k!∏_i=1^k (λ_α^(i)u^2)

plus a reminder coming from neglecting the terms of order bigger than 2k in the Taylor expansion. If we are able to show that the reminder is negligible compared to the term in Eq. (<ref>), then we can prove the statement in Eq. (<ref>). Indeed, suppose that the sequence we are analysing belongs to B_𝐦; then we can partition B_𝐦 into

    k!/∏_i=1^k m_α^(i)!

disjoint subsets containing the excitation patters in (α^(1),m_α^(1)), …, (α^(k),m_α^(k)) in a fixed order and whose probability asymptotically behaves as (as we just showed)

    1/k!∏_i=1^k (u^2λ_α^(i))^m_α^(i).




We now prove (<ref>).
Notice that, since

    |⟨χ^ ss| K̃_α_|α|⋯K̃_α_1(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩|^2=
       ⟨ (1-K̃_0)^-1K̇̃̇_0 χ^ ss | B_α(|χ^ ss⟩⟨χ^ ss| )|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩,

we need to prove that

    R_0*( B_0*+2Ḃ_0* R_0*Ḃ_0*)(ρ̃^ ss))=
       2|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|+ rem,

where by R_0* we mean the Moore-Penrose inverse and "rem" is a term which is gives 0 when evaluated against B_α(|χ^ ss⟩⟨χ^ ss|). 
Equivalently,

    ( B_0*+2Ḃ_0* R_0*Ḃ_0*)(ρ̃^ ss)=
       2( Id- B_0*)(|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|)+ rem^',

where R_0*( rem^')= rem. 

By explicit computations, one can see that

    ( Id- B_0*)(|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|)=
       |(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|-
       |K̃_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨K̃_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss|.

Using that K̃_0(1-K̃_0)^-1=(1-K̃_0)^-1-1, one gets

    ( Id- B_0*)(|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|)=
       |(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨K̇̃̇_0 χ^ ss|+
    |K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|-
       |K̇̃̇_0 χ^ ss⟩⟨K̇̃̇_0 χ^ ss|.

On the other hand,

    ( B_0*+2Ḃ_0* R_0*Ḃ_0*)(ρ)
          =
       |K̃_0 χ^ ss⟩⟨χ^ ss|+2|K̇̃̇_0χ^ ss⟩⟨K̇̃̇_0χ^ ss|+|χ^ ss⟩⟨K̃_0 χ^ ss|
          +2|K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨χ^ ss|
          +2|χ^ ss⟩⟨K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss|
          +2|K̃_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨K̇̃̇_0χ^ ss|
          +2|K̇̃̇_0χ^ ss⟩⟨K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0 χ^ ss|
          =|(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss⟩⟨χ^ ss|
          +|χ^ ss⟩⟨(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss|
          +2|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨K̇̃̇_0. χ^ ss|
          +
    2|K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|-2|K̇̃̇_0 χ^ ss⟩⟨K̇̃̇_0 χ^ ss|.

Note that the last two lines are exactly equal to 2( Id- B_0*)(|(1-K̃_0)^-1K̇̃̇_0 χ^ ss⟩⟨(1-K̃_0)^-1K̇̃̇_0 χ^ ss|). Let us look at the remaining part:

    rem^'=   |(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss⟩⟨χ^ ss|+
       |χ^ ss⟩⟨(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss|.

one can easily see that

    R_0*( rem^')=
       |(1-K̃_0)^-1(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss⟩⟨χ^ ss|
       +|χ^ ss⟩⟨(1-K̃_0)^-1(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss|.

In the previous Eq. we used (1-K_0)^-1 for the Moore-Penrose inverse: in general, |(K̃_0 +2K̇̃̇_0(1-K̃_0)^-1K̇̃̇_0)χ^ ss⟩ is not orthogonal to |χ⟩^ ss.
It is now clear that 
( R_0*( rem^') B_α(|χ^ ss⟩⟨χ^ ss|)=0
which proves (<ref>).


Remainder. Now, we need to take care of the reminder: it is enough to show that the following expression is o(n^2kϵ):

    u^2k+1/n^k+1/2∑_x_1 +… +x_k+1 = n-K
    x_2,…,x_k≥ n^γ   d^2k+1/u^2k+1ρ̃^ ss (u/√(n) ) (B̃^x_1_0,uB̃_α_1,uB̃^x_2_0,u⋯.
        . .⋯B̃^x_k_0,uB̃_α_k,uB̃^x_k+1_0,u(1)) )  |_u=η


for any |η|≤ u/√(n), where

    B̃_α,u=e^λ_tot |α|u^2/n B_α,u

for any string α.










Let us first point out some properties of the maps 
B̃_α,u:


  * for u small enough,

    B̃_0,u(·)=a(u) l(u)(·)r(u) + Ẽ_0,u(·)

where

    
  * a(u)=1+O(u^3/n^3/2), 
    
  * l(u)(r(u))≡ 1, 
    
  * l(u)(Ẽ_0,u(·))=0,
    
  * Ẽ_0,u(r(u))=0 and
    
  * Ẽ^k_0,u≤ Cλ^k for some C ≥ 0 and 0<λ <1.

The order of the reminder in the expression a(u) is due to the fact that λ̇ is equal to the first derivative at 0 of the spectral radius of B_0,u which is equal to 0 because it attains a maximum there; the fact that λ=0 as well is due to the multiplicative factor in front of B_0,u in the definition of B̃_0,u.


  * ρ̃^ ss(u/√(n))(Ẽ_0,u(·))=O(u/√(n)).

  * For any excitation pattern α one has

    l(u)(B̃_α,u(·))=O(u^2/n),
       l(u)(Ḃ̃̇_α,u(·))=O(u/√(n)).

Indeed, the first derivative of l(u)(B̃_α,u(·)) at 0 is given by

    ρ̃^ ss(Ḃ_α(·))+ l̇( B_α(·)).

That the first addend is 0 has been shown earlier, while for the second one, it is clear using l̇=ρ̃^ ss(Ḃ_α ( Id- B_0)^-1(·)).

  * Moreover,

    l(u)(Ḃ̃̇_0,u(r(u)))=O(u^2/n)  and
       l(u)((B̃_0,u+2Ḃ̃̇_0,uẼ_0,uḂ̃̇_0,u(r(u)))=O(u/√(n)).

This can be seen differentiating

    l(u)B̃_0,u(x(u))=a(u)

and evaluating at 0.

  * Finally,

    l(u)(Ḃ̃̇_0,uẼ_0,uB̃_α,u(·))=O(u/√(n)),

since 
ρ̃^ ss(Ḃ_0)(·)=0.






Let us now study the growth of the derivatives of B̃_0,u^xB̃_α,u(·) for 0 ≤ x  ≤ n→ +∞:


  * 0^ th order:

    B̃_0,u^xB̃_α,u(·)   =a(u)l(u)(B̃_α,u(·))x(u)+O(λ^x) 
       =O  (u^2/n+λ^x  );


  * 1^ st order:

    ∑_1 ≤ l ≤ xB̃_0,u^x-lḂ̃̇_0,uB̃_0,u^l-1B̃_α,u(·)+B̃_0,u^xḂ̃̇_α,u(·)=
       xa(u)^2l(u)(Ḃ̃̇_0,u(x(u)))l(u)(B̃_α,u(·))+
       a(u)R̃_0,uḂ̃̇_0,u(x(u)))l(u)(B̃_α,u(·))+
       a(u)l(u)(Ḃ̃̇_0,uR̃_0,uB̃_α,u(·))x(u) + O (u/√(n) +xλ^x )
        
       =O ( u/√(n)+xλ^x ),

    where we used that ϵ <1/6;
    
    We remark that in the case where the block B̃_0,u^xB̃_α,u(·) is the first one, due to the action of ρ̃^ ss(u/√(n)), the 0^ th-order term becomes O(u^2/n+uλ^x/√(n)) and the 1^ st-order one becomes O(u/√(n)), while if the block is not the first one, λ^x decays exponentially fast in n since x ≥ n^γ. 
    
    
    
  * 2^ nd order:
    
    ∑_1 ≤ l ≤ xB̃_0,u^x-lB̃_0,uB̃_0,u^l-1B̃_α,u(·)+
       2∑_1 ≤ l<k  ≤ xB̃_0,u^x-kḂ̃̇_0,uB̃_0,u^k-l-1Ḃ̃̇_0,uB̃_0,u^l-1B̃_α,u(·)+
       2∑_1 ≤ l ≤ xB̃_0,u^x-lḂ̃̇_0,uB̃_0,u^l-1Ḃ̃̇_α,u(·)+
       B̃_0,u^xB̃_α,u(·)=
       2x^2a(u)^3(l(u)(Ḃ̃̇_0,u(x(u)))^2l(u)(B̃_α,u(·))+
       xa(u)^2l(u)((B̃_0,u+2Ḃ̃̇_0,uR̃_0,uḂ̃̇_0,u)(x(u)))l(u)(B̃_α,u(·))+
       2xa(u)^2l(u)(Ḃ̃̇_0,u(x(u))l(u)(Ḃ̃̇_α,u(·))+O(1)=O(1);

    
  * 3^ rd order:

    ∑_1 ≤ l ≤ xB̃_0,u^x-l⃛B̃_0,uB̃_0,u^l-1B̃_α,u(·)+
       3∑_1 ≤ l<k  ≤ xB̃_0,u^x-kB̃_0,uB̃_0,u^k-l-1Ḃ̃̇_0,uB̃_0,u^l-1B̃_α,u(·)+
       3∑_1 ≤ l<k  ≤ xB̃_0,u^x-kḂ̃̇_0,uB̃_0,u^k-l-1B̃_0,uB̃_0,u^l-1B̃_α,u(·)+
       6∑_1 ≤ l<k <m ≤ xB̃_0,u^x-mḂ̃̇_0,uB̃_0,u^m-k-1Ḃ̃̇_0,uB̃_0,u^k-l-1Ḃ̃̇_0,uB̃_0,u^l-1B̃_α,u(·)+
       3∑_1 ≤ l ≤ xB̃_0,u^x-lB̃_0,uB̃_0,u^l-1Ḃ̃̇_α,u(·)+
       6∑_1 ≤ l<k  ≤ xB̃_0,u^x-kḂ̃̇_0,uB̃_0,u^k-l-1Ḃ̃̇_0,uB̃_0,u^l-1Ḃ̃̇_α,u(·)+
       3∑_1 ≤ l ≤ xB̃_0,u^x-lḂ̃̇_0,uB̃_0,u^l-1B̃_α,u(·)+
       B̃_0,u^x⃛B̃_α,u(·)=O(n^2ϵ);

       
  * m^ th order for m ≥ 4: first, notice that
       
    d^m/du^m (B̃_0,u^xB̃_α,u(·) )=
       ∑_l=0^mmld^l/du^l(B̃_0,u^x) d^(m-l)/du^(m-l)(B̃_α,u(·)).

The term d^l(B̃_0,u^x)/du^l is a sum over all possible ways of distributing the derivatives among the factors B̃_0,u.

Then one can use the spectral decomposition of the B̃_0,u which have not been differentiated and glue together differentiated terms using R_0,u or turning them into products using the projection l(u)(·)x(u) as we did in the previous items. From Eq. (<ref>), one can see that the blocks with a single derivative of a single term B̃_0,u bring a growth of the order n^2ϵ^', the terms with two derivatives (B̃_0,u+2Ḃ̃̇_0,uR̃_0,uḂ̃̇_0,u) bring a growth of the order n^1/2+ϵ^' while all other blocks (with at least 3 elements), cause a growth of the order at most n. Notice that the highest growth is attained by making as many groups of three derivatives as possible and this will be used in obtaining the following estimate.

The term d^(m-l)(B̃_α,u)/du^(m-l) does not cause any reduction in the growth if m-l ≥ 2, while causes a decay equal to n^-1/2+ϵ^' if m=l+1 and n^-1+2ϵ^' if m=l.

Therefore,

    d^m/du^m (B̃_0,u^xB̃_α,u(·) )≲
       n^max{f(m-2), f(m-1)-1/2+ϵ^', f(m)-1+2ϵ^'},


where

    f(m)=a(m) + (1/2+ϵ^' )b(m)+2ϵ^' (m-(3a(m)+2b(m)))


    a(m):=⌊m/3⌋,    b(m):=⌊1/2 (m-3⌊m/3⌋ )⌋.

a(m) is the maximum number of groups with three derivatives and b(m) is the maximum number of groups with two derivatives that we can make with the derivatives left. Notice that if ϵ^' <1/6, then f(m) ≤ m/3: indeed,

    a(m) + (1/2+ϵ^' )b(m)+2ϵ^' (m-(3a(m)+2b(m)))=
       2ϵ^' m + (1-6ϵ^')a(m)+  ( 1/2-3ϵ^' ) b(m) ≤
       2ϵ^' m + (1-6ϵ^')a(m)+ 1/2 ( 1/2-3ϵ^' )(m-3a(m)) ≤
       1/2 ( ( 1/2+ϵ^' )m + ( 1/2-3ϵ^' )a(m)  )≤
       1/2 ( ( 1/2+ϵ^' )m + ( 1/6-ϵ^' )m  )≤m/3.

Therefore, the growth of the term in Eq. (<ref>) is upper bounded by

    n^max{m-2/3, m-1/3-1/2+ϵ^', m/3-1+2ϵ^'}.

This implies that the first two derivatives spent on a block of the form B̃_0,u^xB̃_α,u cause a growth equal to n^1/2-ϵ^', while the other m-2 only brings a growth at most equal to the term in Eq. (<ref>). Notice that it is more convenient to spend them in a way that every block of the form B̃_0,u^xB̃_α,u has two derivatives, since one has that

    (1/2-ϵ^' )(m-2) ≥m-2/3⇔ m ≥ 2,

moreover

    (1/2-ϵ^' )(m-2) ≥m-1/3-1/2+ϵ^'⇔ m ≥ 1

and

    (1/2-ϵ^' )(m-2) ≥m/3-1+2ϵ^'⇔ m ≥ 0.

Therefore the growth of

    ∑_x_1 +… +x_k+1 = n-K
    x_2,…,x_k≥ n^γ   d^2k+1/u^2k+1ρ̃^ ss (u/√(n) ) (B̃^x_1_0,uB̃_α_1,uB̃^x_2_0,u⋯.
        . .⋯B̃^x_k_0,uB̃_α_k,uB̃^x_k+1_0,u(1)) )  |_u=η,

is of the order of O(n^2ϵ^') uniformly in u and it is attained by the term where we spend at least two derivatives in each block of the form B̃_0,u^xB̃_α,u and the last derivative in any of such blocks. One can see that it is not convenient to use any derivative in the final term B̃_0,u^x(1), since it is better to contrast the decay induced by the other blocks. To conclude, the term in Eq. (<ref>) grows at most as n^(2k+3)ϵ^'-1/2=o(n^2kϵ^'). We proved Proposition <ref> and notice that Theorem <ref> follows considering ϵ^'=0.




















































§ PROOF OF COROLLARY <REF>
 

    Let us define the measurable space given by the set Ω=ℕ^ℕ together with the σ-field F generated by cylindrical sets; we can consider on (Ω,  F) the law ν_u of {N_α:α∈ P} and the law of {N_α(n): α∈ P}, which, with a slight abuse of notation, we still denote by ν_u,n as well. We know that for every finite set A of patterns, one has
    
    lim_n → +∞sup_|u|<C |ν_u,n(A)-ν_u(A)|=0.

    Notice that, for every ϵ >0, there exists a set A_ϵ of finitely many patter such that
    
    inf_|u|<Cν_u(A_ϵ) > 1-ϵ.

    Therefore, using Eq. (<ref>), one has that there exists N_ϵ such that ∀ n ≥ N_ϵ,
    
    inf_|u|<Cν_u,n(A_ϵ) > 1-ϵ,  and sup_|u|<C |ν_u,n(A_ϵ)-ν_u(A_ϵ)|<ϵ.
 
    Therefore, given a bounded function f:ℕ^ P→ℝ, for every ϵ>0, ∀ n ≥ N_ϵ, one has
    
    sup_|u| < C|𝔼_ν_u,n[f]-𝔼_ν_u[f]| ≤
       f_∞ (sup_|u|<C |ν_u,n(A_ϵ)-ν_u(A_ϵ)| + .
        .sup_|u|<Cν_u,n(A_ϵ^C)+ν_u(A_ϵ^C)  ) ≤
       3f_∞ϵ.

    For the arbitrariness of ϵ, we proved the first statement about the weak convergence.

    Consider the random process N_ tot(n) that counts all the occurrences of 1's in the output up to time n and notice that for every α∈ P, N_α(n) ≤ N_ tot(n). Therefore for every p ≥ 1 and m ∈ℕ, one has
    
    𝔼_ν_u,n[N_α(n)^p 1_{N_α(n) >m}] ≤𝔼_ν_u,n[N_ tot(n)^p 1_{N_ tot(n) >m}].

    If we show that the moments of every order of N_ tot(n) converge to some finite limit and that it converges in law to some limit random variable X_u, we obtain the second statement as well. Indeed, let us call C(p) the limit of the p-moment of N_ tot(n); notice that for every p ≥ 1, m ∈ℕ
    
    lim sup_n → +∞𝔼_ν_u,n[N_α(n)^p 1_{N_α(n) >m}] ≤
       lim sup_n → +∞𝔼_ν_u,n[N_ tot(n)^p 1_{N_ tot(n) >m}] ≤
       lim_n → +∞𝔼_ν_u,n[N_ tot(n)^pq]^1/qν_u,n(N_ tot(n) >m)^1/q^' =
       C(pq)^1/qℙ(X_u >m)^1/q^'.

    Notice that in the last inequality we made use of Hölder inequality for some pair of conjugate indices (q, q^'). Therefore, if we fix p≥ 1, for every ϵ>0, one can choose m_ϵ such that
    
    lim sup_n → +∞𝔼_ν_u,n[N_α(n)^p 1_{N_α(n) >m_ϵ}] ≤ϵ/2,
       𝔼_ν_u[N_α^p 1_{N_α >m_ϵ}] ≤ϵ/2

    and one gets
    
    lim sup_n → +∞|𝔼_ν_u,n[N_α(n)^p]-𝔼_ν_u[N_α^p]| ≤
       lim_n → +∞|𝔼_ν_u,n[N_α(n)^p1_{N_α(n) ≤ m_ϵ}]-𝔼_ν_u[N_α^p 1_{N_α≤ m_ϵ}]|+
       lim sup_n → +∞𝔼_ν_u,n[N_α(n)^p 1_{N_α(n) >m_ϵ}] +
       𝔼_ν_u[N_α^p 1_{N_α >m_ϵ}] ≤ϵ.

Since this holds for every ϵ>0, we proved the statement.

We need to show that, under ν_u,n, N_ tot(n) converges in law to a random variable X_u with finite moments of every order and that we have convergence of the moments as well. One can see that the Laplace transform of N_ tot(n) can be expressed as

    𝔼_ν_u,n[e^zN_ tot(n)]=ρ̃^ ss(u/√(n))(𝒯̃_u,z,n^n(1))    z ∈,

where

    𝒯̃_u,z,n(·)=   K̃^*_0(u/√(n))·K̃_0(u/√(n))+
       e^zK̃^*_1(u/√(n)) ·K̃^*_1(u/√(n)).

Notice that 𝒯̃_0,z:=𝒯̃_0,z,n is independent from n and is an analytic perturbation of 𝒯̃. If we pick z small enough in modulus, perturbation theory ensures that 𝒯̃_0,z has 1 as eigenvalue with maximum modulus with |χ^ ss⟩⟨χ^ ss| as left eigenvector. Let x_z be the corresponding right eigenvector such that 
Tr(ρ^ ss x_z) = ⟨χ^ ss|x_z |χ^ ss⟩=1, and let

    x_z=[ 1 a; b c ]

be its block matrix form with respect to 
the decomposition of ℂ^D into 
ℂ|χ^ ss⟩ and its orthogonal complement. Then one can prove that 

a=b=0 by using the fact that the Kraus operators are of the form

    K_0=[ 1 0; 0 β ],    K_1=[ 0 γ; 0 δ ]
 
for some blocks β, γ, δ 
such that |β|^2+|γ|^2+|δ|^2=1. Let us first fix z small enough, then for n big enough, 𝒯̃_u,z,n has a unique eigenvalue λ_z(u,n) of maximum modulus with corresponding left and right eigenvectors l_z(u,n), x_z(u,n) and one has that

    𝔼_ν_u,n[e^zN_ tot(n)] =
       λ_z(u,n)^n l_z(u,n)(1) ρ̃^ ss(u/√(n))(x_z(u))+o(n)

where o(n) is uniform in u (and z in compact sets). Notice that both l_z(u,n)(1) and ρ̃^ ss(u/√(n)) (x_z(u)) converge to 1 for n → +∞; regarding the behaviour of λ_z(u,n) consider the Taylor expansion up to second order in u around 0:

    λ_z(u,n)=1+u/√(n)λ^(1)_z+u^2/nλ^(2)_z + o(n^-1).

We can choose l_z(u,n) and x_z(u,n) such that (l_z(u,n)x_z(u,n))≡ 1, therefore differentiating (l_z(u,n)𝒯̃_z,u,n(x_z(u,n)))=λ_z(u,n) at 0 one gets

    λ^(1)_z=(l_z ∂_u𝒯̃_z|u=0 x_z)

which can be easily seen to be 0. Summing up, we proved that

    lim_n→ +∞𝔼_ν_u,n[e^zN_ tot(n)]=e^λ_z^(2)

uniformly in u and in z in compact small neighborhoods of 0. Since for every u, f_n(z):=𝔼_ν_u,n[e^zN_ tot(n)] are analytic functions around 0, then we can deduce that e^λ_z^(2) is analytic as well and that we have uniform convergence (on compact small neighborhoods of 0) of all the derivatives, which consists exactly in the convergence of moments of all orders and we are done.




§ ACHIEVABILITY OF THE QCRB UNDER ADDITIONAL ASSUMPTIONS 


 

In this section we present a proof of Theorem <ref> and we comment on the gap between the hypothesis that we need to assume and what we proved in Proposition <ref>.



















By hypothesis, we know that θ belongs to the (random) confidence interval 
    I_n = (θ̃_n-n^-1/2 +ϵ,  θ̃_n+n^-1/2 +ϵ)

with high probability. In order to prove the statement, it suffices to show that 

    |(𝔼_θ[e^ia √(n) (θ̂_n-θ)|θ̃_n]-e^-a^2/2f_θ)|χ_{θ∈ I_n}(θ̃_n)
 
can be upper bounded uniformly in θ̃_n by a sequence converging to 0. Indeed, notice that for every a ∈ℝ

    𝔼_θ[e^ia √(n) (θ̂_n-θ)]   =e^-a^2/2f_θℙ_θ(θ∈ I_n) 
       +∫_θ∈ I_np_θ(dθ̃_n)(𝔼_θ[e^ia √(n) (θ̂_n-θ)|θ̃_n]-e^-a^2/2f_θ)
       +∫_θ∉ I_np_θ(dθ̃_n)𝔼_θ[e^ia √(n) (θ̂_n-θ)|θ̃_n].

Since ℙ_θ(θ∉ I_n) goes to zero, the first term goes to e^-a^2/2f_θ and the third one vanishes. If we show that the second term vanishes then we obtain the convergence in distribution of √(n) (θ̂_n-θ) as in the statement.

First of all notice that for every a ∈ℝ, for every θ̃∈Θ

    𝔼[e^ia(Y_n,θ, θ̃-√(n)(θ-θ̃))]=
       e^λ_ tot(θ̃)(√(n)(θ-θ̃)+τ_n)^2 (e^ia/(2λ_ tot(θ̃) τ_n) -1)-ia  (τ_n/2+√(n)(θ̃-θ)  ).

Therefore for every a ∈ one has

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵ |𝔼[e^ia(Y_n,θ, θ̃-√(n)(θ-θ̃))]-e^-a^2/f_θ |=0




We will denote by u=√(n)(θ-θ̃_n). The definition of θ̂_n implies that

    √(n)(θ̂_n-θ)=Y_n-u.

Therefore we obtain that

    sup_θ̃:|θ̃-θ|<n^-1/2+ϵ |𝔼_θ[e^ia(Y_n-u)|θ̃_n=θ̃]-e^-a^2/2f_θ | ≤
       sup_θ̃: |θ̃-θ|<n^-1/2+ϵ|𝔼_θ[e^iaY_n|θ̃_n=θ̃]-𝔼[e^iaY_n,θ, θ̃]|+
       sup_θ̃:|θ̃-θ|<n^-1/2+ϵ |𝔼[e^ia(Y_n,θ, θ̃-√(n)(θ-θ̃))]-e^-a^2/f_θ |.

Hence,

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵ |𝔼_θ[e^ia(Y_n-u)|θ̃_n=θ̃]-e^-a^2/2f_θ |=0

and we are done.

Let us briefly comment on the relationship between the additional hypothesis we introduced (Eq. (<ref>)) and the result in Proposition <ref>; let us consider the following family of stochastic processes: for every n ∈ℕ, θ̃, θ∈Θ, consider the collection of independent random variables

    N_n,θ, θ̃,α∼Poisson(λ_α(θ̃ (√(n)(θ-θ̃)+τ_n)^2),   α∈𝒫

and their law ν_n, θ, θ̃ on ℕ^ P. (together with the σ-field of cylindrical sets). Notice that
∑_α∈ PN_n, θ,θ̃, α converges in mean square and has the same law as N_n, θ,θ̃.

Inspecting the proof, one can notice that the convergence in the statement of Proposition <ref> holds uniformly in a small neighborhood of the reference parameter θ_0, therefore we can restate it in the following way: for ϵ small enough and for every finite collections of excitation patterns counts 𝐦 one has

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵ | ν_√(n)(θ-θ̃),n(B_𝐦(n))/ν_θ, θ̃,n(𝐦)-1 |=0.

If we were able to show that the previous result still holds integrating with respect to ν_θ, θ̃,n, i.e.

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵ∑_𝐦ν_θ, θ̃,n(𝐦) | ν_√(n)(θ-θ̃),n(B_𝐦(n))/ν_θ, θ̃,n(𝐦)-1 |=
       lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵ∑_𝐦 | ν_√(n)(θ-θ̃),n(B_𝐦(n))-ν_θ, θ̃,n(𝐦) |=0,

this would imply the condition in Eq. (<ref>) (it can be seen using the fact that ∑_α∈ PN_n, θ,θ̃, α and N_n, θ,θ̃ have the same law). Unfortunately, we are not able to prove this.

The last remark we make is that Eq. (<ref>) cannot be true unless

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵν_θ, θ̃,n(G(n)^C)=0,

where G(n) is the set of all 𝐦's such that ν_√(n)(θ-θ̃),n(B_𝐦(n))>0. We can prove that this is indeed the case.

If ϵ is small enough, then

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵν_θ, θ̃,n(G(n)^C)=0.


Notice that G(n) is the set of all those patterns counts m=(m_α^(1), …, m_α^(k)) such that the following conditions are satisfied


  * α^(i) does not contain more than n^γ consecutive 0s for every i=1,…, k;

  * ∑_i=1^km_α^(i)|α^(i)| + (k-1)n^γ≤ n.

We recall that |α| is the length of the pattern α.

Let us consider a positive number η<1-γ and notice that one has G̃(n) ⊆ G(n), where G̃(n) is the set of all those patterns counts m=(m_α^(1), …, m_α^(k)) such that


  * |α^(i)| ≤ηlog_2(n) for every i=1,…, k and

  * ∑_i=1^km_α^(i)|α^(i)|+(k-1)n^γ≤ n.

We denote by A(n) the set of patterns satisfying 1. and B(n) the set of patterns satisfying 2., hence G̃(n)=A(n) ∩ B(n). In order to prove the statement, it suffices to show that

    lim_n → +∞sup_θ̃:|θ̃-θ|<n^-1/2+ϵν_θ, θ̃,n(G̃(n)^C)=0.

Notice that G̃(n)^C=A(n)^C ⊔ (B(n)^C∩ A(n)); let us first show that

    lim_n → +∞inf_θ̃:|θ̃-θ|<n^-1/2+ϵν_θ, θ̃,n(A(n)) → 1.

Notice that, 

    lim_n → +∞inf_θ̃:|θ̃-θ|<n^-1/2+ϵν_θ, θ̃,n(A(n))=
       lim_n → +∞inf_θ̃:|θ̃-θ|<n^-1/2+ϵe^- (∑_|α| > ηlog_2(n)λ_α ) (√(n)(θ-θ̃)-τ_n)^2=1,

since, for every θ̃ such that |θ-θ̃|<n^-1/2+ϵ, one has

    (∑_|α| > ηlog_2(n)λ_α ) (√(n)(θ-θ̃)-τ_n)^2 ≲ n^ηlog_2(λ) + 3ϵ→ 0

if ϵ <-ηlog_2(λ)/3.
Let us now study the probability of B(n)^C ∩ A(n): first notice that

    B(n)^C ∩ A(n) ⊆
          { m :|α^(i)| ≤ηlog_2(n),  ∑_i=1^km_α^(i)|α^(i)|+2^ηlog_2(n)n^γ > n},

because 2^ηlog_2(n) upper bounds the cardinality of all the patterns of length smaller or equal than ηlog_2(n). Therefore

    ν_θ, θ̃,n(B(n)^C ∩ A(n)) ≤ν_θ, θ̃,n(C(n))
    
    
       ≤ν_θ, θ̃,n ( { m:∑_|α| ≤ηlog_2(n)m_α > n-n^η+ γ/ηlog(n)} ).

where C(n) is the set of pattern counts such that |α^(i)| ≤ηlog_2(n) and ∑_i=1^km_α^(i)ηlog_2(n)+n^η +γn > n.
The last term amounts to the probability that a Poisson random variable of parameter ∑_|α| ≤ηlog_2(n)λ_α (√(n)(θ-θ̃)-τ_n)^2 ≲ n^3ϵ is bigger than something that grows as
n/(ηlog(n)).
Such a probability goes to 0 uniformly in θ̃ if 3ϵ<1 and we are done.





    




§ PRIMITIVITY OF SYSTEM AND ABSORBER DYNAMICS



In this section we investigate the validity of the hypothesis that system and matching absorber generate primitive dynamics. Currently, it is unclear whether this property follows from the primitivity of the system dynamics, and if not, whether it holds for “generic” dynamics or fails for a larger class of systems.



To explore this question we generated random sets of Kraus operators for the system dynamics and  corresponding sets for the system and absorber and computed the spectral gaps of the system and system plus absorber transition operators.


We could start looking at the case 𝔥_S=𝔥_A=𝔥_E=ℂ^2.



  * Sampling a primitive quantum channel, which is the generic type (Generating random quantum channels, JMP, 2021);

  * K_0, K_1, ρ^ss=∑_i=0^1λ_i |i⟩⟨i| with λ_0∈ (0,1/2).

  * V=∑_i=0^1|i⟩_A ⊗|0⟩_E ⟨v_i|+ |i⟩_A ⊗|1⟩_E ⟨w_i| where

    |v_i⟩=∑_j,k=0^1√(λ_j/λ_i)⟨i| K_k |j⟩|j⟩_A ⊗|k⟩_E.

|w_i⟩ are free to be chosen (we could sample random 2×2 unitaries).

  * the Kraus operators of the evolution of system and absorber are given by

    K̃_i=∑_k=0^1 (W_ik⊗1_S )(1_A ⊗ K_k),

where W_ik=⟨i|_E V |k⟩_E, hence

    W_0,k=∑_i,j=0^1√(λ_j/λ_i)⟨j|K_k |i⟩|i⟩_A⟨j|_A

and

    W_1,k=∑_i^1|i⟩_A⟨w_i||k⟩_E.

Check that ∑_i=0^1K̃^*_i ·K̃_i is ergodic and aperiodic.

Does the assumption that one has a strong interaction between the system and the environment (we have d^2 l.i. Kraus operators) improve things?








apsrev4-1









