Paper Title: Solving cluster moment relaxation with hierarchical matrix.


	[
    *
    August 5, 2024
==================

§ ABSTRACT 
		Convex relaxation methods are powerful tools for studying the lowest energy of many-body problems. By relaxing the representability conditions for marginals to a set of local constraints, along with a global semidefinite constraint, a polynomial-time solvable semidefinite program (SDP) that provides a lower bound for the energy can be derived. In this paper, we propose accelerating the solution of such an SDP relaxation by imposing a hierarchical structure on the positive semidefinite (PSD) primal and dual variables. Furthermore, these matrices can be updated efficiently using the algebra of the compressed representations within an augmented Lagrangian method. We achieve quadratic and even near-linear time per-iteration complexity. Through experimentation on the quantum transverse field Ising model, we showcase the capability of our approach to provide a sufficiently accurate lower bound for the exact ground-state energy.
	
	

§ INTRODUCTION

	
	Determining the lowest energy state of a many-body system is one of the most fundamental problems in science and engineering. This type of problem arises in the study of the Ising model <cit.>, graphical modeling <cit.>, sensor network localization <cit.>, and the structure from motion problem <cit.>, to name a few. In these problems, one is usually concerned with minimizing an energy function  ℰ. With the exception of some simple cases, the energy landscape of ℰ is plagued with spurious local minima. Without loss of generality, one can recast the problem of minimizing ℰ as another equivalent minimization problem <cit.>
	
    E_0 = inf_μ∈𝒫(𝒮)∫ℰ(x)μ(dx)

	on the space of measure over a set 𝒮, denoted as 𝒫(𝒮). By moving to the space of measure, one effectively obtains a linear optimization problem that circumvents the non-convexity issue in minimizing ℰ, at the expense of dealing with the high-dimensional measure μ. Generically, the solution of (<ref>) is an extreme point of 𝒫(𝒮) which is a Dirac measure, and the support of such measure gives a minimizer of ℰ. Such a view is adopted in <cit.> when devising a moment-based convex program for the case when ℰ is a low-degree polynomial. 
	
	
	There is an analogous problem in quantum many-body physics, where the ground-state energy minimization problem
	
    E_0 = min_ρ (Ĥρ),   s.t. ρ is positive semidefinite,  (ρ) = 1,

	is commonly solved (for example the quantum Ising and Hubbard model <cit.>). Here, ρ is a density operator subject to certain constraints, and Ĥ is a high-dimensional Hamiltonian operator capturing the interactions between d-sites <cit.>. The difficulty of solving (<ref>) is that the matrix ρ scales exponentially as the number of bodies grows.
	
	

 §.§ Prior works

	The issue with measure or density operator minimization is that minimizing these high-dimensional objects is prohibitively expensive due to the curse of dimensionality. Therefore, instead of working with the high-dimensional measure or density operator, approaches based on moments have been proposed to solve (<ref>) and (<ref>) without the curse of dimensionality. <cit.> proposes the use of moments to solve (<ref>):
	
    m_α := 𝔼_μ(x^α),   x = (x_1,…,x_d),  α = (α_1,…,α_d),

	where α∈{0∪ℕ}^d is a multi-index, and x^α := x_1^α_1… x_d^α_d. A convex relaxation is applied to  the space of {m_α}_α, where an outer approximation to the set of valid moments is given by a convex semidefinite program (SDP). Suppose we place the limit ∑_i α_i ≤ q on the degree of the moments, we have a convex problem in terms of d +q  q moments. Although by increasing q, the truncation threshold for the moments, one can improve the solution quality or even exactly recover the minimizer of the polynomial p <cit.>, for most practical situations, one can only use q=2 due to a large d. 
	
	
	An analogous fermionic quantum mechanical version of the moment-based relaxation is detailed in <cit.>. There, one deals with quantum moments of the form
	
    m_α := Tr(O^αρ),   O=(O_1,…,O_d)

	where O^α := O_1^α_1… O_d^α_d. Again, such a method can scale badly with d, constraining its application to small systems. 
	
	
	To improve the scaling of these methods, recently, cluster moments/marginals semidefinite programming relaxations have been proposed, both for minimizing classical <cit.> and quantum energies  <cit.>. The general idea is that one first clusters the variables/operators, and only forms higher-order moments for intra-cluster variables/operators. This significantly lowers the number of decision variables involved, resulting in O(d)× O(d) type scaling. 
	
	In this paper, we adopt a strategy similar to the variational embedding method <cit.>, where we try to determine local cluster moments and combine them through a global PSD constraint. The difference is that the local cluster density matrices are represented through their moments. In this case, the decision variable is a PSD moment matrix. The main point of this paper is to propose a method to accelerate the PSD optimization problem. Typically, the most computationally expensive step in such an optimization problem is the projection onto the PSD cone, which scales cubically. In <cit.>, translation invariance of the Hamiltonian is exploited to diagonalize the PSD matrix in the Fourier basis with linear time complexity. However, it is unclear how such computational scaling can be achieved for general systems. 
	
	
	

 §.§ Our contributions

	We propose a convex relaxation, the cluster moment relaxation, to solve the energy minimization problem in both classical and quantum settings. Furthermore, we introduce a specific form of hierarchical matrices, which differs from the conventional definition, to represent the primal and dual variables of the proposed semidefinite relaxation. 
	
	The key point is that the constraints of the proposed relaxation can be enforced efficiently using the algebra of hierarchical matrices. Within an augmented Lagrangian method, with a hierarchical dual PSD variable, the optimization can be carried out with quadratic per-iteration complexity. Additionally, if one assumes the primal moment matrix also takes the form of a hierarchical PSD matrix, near-linear per-iteration complexity can be achieved.
	
	
	

 §.§ Organizations

	In Section <ref>, we detail a  convex relaxation framework to solve energy minimization problems. In Section <ref>, we review the augmented Lagrangian method (ALM) for solving the proposed convex program. In Section <ref> and <ref>, we propose the use of hierarchical matrices to accelerate the ALM. In Section <ref>, we demonstrate the efficacy of the method for a quantum spin model.  
	
	

 §.§ Notations

	We use I_n to denote the identity matrix of size n × n. Additionally, we use 0_m× n to denote a zero matrix of size m × n, and when the context is clear, we will omit m and n. Furthermore, let ^n be the space of real symmetric matrices of size n × n, and let ^n_+ be the positive semidefinite matrices in ^n. Similarly, let ^̋n be the space of Hermitian matrices of size n × n, and let ^̋n_+ be the positive semidefinite matrices in ^̋n. For any matrix X in ^n or ^̋n, we may also use X ≽ 0 to denote that X is positive semidefinite.
	
	When discussing a matrix A, the notation A(p,q) refers to its (p,q)-th entry. In a block matrix A, A_ij represents its (i,j)-th block. We may occasionally use A_ij(p,q) to denote the (p,q)-th entry of the (i,j)-th block. For a complex-valued matrix A, (A) and (A) denote its real and imaginary parts, respectively. For a linear operator  on matrices or vectors, its adjoint is denoted by ^*. Lastly, for any positive integer N, we use [N] to denote the set {1,2,⋯,N-1,N}.
	
	
	
	

§ PROPOSED CONVEX RELAXATIONS

	
	While there are many versions of the cluster moments/marginals approach to obtain a convex relaxation of the energy minimization problems, in this paper, we examine a specific kind that only has equality constraints besides a global positive semidefinite constraint. As we shall see, this formulation can be optimized efficiently by our proposed method. The convex relaxation is constructed out of the following ingredients: 
	
		
  * Cluster basis: We first form monomials of variables/operators and cluster them into K different groups 𝒞_i⊂{0∪ℕ}^d, i∈ [K]. Each 𝒞_i is assumed to have C elements, i.e. | C_i | = C. These clusters of monomials are called the cluster basis:
		
			
  * Classical: 𝐯= [𝐯_i]_i∈[K], 𝐯_i = [x^α]_α∈𝒞_i.
			
  * Quantum: 𝐯 = [𝐯_i]_i∈[K], 𝐯_i = [O^α]_α∈𝒞_i.
		
		
  * Product cluster basis: We then take the cluster basis and form their products as follows:
		
			
  * Classical: 𝐯𝐯^*, and 𝐯_i𝐯_j^* = [x^αx^β^*]_α∈𝒞_i,β∈𝒞_j.
			
  * Quantum: 𝐯𝐯^*, and 𝐯_i𝐯_j^* = [O^αO^β^*]_α∈𝒞_i,β∈𝒞_j.
		
		
  * Intra-cluster relationship: The cluster basis and the product of the basis elements within the same cluster satisfy the following linear constraint for j ∈ [K]:
		
			
  * 𝒟_U(𝐯_j)+𝒟_L(𝐯_j^*)+𝒟(𝐯_j𝐯_j^*) = 𝐳. 𝐳 is a vector of scalars (operators) in the classical (quantum) case.
		
		
  * Inter-cluster relationship: The products of the basis elements, 𝐯_i 𝐯_j^* and 𝐯_j 𝐯_i^*, for i< j, i,j∈[K], satisfy the following relationship:
		
			
  * 𝒜_U(𝐯_i𝐯^*_j)+𝒜_L(𝐯_j𝐯^*_i) = 𝐰. 𝐰 is a vector of scalars (operators) in the classical (quantum) case.
		
	
	
	
	
	
	We now use these four ingredients to provide a convex relaxation for the         energy minimization problem in terms of the moment matrix M ∈ℂ^(CK+1) × (CK +1)
	
    Classical: M = 𝔼_μ([ 𝐯; 1 ][ 𝐯^*   1 ]),  Quantum: M = 𝔼_ρ([ 𝐯; I ][ 𝐯^*   I ]).

	Here, the expectations are taken entry-wise and defined as  𝔼_μ(·) := ∫ (·) μ(dx) and  𝔼_ρ(·) := ⟨·, ρ⟩. In what follows, for a matrix A∈ℂ^(CK+1) × (CK +1), we partition A into
	
    A := [   A^(2)   A^(1); A^(1)^*   A^(0) ],    A^(2)∈ℂ^CK × CK, A^(1)∈ℂ^CK × 1, M^(0)∈ℂ.

	We often also write 
	
    A^(2) = [A^(2)_ij]_i=1,j=1^K

	where each A^(2)_ij is a C× C block. 
	
	These ingredients give a set of necessary conditions on the moment matrix:
	
		
  * The inter-cluster relationship gives 𝒜_U(M^(2)_ij)+𝒜_L(M^(2)_ji) = w for i< j, i,j∈ [K], where w:=𝔼_μ,ρ(𝐰). 
		
  * The intra-cluster relationship gives 𝒟_U(M_j^(1))+𝒟_L(M_j^(1)^*)+ 𝒟(M_jj^(2)) = z for j∈[K], where z:=𝔼_μ,ρ(𝐳). Furthermore, M^(0) = 1, since 𝔼_μ,ρ(1) = 1. 
		
  * M≽ 0.
	
	For convenience, we assume w and z are real-valued vectors of sizes P_1 and P_2, respectively. With these necessary conditions, an energy minimization problem can be relaxed into a convex problem as follows:
	
    (P)       min_M Tr(JM)s.t.       Λ_ij∈ℝ^P_1 × 1:𝒜_U(M^(2)_ij)+𝒜_L(M^(2)_ji) = w,  i< j,  i,j∈ [K],       λ_j∈ℝ^P_2 × 1: 𝒟_U(M_j^(1))+𝒟_L(M_j^(1)^*)+ 𝒟(M_jj^(2)) = z,  j∈[K],       γ∈ℝ : M^(0) = 1,       S∈ℍ_+^CK+1: M∈ℍ_+^CK+1.

	
	
	
	

 §.§ Example of quantum energy minimization

	
	
	In physics, it is often the case that we have Hamiltonians with only pairwise interactions, i.e., the loss function in (<ref>) is equal to 
	
    ∑_1 ≤ i ≤ N(Ĥ_i ρ) + ∑_1 ≤ i<j ≤ N(Ĥ_ijρ).

	where Ĥ_i's and Ĥ_ij's are effectively some one-variable and two-variable operators.
	
	One example that we study in this paper is the quantum spin-1/2 system. The basic building blocks of these Hamiltonians are the Pauli matrices and the 2-dimensional identity matrix:
	
    σ^x = [ 0 1; 1 0 ],   σ^y = [  0 -i;  i  0 ],   σ^z = [  1  0;  0 -1 ],   
    	I_2 = [ 1 0; 0 1 ].

	These matrices form a basis for the real vector space of Hermitian operators on ℂ^2. Furthermore, let σ_i^x/y/z denote the operator obtained by tensoring σ^x/y/z on the i-th site with identities I_2 on all other sites, i.e.,
	
    σ_i^α := I_2^⊗ (i-1)⊗σ^α⊗ I_2^⊗ (N-i),  α∈{x,y,z},

	and let I:= I_2^⊗ N. In this case, all Ĥ_i and Ĥ_ij can be written as:
	
    Ĥ_i = ∑_α∈{x,y,z} a^α_i σ_i^α,     1≤ i ≤ N, 
       Ĥ_ij = ∑_α, β∈{x,y,z} a^αβ_ijσ_i^ασ_j^β,    1 ≤ i < j ≤ N,

	for some real constants {a_i^α}_i, α and {a_ij^αβ}_ij, αβ. 
	
	
	
	
	
	
	For example, the celebrated 1-D transverse field Ising (TFI) model <cit.>, a spin-1/2 quantum model, is defined by its Hamiltonian:
	
    Ĥ_TFI = -h ∑_i=1^N σ_i^x -∑_i=1^N σ_i^z σ_i+1^z,

	where the TFI model is assumed to have periodic boundary conditions, i.e., σ_N+1^x/y/z should be identified with σ_1^x/y/z, and h ∈ is a scalar parameter controlling the strength of the external magnetic field along the x axis. 
	
	
	

  §.§.§ Cluster moment relaxation for the TFI

	
	We go through the four ingredients needed to construct a cluster moment relaxation. For an N-spin system, we define 
	
    𝐯 := [𝐯_i]_i=1^N,   𝐯_i = [ σ_i^x^* σ_i^y^* σ_i^z^* ]^*,

	  and 𝐯 is a vector of operators of length 3N. In this case, K=N and C=3. Then we let M of size (3N+1) × (3N+1) be defined by
	
    M^(2)(i,j) =  (𝐯(i) 𝐯(j)^* ρ),    M^(1)(i) =  (𝐯(i)ρ).

	The properties of Pauli operators give rise to inter-cluster and intra-cluster constraints, which are summarized in the second column of Table <ref>.
	
	These relationships between operators then give constraints on the moment matrix
	
		
  * i< j,   i,j∈ [K],
            
    𝒜_U(M^(2)_ij)+𝒜_L(M^(2)_ji) = w ⇒(M^(2)_ij) = 0.
 
		
  * j∈[K],   𝒟_U(M_j^(1))+𝒟_L(M_j^(1)^*)+ 𝒟(M_jj^(2)) = z  ⇒  
		
    (M_j^(1))    =    0  
    		M^(2)_jj - diag({M^(2)_jj(k,k)}^3_k=1)    =   [            0  iM^(1)_j(3) -iM^(1)_j(2); -iM^(1)_j(3)            0  iM^(1)_j(1);  iM^(1)_j(2) -iM^(1)_j(1)            0 ] 
    		M^(2)_jj(k,k)    =    1, k=1,2,3.

	
	
	
	
	
	
	
	
	
	

§ STANDARD AUGMENTED LAGRANGIAN METHOD

	
	In this section, we first derive a standard ALM <cit.> to solve (<ref>), which serves as a foundation to discuss algorithms with reduced complexities in subsequent sections. The dual problem of (P) can be written as
	
    (D)      {Λ_ij}_i<j,{λ_j}_j,
    γ,S∈ℍ_+^CK+1max ∑_i<j≤ K⟨Λ_ij,  w ⟩+∑_j∈[K]⟨λ_j, z⟩ + γs.t.        J- ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ)  = S

	where 
	
    ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ)  := [        𝒟^*(λ_1)     𝒜^*_U(Λ_12)               ⋯     𝒜^*_U(Λ_1K)                      𝒟_U^*(λ_1);     𝒜^*_L(Λ_12)        𝒟^*(λ_2)               ⋱                                               ⋮;               ⋮               ⋱               ⋱ 𝒜^*_U(Λ_(K-1)K)                                ;     𝒜^*_L(Λ_1K)               ⋯ 𝒜^*_L(Λ_(K-1)K)        𝒟^*(λ_K)                      𝒟_U^*(λ_K);      𝒟_L^*(λ_1)               ⋯                      𝒟_L^*(λ_K)                               γ ].

	The dual problem (D) admits an augmented Lagrangian of the form
	
    L_σ (S,{Λ_ij}_i<j, {λ_j}_j,γ; M) := -∑_i<j≤ K⟨Λ_ij, w ⟩ -∑_j∈[K]⟨λ_j, z⟩ - γ +  σ/2J - 
    	ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ)  - S-σ^-1 M_F^2 - M_F^2/2σ

	with a penalty parameter σ > 0. Then, the ALM algorithm for solving (D) is summarized in Algorithm <ref>. 
	
	In Algorithm <ref>, it is worth noting that the primal variable M always satisfies the linear constraint in (P) during the updates. This arises from the property that ALM always provides dual-feasible variables <cit.> (where (P) is the dual problem of (D)). 
	
	In the next subsection, we detail how to perform the minimization sub-problem in Algorithm <ref>. Indeed, since {Λ_ij}_i<j, {λ_j}_j, γ are unconstrained, one can eliminate them from the minimization subproblem, leaving us with a minimization problem only in terms of S. 
	
	
	
	

 §.§ Minimization subproblem in Algorithm <ref>
 
	
	The subproblem in Step <ref> of Algorithm <ref> is a joint minimization problem involving both S and the rest of the dual variables. However, since the minimization subproblem is unconstrained with respect to {Λ_ij}_i<j, {λ_j}_j, γ, one can first eliminate them using the first-order optimality condition and express them in terms of S (and M). We are then left with a minimization subproblem that involves only S. More precisely, 
	
    Λ_ij(S^(2)_ij,S^(2)_ji) = (𝒜_U 𝒜_U^*+ 𝒜_L 𝒜_L^*)^-1( 𝒜_U (J^(2)_ij -S^(2)_ij - σ^-1M^(2)_ij )+ 𝒜_L(J^(2)_ji -S^(2)_ji - σ^-1 M^(2)_ji ) + w/σ),   i<j,  i,j∈ [K],

	
    λ_j(S^(2)_jj,S^(1)_j,S^(1)_j^*) = ( 𝒟𝒟^*+𝒟_U 𝒟_U^* + 𝒟_L 𝒟_L^*)^-1(𝒟(J^(2)_jj-S^(2)_jj -σ^-1M^(2)_jj)+
    𝒟_U(J^(1)_j-S^(1)_j-σ^-1M^(1)_j)+𝒟_L(J^(1)_j^*-S^(1)_j^*-σ^-1M^(1)_j^*)+ z/σ),  i∈ [K],

	and
	
    γ(S^(0)) = J^(0)-S^(0) -σ^-1M^(0) +σ^-1.

	
	
	We now substitute the above dual variables into L_σ:
	
    L^S_σ(S; M) := L_σ(S,{Λ_ij(S^(2)_ij,S^(2)_ji)}_i<j, {λ_j(S^(2)_jj,S^(1)_j,S^(1)_j^*)}_j,γ(S^(0)); M)

	and instead of minimizing L_σ (S,{Λ_ij}_i<j, {λ_j}_j,γ; M), we can minimize L^S_σ(S; M) as we have just eliminated all other dual variables from L_σ (S,{Λ_ij}_i<j, {λ_j}_j,γ; M) except S using first-order optimality conditions. The ALM solely in S is summarized in Algorithm <ref>.
	
	
	
	
	

§ ALM WITH HIERARCHICAL DUAL PSD VARIABLE

	As shown previously, the ALM method (Algorithm <ref>) requires minimizing (<ref>), which is an optimization problem over the PSD cone. Typically, this requires  computing the projection onto the PSD cone, and the computational complexity of this projection is cubic, making it impractical for large-scale problems. 
	
	In <cit.>, the authors propose a solution by introducing a change of variables for the PSD variable in the form of S = RR^*. This strategy effectively circumvents the difficult PSD constraint in the optimization problem, converting it into an unconstrained optimization problem. Moreover, when low-rank solutions of the SDP problem exist, the number of columns of R is chosen minimally, enabling the development of an efficient algorithm using the limited-memory BFGS algorithm. However, experiments conducted using the package <cit.> to directly solve either the primal problem (P) or the dual problem (D) for the TFI model (Section <ref>) indicate a linear increase in the rank in N of both the primal PSD variable M and the dual PSD variable S. Hence, employing a vanilla low-rank decomposition of M or S to solve either the primal or dual problem via the limited-memory BFGS algorithm is unlikely to yield substantial reductions in computation time. 
	
	In this section, we propose a structure for the dual variable S in Algorithm <ref> that allows us to perform the ALM updates with reduced time complexity. For a 1-D TFI model (<ref>) with a small system size N=128 (CK = 384) and an external magnetic field strength parameter h=1, we solve (D) using an ADMM-type method with direct projection onto the PSD cone. We show a heatmap of the PSD variable S in Figure <ref>. We can see from the plot that even though S is not low-rank, it is nearly zero except on a few diagonals near the main diagonal. This observation inspired us to represent the dual PSD variable S using a hierarchical low-rank matrix <cit.>, resulting in an algorithm with quadratic scaling of the per-iteration time complexity. We emphasize that the matrix structure we propose is different from the typical hierarchical structure in the literature in order to encode the PSDness of the variable S. For simplicity, we assume that K is a power of 2 in this and the following sections.
	
	
	
	
	

 §.§ Approximating S with a hierarchical matrix
 
	
	Let S denote the solution of the dual problem (D) with size (CK+1)× (CK+1). We first examine the S^(2) block of the matrix S (as defined in (<ref>)). While our objective is to use a hierarchical matrix to represent S^(2), we must also ensure that S^(2) remains a positive semidefinite matrix. To this end, we use m levels of hierarchy to characterize S^(2). For the l-th level, we define a block diagonal matrix with n_l diagonal blocks, where each block is of size c_l× c_l. Furthermore, we want the block diagonal matrix to be positive semidefinite. Therefore, for the l-th level, we form a matrix:
	
    ^(2)(y^(l)) :=
    	[     y^(l)_1 (y^(l)_1)^*;                             y^(l)_2 (y^(l)_2)^*;                                                                       ⋱;                                                                         y^(l)_n_l (y^(l)_n_l)^* ]∈_̋+^CK,    y^(l)_j ∈^c_l × r_l,    1 ≤ j ≤ n_l.

	This naturally requires n_l c_l = CK, since the size of the matrix S^(2) is CK× CK. Then, as an approximation to S^(2), we define
	
    ^(2)(y) := ^(2)(y^(1))+^(2)(y^(2))+⋯+^(2)(y^(m))
    
    	:=y^(1) (y^(1))^* + 
    	[ y^(2)_1 (y^(2)_1)^*                   0;                   0 y^(2)_2 (y^(2)_2)^* ]
    
    	+⋯+[     y^(m)_1 (y^(m)_1)^*                       0                       ⋯                       0;                       0     y^(m)_2 (y^(m)_2)^*                                               0;                       ⋮                                               ⋱                       ⋮;                       0                       ⋯                       0 y^(m)_n_m (y^(m)_n_m)^*;                         ],

	with 
	
    y := {y^(1), y^(2), ⋯, y^(m)},   and   y^(l) := 
    	[   y_1^(l);   y_2^(l);         ⋮; y_n_l^(l) ]∈^CK × r_l.

	Here, we assume n_l = 2^l-1 for 1 ≤ l ≤ m and n_m = 2^m-1 < K. Our proposal involves representing S^(2) with ^(2)(y), where the number of levels m and the number of columns for each level r_l for 1 ≤ l ≤ m are determined based on the desired accuracy of the algorithm. 
	
	
	
	To approximate the full matrix S, which has one extra row and one extra column compared to S^(2), we just pad ^(2)(y) with an extra row and column of zeros, plus a low-rank matrix:
	
    (y,t) := 
    	[  ^(2)(y) 0_CK × 1; 0_1 × CK  0_1 × 1 ]+tt^* ≈ S.

	Here, y = {y^(1), ⋯, y^(m)} is a set of matrices with y^(l)∈^CK × r_l, and t ∈^CK+1.
	
	
	
	
	

  §.§.§ Validity of the hierarchical matrix representation for S
 
	We now investigate the validity of representing S by a hierarchical matrix (y,t). For this purpose, we use the TFI model as a test problem and investigate the relationship between the system size N, the number of levels m needed, and the number of columns needed for each y^(l). We first solved the dual problem (D) with the Hamiltonian specified in (<ref>), with system sizes N=64, 128, 256, and an external magnetic field strength parameter h=1. For these problem sizes, we solved for S in the full PSD cone rather easily with an accuracy of 10^-4. Then, to see how well these PSD S can be approximated by a hierarchical structure, we fitted the resulting PSD dual variable S with the structure outlined in (<ref>). Let S^* denote the approximate solution to the dual problem (D). Additionally, let y,t be complex-valued parameters that parameterize a hierarchical matrix of the corresponding size, with number of levels m = 3, 4, 5 respectively for N= 64, 128, 256. The number of columns for each level was fixed at 20 for all system sizes. We solved the following optimization problem with system sizes N=64,128,256 for the variables y and t:
	
    min_y, z  Err_S^2 := (y,t)-S^*_F^2/S^*_F^2,

	by running the limited-memory BFGS algorithm provided in the toolbox <cit.> for 1000 iterations. The approximation errors are presented in Table <ref>. From the table, we can see that even with fixed r_l = 20 for all 1 ≤ l ≤ m, we obtained similar accuracy for different system sizes. Therefore, we assume one can use a fixed rank approximation in the hierarchical matrix even for large system sizes.
	
	
	
	
	

 §.§ Update rule with a hierarchically structured variable S
 
	
	
	By substituting the PSD variable S in (<ref>) with a data-sparse hierarchical PSD representation, we can eliminate the challenging PSD constraint on S, hence significantly reducing the per-iteration computational costs. With this hierarchical representation of S, when performing Algorithm <ref> where one needs to minimize the variable-reduced augmented Lagrangian function L^S_σ, we replace S with the hierarchical matrix (y,t) defined in (<ref>). We remind the reader again y is a collection of matrices y := {y^(1), y^(2), ⋯, y^(m)} for y^(l)∈^CK × r_l and t ∈^(CK+1) × 1, with pre-specified number of levels m and number of columns r_1, ⋯, r_m. The resulting algorithm is outlined in Algorithm <ref>.
	
	
	
	We now conduct a complexity analysis of Algorithm <ref>, examining its computational scaling step by step.  To remind the readers, the Lagrangian  L_σ can be split into three different terms:
	
    -∑_i<j⟨Λ_ij, w ⟩ + ∑_i< jσ/2( J^(2)_ij - 𝒜^*_U(Λ_ij) - S^(2)_ij - σ^-1 M^(2)_ij_F^2 + J^(2)_ji - 𝒜^*_L (Λ_ij) - S^(2)_ji - σ^-1 M^(2)_ji_F^2  ),

	
    -∑_j⟨λ_j,z⟩  + σ/2∑_j J^(2)_jj - 𝒟^*(λ_j)- S^(2)_jj - σ^-1 M^(2)_jj_F^2  
    
    	+ σ/2∑_j  J^(1)_j - 𝒟^*_U(λ_j)- S^(1)_j - σ^-1 M^(1)_j_F^2 +σ/2∑_j  J^(1)_j^* - 𝒟^*_L(λ_j)- S^(1)_j^* - σ^-1M^(1)_j^* _F^2,

	and
	
    -γ  + σ/2| J^(0) -γ- S^(0) - σ^-1 M^(0)|^2,

	and L^S_σ is obtained by substituting (<ref>), (<ref>) and (<ref>) into (<ref>), (<ref>) and (<ref>) respectively.
	
	
	
	
	Suppose we use gradient-based methods such as the limited-memory BFGS algorithm for Step 2 in Algorithm <ref>. Since the complexity of computing the gradient is asymptotically the same as the complexity of evaluating the loss function <cit.>, we simply analyze the computational cost of evaluating L^S_σ. The key operations in evaluating the loss L^S_σ consist of evaluating the terms (<ref>), (<ref>) and (<ref>). Since (<ref>) and (<ref>) have O(CK) terms and O(1) terms respectively, the computational complexity for these terms is negligible compared to (<ref>), which has O(C^2 K^2) terms. When substituting (<ref>) into (<ref>), we have 
	
    -∑_i<j⟨Λ_ij(S^(2)_ij,S^(2)_ji), w ⟩ + ∑_i< jσ/2( J^(2)_ij - 𝒜^*_U(Λ_ij(S^(2)_ij,S^(2)_ji)) - S^(2)_ij - σ^-1 M^(2)_ij_F^2 
     + J^(2)_ji - 𝒜^*_L(Λ_ij(S^(2)_ij,S^(2)_ji) ) - S^(2)_ji - σ^-1 M^(2)_ji_F^2  ).

	The question now is, with S = H(y,t), meaning S^(2) is the sum of H^(2)(y) plus a rank-one correction term, what is the complexity of evaluating (<ref>). As the rank-one correction term can also be assimilated into the hierarchical structure, by adding one more column to the first hierarchy y^(1), we can, without loss of generality, assume S^(2) is equal to a hierarchical matrix. From the construction of w in the linear constraints in (<ref>), we know w is a matrix of all zeros, so the term ∑_i<j⟨Λ_ij(S^(2)_ij,S^(2)_ji), w ⟩ disappears. Additionally, the remaining two terms J^(2)_ij - 𝒜^*_U(Λ_ij(S^(2)_ij,S^(2)_ji)) - S^(2)_ij - σ^-1 M^(2)_ij_F^2 and J^(2)_ji - 𝒜^*_L (Λ_ji(S^(2)_ij,S^(2)_ji) ) - S^(2)_ji -σ^-1 M^(2)_ji_F^2 take similar forms, so it suffices to just analyze the complexity of the first term.
 
    As w is the zero matrix, from the definitions of the operator 𝒜^*_U and the optimal Λ_ij(S^(2)_ij,S^(2)_ji) in (<ref>) and (<ref>), it is apparent that ( J^(2)_ij - 𝒜^*_U(Λ_ij(S^(2)_ij,S^(2)_ji)) - S^(2)_ij - σ^-1 M^(2)_ij) is a linear function of J_ij^(2), J_ji^(2), S_ij^(2), S_ji^(2), M_ij^(2) and M_ji^(2). Therefore, we need to evaluate a term of the form 
    ∑_i < j^<_S(S_ij^(2))+^>_S(S_ji^(2))+^<_J(J_ij^(2))+^>_J(J_ji^(2))+^<_M(M_ij^(2))+^>_M(M_ji^(2))_F^2,
 
    where A^<_S, A^<_J, A^<_M, A^>_S, A^>_J, A^>_M: ^C × C→^C × C are all linear operators. The complexity of such an evaluation, up to a constant independent of S^(2), is dominated by the inner products between these terms, and Lemma <ref> gives the complexity of these operations.
    
    It is important to note that although w is assumed to be the zero matrix in our proposed relaxation, the complexity analysis can be conducted in a similar way as long as w is a constant matrix. To illustrate this, let W ∈^CK × CK be a matrix in which every C × C block matrix is equal to w. Consequently, W is a matrix with rank less than or equal to C, and thus is also a hierarchical matrix. Therefore, we can apply Lemma <ref> to analyze the complexity of the inner product between S^(2) and W.
	
	
		Assuming C is a constant, the complexity of
		
    ∑_i<j≤ K(B_ij𝒜(B'_ij))

		for some matrices B,B'∈ℂ^CK× CK and some linear operator 𝒜:ℂ^C× C→ℂ^C× C is:
		
			
  * O(K m^2 r^2) if both B and B' are hierarchical matrices in the form of (<ref>) with m levels and each level having rank r.
			
  * O(K^2 r) if B is an arbitrary matrix and B' is a hierarchical matrix.  
			
  * O(Kmr) if B is sparse with O(K) non-zero entries and B' is a hierarchical matrix.
		
	
	
		For any linear operator 𝒜:ℂ^C× C→ℂ^C× C, ∑_i<j≤ K(B_ij(B'_ij)) can be written as 
        
    ∑_i<j
    i,j∈ [K]∑_k,κ,k',κ'∈ [C] D_(k,k',κ,κ')B_ij(k,κ)B'_ij(k',κ')
 
        for a 4-tensor D_ whose values depend on . If both B and B' take the form in (<ref>), the sum 
 
        ∑_i<j
i,j∈ [K]B_ij(k,κ)B'_ij(k',κ') can be computed with O(Km^2 r^2) complexity (see Proposition <ref>). Then the sum ∑_k,κ,k',κ'∈ [C] contributes a factor of C^4, giving a total complexity of  O(C^4 K m^2 r^2). We ignore the factor C^4 as C is assumed to be a constant. 
		
		The complexity of the second statement can be shown in a similar way. If B' is a hierarchical matrix, using Proposition <ref>, one can show that the sum ∑_i<j
i,j∈ [K]B_ij(k,κ)B'_ij(k',κ') can be computed with O(K^2 r) complexity. The summation ∑_k,κ,k',κ'∈ [C] further contributes a factor of C^4, which is again ignored as C is assumed to be a constant.
		
		The last statement is a direct consequence of B being a sparse matrix. 
	
	
	Based on this lemma, the complexity of ∑_i < j⟨^<_S(S_ij^(2)), ^<_J(J_ij^(2))+^>_J(J_ji^(2)) ⟩ and 
 ∑_i < j⟨^>_S(S_ji^(2)), ^<_J(J_ij^(2))+^>_J(J_ji^(2)) ⟩ is O(Kmr), due to the fact that J is sparse with O(K) non-zero entries. The complexity of ∑_i<j^<_S(S_ij^(2))+^>_S(S_ji^(2))_F^2 is O(Km^2r^2), and the complexity of ∑_i < j⟨^<_S(S_ij^(2)), ^<_M(M_ij^(2))+^>_M(M_ji^(2)) ⟩ and ∑_i < j⟨^>_S(S_ji^(2)), ^<_M(M_ij^(2))+^>_M(M_ji^(2)) ⟩ is O(K^2 r). Assuming r is a constant and m = log_2(CK), the computational cost is dominated by the inner products between blocks of S and M as it has a quadratic growth with respect to K. This stems from the fact that M^(2) is an unstructured matrix. At this point, we have successfully reduced the per-iteration cost of vanilla ALM (Algorithm <ref>) from O(K^3) to O(K^2) by assuming S^(2) takes the form of a hierarchical positive semidefinite matrix in Algorithm <ref>. 
	
	
	
	
	
	
	
	
	

§ ALM WITH HIERARCHICAL PRIMAL AND DUAL PSD VARIABLES

	
	As analyzed in Section <ref>, Algorithm <ref> has an O(K^2) per-iteration complexity due to the lack of structure in the primal variable M. While this is already a speed-up compared to a vanilla ALM with cubic complexity, we propose replacing the direct update rule in  Step <ref> of Algorithm <ref> with a projection step that compresses M in order to obtain a nearly linear per-iteration cost. 
	
	Before discussing how to form a compressed representation for the primal variable, we rewrite the primal variable update in Algorithm <ref> as the solution to the following problem:
	
    _M̃  M̃ - (M + σ(S-J+ ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ))))_F^2, 
       s.t.(M̃) = b.

	Here, S is represented hierarchically as S = (y,t), with  {Λ_ij}_i<j, {λ_j}_j, γ being defined in (<ref>), (<ref>) and (<ref>). The linear constraint (M̃)=b summarizes all linear constraints on the matrix M. Although this constraint in (<ref>) may seem redundant since the primal variable always satisfies the linear constraint in (P) during the updates, it becomes crucial when replacing M̃ with a compressed representation, as the constraint will not be naturally satisfied.
	
	To this end, we aim to find an affine function g: ^̋CK+1→^̋CK+1 such that 𝒜(g(A)) = b for any A ∈^̋CK+1. A choice of g is such that g(A) = 𝒫_null(𝒜)(A)+𝒜^†(b). Here, 𝒜^† denotes the pseudo-inverse of the operator 𝒜. Moreover, the linear constraints in cluster moment relaxation are local in nature and only constrain local blocks. This means one can choose g with
	
    g(A) := 
    	[    g^(1)(A^(2)_11,A^(1)_1)                                                                                                               g̃^(1)(A^(2)_11,A^(1)_1);                               g^(1)(A^(2)_22,A^(1)_2)                                                                                                           ⋮;                                                                                ⋱                                                                                 ;                                                                                     g^(1)(A^(2)_KK,A^(1)_K)                              g̃^(1)(A^(2)_KK,A^(1)_K); g̃^(1)(A^(2)_11,A^(1)_1)^*                          ⋯                            g̃^(1)(A^(2)_KK,A^(1)_K)^*                                          g^(0)(A^(0)) ] +
    	
    [                             0_C× C           g^(2)(A^(2)_12,A^(2)_21)                                  ⋯           g^(2)(A^(2)_1K,A^(2)_K1)                                                                0_C× 1;         g^(2)(A^(2)_12,A^(2)_21)^*                             0_C× C                                  ⋱                                                                                                        ⋮;                                  ⋮                                  ⋱                                  ⋱   g^(2)(A^(2)_(K-1)K,A^(2)_K(K-1))                                                                      ;         g^(2)(A^(2)_1K,A^(2)_K1)^*                                  ⋯ g^(2)(A^(2)_(K-1)K,A^(2)_K(K-1))^*                             0_C× C                                                                0_C× 1;                             0_1× C                                  ⋯                                                                0_1× C                                                                     0 ].

	where g^(2):ℂ^C× C×ℂ^C× C→ℂ^C× C, g^(1):ℂ^C× C×ℂ^C× 1→ℂ^C× C, g̃^(1):ℂ^C× C×ℂ^C× 1→ℂ^C× 1 and g^(0):ℂ→ℂ are all affine transformations that depend on the constant vector b. For example, in order to satisfy the linear constraints of the TFI model of N sites (Section <ref>), we let g: ^̋3N+1→^̋3N+1 be:
	
    g(A) = I_3N+1 + 
    	([         0  A^(2)_12         ⋯  A^(2)_1N   A^(1)_1;  A^(2)_21         0         ⋯  A^(2)_2N   A^(1)_2;         ⋮         ⋮         ⋱         ⋮         ⋮;  A^(2)_N1  A^(2)_N2         ⋯         0   A^(1)_N; A^(1)_1^* A^(1)_2^*         ⋯ A^(1)_N^*         0 ]) 
    
    	+2[ ^*((A^(1)_1))                                          ;                           ⋱                            ;                             ^*((A^(1)_N))              ;                                                       0 ],

	with 
    (D): = 
    	[ D(2,3); D(3,1); D(1,2) ],   : ^̋3 →^3.

	This reparameterization of primal-feasible matrices allows us to rewrite (<ref>) in an equivalent form:
	
    _A∈^̋CK+1  g(A)- (M + σ(S-J+ ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ)) )_F^2.

	Our final step is to approximate A ∈^̋CK+1 with a hierarchical matrix to reduce the cost of the loss function evaluation:
	
    _x,v,A  g(A)- (M + σ(S-J+ ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ)))_F^2 
       s.t. A = (x,v),

	where x={x^(i)∈^CK × r_i}_1 ≤ i ≤ m and v ∈^(CK+1) × 1 for some pre-specified number of levels m, and number of columns r_1, ⋯, r_m. In other words, we propose a hierarchical representation for the parameterization of any primal-feasible variable M. This allows efficient evaluation of the loss function in (<ref>). If the current guess M is represented as M = g((x, v)) for some x and v, and S = (y,t), then the main computational task in evaluating the loss function (<ref>) can be reduced.
	
	
	
	
	We now introduce Algorithm <ref>, which utilizes a hierarchical representation for both the primal and the dual PSD variables.
	
	We highlight that M and S are never explicitly formed as (CK+1)× (CK+1) matrices to ensure efficient computations. The per-iteration computational complexity of Algorithm  <ref> can be analyzed following the approach in Section  <ref> for Algorithm <ref>. We assume the hierarchical representation of the primal and dual PSD variables has the same number of levels m, with a constant number of columns r for each level. As before, in Step <ref> and Step <ref> of the algorithm, the evaluation of the loss consists of computations in Lemma <ref>. When we replace both primal and dual variables with hierarchical matrices, the second scenario in Lemma <ref> is eliminated, and we achieve a near-linear complexity of O(Km^2r^2) with an r that does not grow with K and m = O(log_2(CK)).
	
	

 §.§ Validity of the hierarchical matrix representation for M

        In this section, we investigate the validity of representing M by a hierarchical matrix (x,v), using the same method as in section <ref> for the dual PSD variable S. Instead of fitting the dual PSD variable S, we fitted the primal PSD variable M resulting from solving (D) with the Hamiltonian specified in (<ref>), for system sizes N=64, 128, 256, an external magnetic field strength parameter h=1 and an accuracy of 10^-4. Let M^* be the approximate solution. Let x, v be complex-valued parameters that parameterize a hierarchical matrix of the corresponding
        size, with the number of levels m = 3, 4, 5 respectively for N = 64, 128, 256. The number of columns for each level was fixed at 20 for all system sizes. We solved the following optimization problem
        with system sizes N = 64, 128, 256 for the variables x and t:

        
    min_x,v  Err_M^2 := (x,v)-M^*_F^2/M^*_F^2,

	by running the limited-memory BFGS algorithm provided in the toolbox <cit.> for 1000 iterations. The approximation errors are presented in Table <ref>. The experiment results indicate that as the system size increases, the relative error of the fitted primal PSD variable also tends to increase. This likely contributes to the failure of Algorithm <ref> for  large system sizes, such as N=4096 (see Section <ref> for further details on the experiments). 
	
	
	
        
	
	

§ NUMERICAL EXPERIMENTS

	In this section, we present numerical experiments for the 1-D TFI model using Algorithms <ref> and <ref>, with system sizes N ∈{64,128,256,512,1024,2048,4096}. The penalty parameter is initialized at σ = 0.1 and is adjusted dynamically <cit.> based on primal and dual feasibility to speed up the convergence of the ALM algorithm. For both algorithms, the number of levels m in the hierarchy is set to be m = 3,4,⋯,9 for N=64,128,⋯,4096, and the number of columns for all levels is set to be r=20. In Algorithm <ref>, y, t and M are initialized from the standard normal distribution, with M being primal-feasible. This is achieved by having the initial primal variable M=g(A) for some A∈^̋3N+1, where g is defined in (<ref>). In Algorithm <ref>, x, v, y and t are randomly initialized from the standard normal distribution. Throughout the updates of Algorithms <ref> and <ref>, we evaluate the accuracy of approximate solutions by monitoring the relative primal feasibility, the relative dual feasibility, and the relative duality gap, as detailed in the following.
	
	The primal feasibility of the variable M is governed by how well it satisfies the PSD and linear constraints in (P). In Algorithm <ref>, M is directly updated as a dense matrix, while in Algorithm <ref>, M is maintained using a compressed representation that satisfies the linear constraints. Since M is guaranteed to satisfy the linear constraints in (P), we only monitor its PSDness using the following measure:
	
    η_P := max(0, -λ_min)/1+max(0, λ_max),
 where λ_min and λ_max are the smallest and largest eigenvalues of M.
	
	For the dual problem (D) and a candidate dual variable S, the PSD constraint for S is automatically satisfied because S is maintained as a positive semidefinite hierarchical matrix in both algorithms. We thus only monitor the dual feasibility by how well the dual equality constraint is satisfied using the following measure:
	
    η_D:= S - J
    		+ℱ^*({Λ_ij}_i<j,{λ_j}_j,γ) _F/1+J_F.

	
	Finally, we monitor the relative duality gap by:
	
    η_g := |primal objective-dual objective|/1+|primal objective|+|dual objective|.
 
	
	We terminate the algorithm when η := max(η_P, η_D, η_g) ≤ 10^-3, or when the ALM algorithm has run for 1500 iterations. It is important to highlight that we exploit the hierarchical structure present in the PSD primal and dual variables to efficiently evaluate these convergence metrics.
	
	We examine the ground-state energy recovery for the TFI model on an N × 1 lattice, for system sizes N ∈{64, 128, 256, 512, 1024, 2048, 4096} and an external magnetic field strength parameter h ∈{0.1, 1, 1.5}. Let E_0 denote the true ground-state energy and Ẽ_0 the lower bound of the ground-state energy obtained from (P). The relative error is defined as:
	
    Err_rel := E_0 - Ẽ_0/|E_0|.
 The relative errors for Algorithms <ref> and <ref> are given in Tables <ref> and <ref>. Additionally, we present the evolution of our convergence metrics as a function of the ALM iteration number in Figures <ref> and <ref>, focusing on the 1-D TFI model with a fixed external magnetic field strength parameter h=1 and various system sizes. Alongside the relative primal and dual feasibility measures and the relative duality gap, we also track the per-site primal objective change between subsequent iterations. All convergence metrics are transformed using a base-10 logarithm function. Within 1500 ALM iterations, all metrics drop below 10^-3 for experiments with an external magnetic field strength parameter h=1. We note that it is harder to make Algorithm <ref> converge compared to Algorithm <ref>. For example, we are not able to get the case for N=4096 to converge to the prescribed accuracy 10^-3. We suspect this is due to the fact that it is harder to fit the primal PSD variable by a low-rank hierarchical matrix compared to the dual PSD variable.
	
	
	
	
	
	
	
	
	
	
	
	
	

§ CONCLUSION

	In this paper, we explored the computation of a specific semidefinite relaxation for determining the ground-state energy of many-body problems, which can be solved in polynomial time and provides a reasonable lower bound for the ground-state energy. Additionally, we identified a hierarchical structure in both the positive semidefinite (PSD) primal and dual variables, allowing us to circumvent the expensive projection onto the PSD cone, thereby reducing the per-iteration complexity of the ALM-type algorithm from cubic to quadratic or almost linear. 
	
	The relaxed problem provides only a lower bound for the lowest energy. To evaluate the effectiveness of our approach, we compare the recovered lower bound with the true ground-state energy for the 1-D transverse field Ising model. Notably, for the most challenging case of h=1, where the system undergoes a quantum phase transition, our algorithm still produces a reasonable lower bound. Furthermore, our algorithm can handle systems consisting of up to 4096 spins, whereas previous work on the variational embedding method, such as <cit.>, which employs more accurate yet more expensive constraints, can only manage systems of a few dozen spins without leveraging the periodicity of the model.
	
	Currently, unlike <cit.>, we do not impose conic constraints on each of the off-diagonal blocks of the moment matrix. Including them while leveraging the computational benefits of a hierarchical matrix will be the subject of future investigations. 

    

§ ACKNOWLEDGEMENTS

        Y. W. acknowledges partial support by ASCR Award DE-SC0022232 from the Department of Energy. Y.K. acknowledges partial support by NSF-2111563, NSF-2339439 from the National Science Foundation. Y.K. also acknowledges various interesting discussions with Michael Lindsey on speeding up semidefinite programming. 
	
	
	

§ COMPLEXITY OF OPERATIONS WITH HIERARCHICAL MATRICES
 
	
	In this section, we present several propositions that describe the complexity of manipulating hierarchical matrices.
	
	 
		
			
  * Let A=a_1 a_2^* and B=b_1 b_2^* be two low-rank matrices with a_1, a_2, b_1, b_2∈ℂ^p× r. Then ∑_i<j≤ pA(i,j)B(i,j) has a complexity of O(pr^2).
			
  * Let A=a_1 a_2^* be a low-rank matrix with a_1, a_2∈ℂ^p× r and B∈ℂ^p× p. Then ∑_i<j≤ pA(i,j)B(i,j) has a complexity of O(p^2 r).
		
	
	Based on this, we have the following propositions:
	 
		For two hierarchical matrices A =ℋ^(2)(y) and A' =ℋ^(2)(y') with y = {y^(1), ⋯, y^(m)} and y' = {y'^(1), ⋯, y'^(m)}, where y^(l), y'^(l')∈^CK × r for 1 ≤ l,l' ≤ m, the formula 
		
    ∑_i<j≤ K A_ij(k,κ) A_ij'(k',κ'),  ∀ k,k',κ,κ'

		can be computed with O(Km^2 r^2) complexity. 
	
	
		
    ∑_i<j≤ K A_ij(k,κ) A_ij'(k',κ')   =   ∑^m_l=1∑^m_l'=1∑_i<j≤ K(_ij^(2)(y^(l))(k,κ)  _ij^(2)(y'^(l'))(k',κ')).

		For fixed k, k', κ and κ', [_ij^(2)(y^(l))(k,κ)]_i,j∈ [K] and [_ij^(2)(y'^(l'))(k',κ')]_i,j∈ [K] take the form of 
		
    B = [   B_1^(l);             B_2^(l);                             ⋱;                               B_n_l^(l) ],   B' = [    B'_1^(l');                 B'_2^(l');                                      ⋱;                                        B'_n_l'^(l') ]

		respectively, where the diagonal blocks B_i^(l)∈ℂ^K/n_l× K/n_l and B'_j^(l')∈ℂ^K/n_l'× K/n_l' are rank-r matrices. The complexity of ∑_i<j(_ij^(2)(y^(l))(k,κ)  _ij^(2)(y'^(l'))(k',κ')) can thus be obtained by applying Proposition <ref> (Part 1) for max{n_l,n_l'} times, each time having O(K/max{n_l,n_l'}r^2) complexity, which gives a total complexity of O(K r^2). Incorporating the double sum ∑^m_l=1∑^m_l'=1 gives a final complexity of O(Km^2r^2)
	
	
	
	
	
	
	 
		For a hierarchical matrix A =ℋ^(2)(y) with y = {y^(1), ⋯, y^(m)} where y^(l)∈^CK × r for 1 ≤ l ≤ m, and a dense matrix A'∈ℂ^CK× CK, the formula 
		
    ∑_i<j≤ K A_ij(k,κ) A_ij'(k',κ'),  ∀ k,k',κ,κ'

		can be computed with O(K^2 r) complexity. 
	
	
		
    ∑_i<j≤ K A_ij(k,κ) A_ij'(k',κ')    =   ∑^m_l=1∑_i<j≤ K(_ij^(2)(y^(l))(k,κ)   A_ij'(k',κ')).

		For fixed k and κ, [_ij^(2)(y^(l))(k,κ)]_i,j∈ [K] takes the form of 
		
    B = [   B_1^(l);             B_2^(l);                             ⋱;                               B_n_l^(l) ],

		with some matrices B^(l)_i ∈ℂ^K/n_l × K/n_l each having rank r. With this form, the complexity of 
 ∑_i<j≤ K_ij^(2)(y^(l))(k,κ)   A_ij'(k',κ') can be determined by applying Proposition <ref> (Part 2) n_l times, each time having complexity O(rK^2/n_l^2 ), which gives a total complexity of O(rK^2/n_l). Summing this complexity ∑^m_l=1O(rK^2/n_l), we arrive at a final complexity of O(K^2r).
	
	
	
	plain
	


 Paper Title: Constraint Energy Minimizing Generalized Multiscale Finite Element Method for Convection Diffusion Equations with Inhomogeneous Boundary Conditions.

=1








 




















theoremTheorem[section]
CorollaryCorollary[theorem]
lemma[theorem]Lemma
ProblemAbstract Problem
AssumptionAssumption
remarkRemark

references0.bib



































.5
-1in
-1in






Abstract
        
	§ ABSTRACT 

        


keywords
        
	§ ABSTRACT 

        



1]Po Chai WONG
1]Eric T. CHUNG
1]Changqing YE
2]Lina ZHAO

[1]Department of Mathematics, The Chinese University of Hong Kong, Hong Kong SAR, China. {pcwong, tschung, cqye}@math.cuhk.edu.hk
[2]Department of Mathematics, City University of Hong Kong, Hong Kong SAR, China. linazha@cityu.edu.hk



[L]CEM-GMsFEM for Convection Diffusion with Inhomogeneous Boundary Conditions

[R]
fancy


same 


Constraint Energy Minimizing Generalized Multiscale Finite Element Method for Convection Diffusion Equations with Inhomogeneous Boundary Conditions
    [
    August 5, 2024
===================================================================================================================================================




§ ABSTRACT 
    
In this paper, we develop the constraint energy minimizing generalized multiscale finite element method (CEM-GMsFEM) for convection-diffusion equations with inhomogeneous Dirichlet, Neumann and Robin boundary conditions, along with high-contrast coefficients. For time independent problems, boundary correctors 𝒟^m and 𝒩^m for Dirichlet, Neumann, and Robin conditions are designed. For time dependent problems, a scheme to update the boundary correctors is formulated. Error analysis in both cases is given to show the first-order convergence in energy norm with respect to the coarse mesh size H and second-order convergence in L^2-norm, as verified by numerical examples, with which different finite difference schemes are compared for temporal discretization. Nonlinear problems are also demonstrated in combination with Strang splitting. 



constraint energy minimization, multiscale finite element methods, inhomogeneous boundary value problem, convection-diffusion equation






§ INTRODUCTION

Convection diffusion equation is involved in many physical applications of partial differential equations. Computational difficulty may arise in two-fold: (1) coefficients in high contrast and multiple scales and (2) demanding discretization for a high Péclet number. A lot of multiscale effort has contributed to the problems such as multiscale finite element method <cit.>,
variational multiscale method <cit.>, multiscale discontinuous Galerkin method <cit.>, multiscale stablization <cit.>.
In particular, the Generalized Multiscale Finite Element Method (GMsFEM) aims to create multiscale basis to apply Galerkin approximation and it has been applied to an array of partial differential equations <cit.>.
Similar to the finite element method, it consists of two stages: first the offline stage where basis functions are generated and used to span the approximation solution manifold; and second the online stage where the actual approximation is found in the generated space. However, due to the complexity of the problem, it is necessary to further reduce the computational cost of the offline stage. 

A spectral decomposition method, Constraint Energy Minimization Generalized Multiscale Finite Element Method (CEM-GMsFEM), is then applied to the scheme to generate a sufficiently large solution manifold by using a few basis functions <cit.>.
Each basis function is formed by the eigenfunction in a local spectral problem and captures some information about the medium and velocity, and thereby dependent on the Péclet number. One main important property of such eigenfunction is exponential decay. The span of such basis therefore can capture the cell decaying part of the solution while the non-decaying part is left to be dealt with in the online stage. However, it is only common to see the above results conducted in a homogeneous setting whilst, in practice, inhomogeneous boundary conditions are necessary. We propose a numerical scheme to apply CEM-GMsFEM to solve the convection-diffusion equation with inhomogeneous boundary conditions. There are two versions of CEM-GMsFEM: constrained and relaxed. The relaxed-CEM-GMsFEM is used to develop the error analysis, which takes advantage of the elliptic projection of the solution as a bridge between our multiscale space and the continuous space. An account of relaxed-CEM-GMsFEM applied to some inhomogeneous Boundary Value Problems was given already in <cit.>. Our goal is to extend this idea to the convection-diffusion equation. We will study three main cases, Dirichlet, Neumann, and Robin conditions. We also give a theoretical account of the convergence analysis and prove the scheme has first-order convergence in energy norm and second-order in L^2-norm, as verified by numerical examples.

For non-time independent problems, some work of applying CEM-GMsFEM has been done for parabolic equations <cit.> on homogeneous conditions. Our goal is to extend our method to the convection-diffusion equation with inhomogeneous boundary conditions. Since we assume the medium and velocity are independent of time, we can reuse the multiscale space in the time-independent case. For time-invariant boundary conditions, the corrector can be pre-computed once. However, for time-variant boundary conditions, the time derivative of the corrector and the boundary conditions need to be taken into account. Therefore, we give a new formulation to update the corrector at each time step in a relaxed CEM-GMsFEM fashion. The error of such approximation is also proved to be exponentially decaying in space. In addition, there are two versions of Backward Euler schemes to compute the next steps of the solution: the diffusion approach (D-approach) and the convection-diffusion approach (CD-approach). The former is less expensive but can be shown to bear a lower accuracy. We compare them via convection diffusion IBVPs and verify that our proposal gives a more accurate result at a higher computational expense. 

Moreover, we introduce a nonlinearity into the IBVPs, which greatly increases the difficulty. A classical approach is to apply the Strang splitting method which considers convection, diffusion, and the nonlinearity terms in separate intermediate steps. This method utilizes the symmetric property of the algorithm to split operators in intermediate timesteps. This has been applied to a variety of problems including parabolic equations <cit.>,
diffusion-reaction equations <cit.> 
and diffusion-reaction-advection equations with a homogeneous boundary conditions <cit.>. 
However, it was numerically tested that the inhomogeneous boundary conditions would drag the overall accuracy of the algorithm <cit.>.
To resolve this, some classical approaches can be found in <cit.>.
More recent approaches involve designing a time-dependent boundary corrector in the intermediate steps <cit.>,
which is aligned with our previous sections. In particular, in the classical Strang splitting method applied to convection diffusion equations, the convection and diffusion operators are split and considered separately with their corresponding nonlinear terms <cit.>.
Now with our scheme, we can consider them in the same step. We tested our choice of boundary corrector, the same as the previous part, to attain both spatial and temporal convergence. 

The paper is organized as follows: we first give the problem setting and some preliminaries in section 2. The convergence analysis of its application to the time-independent convection diffusion equation is given in section 3, along with numerical results on the Dirichlet, Neumann, and Robin conditions. In section 4, both time-variant and time-invariant IBVPs are presented with analysis and numerical results, along with the comparison of different finite difference schemes for temporal discretization. A demonstration of applying this to nonlinear problems with Strang splitting is presented in this section as well.  


§ PRELIMINARIES



 §.§ Problem setting


We consider the following convection diffusion initial boundary value problem:

    ∂_t u + (x) ·∇ u = ∇·(A(x)∇ u)+f,     in Ω× (0,T],
    
        u = g(x,t),     on Γ_D× (0,T],
    
        b(x) u + ·(A(x)∇ u -(x) u) = q(x,t),     on Γ_N× (0,T],
    
        u(·,0)=u_init,     in Ω,

where Ω⊂ℝ^d is the computational domain, ∈ L^∞(Ω)^2, and 0<T<∞. 
The medium A∈ L^∞(Ω;ℝ^d× d) and the velocity  are heterogeneous coefficients with multiple scales and potentially high contrast. A is a positive definite matrix. There exist κ_0 and κ_1 such that 0<κ_0≤λ_min (A(x))≤λ_max(A(x))≤κ_1 and κ_1/κ_0 can be large.
We denote  the outward unit normal vectors to ∂Ω and Γ_D and Γ_N two nonempty disjoint part of ∂Ω.
We assume the velocity flows inward on the Neumann boundary Γ_N and  is incompressible, i.e.,
∇·=0. i.e., ·≤ 0 on Γ_N. Denote β_0≥ 1 and β_1 as the infimum and supremum of || respectively. 
The function b(x)≥ 0 a.e. x∈∂Ω and there exists a positive constant b_0>0 and a subset Γ⊂Γ_N with positive measure such that b(x)≥ b_0 for a.e. x∈Γ. 
The Dirichlet boundary value term g∈ℋ^1/2(Γ_D×[0,T]) and the Neumann boundary value term q∈ L^2(Γ_N×[0,T]). 

From now on, we denote

    a(w,v)   =∫_ΩA∇ w·∇ v  + ∫_Γ_N(b-·) wv σ,
    𝒜(w,v)   =∫_ΩA∇ w ·∇ v +∫_Γ_N(b-·) wv σ + ∫_Ω(·∇ w) v .


So the variational form of the problem is: for t∈ (0,T], find u_0(·,t)∈ V{v∈ℋ^1(Ω) v=0  on Γ_D} such that at t, 

    (∂_t u_0,v)+𝒜(u_0,v)    = (f,v) - 𝒜(g,v) + (q,v)_Γ_N    for v∈ V
    
    (u_0(·,0),v)   =(u_init-g(·,0),v)     for v∈ V

where g∈ℋ^1(Ω×[0,T]) with g=g on Γ_D. The solution u to equation (<ref>) is then u=u_0+g.


Denote the energy norms on V

    v_a(ω)=(∫_ωA∇ v·∇ v  + ∫_Γ_N∩∂ω(b-·) v^2 σ)^1/2 and v_a = v_a(Ω).

Also, notice that under the assumptions on   and b, for v∈ V,

    𝒜(v,v)    = ∫_ΩA∇ v·∇ v + ∫_Ω (·∇ v) v + ∫_Γ_N (b-· ) v^2 σ
       = ∫_ΩA∇ v·∇ v+ ∫_Γ_N(b-1/2·) v^2 σ≥ 0.

So, the following quasi-norms are also well defined on V:

    v^2_𝒜(ω) = ∫_ωA∇ v ·∇ v + ∫_∂ω∩Γ_N(b - 1/2·) v^2σ,

for ω⊂Ω and v_𝒜=v_𝒜(Ω).
It is also easy to show that ·_𝒜 and ·_a are equivalent on V.



 §.§ Constraint energy minimizing generalized multiscale finite element method



  §.§.§ Auxiliary basis functions


We first introduce 𝒯^H to be a conforming partition of Ω into rectangular elements. 
The resulting elements are called the coarse grids.
Let H be the meshsize of the coarse grid and N be the number of elements. 
For each K_i∈𝒯^H with 1≤ i ≤ N, we define an oversampled domain K^m_i <cit.> by

    K^m_i{(⋃_K∈𝒯^H,
    (K)∩(K^m-1_i)≠∅(K))∪(K^m-1_i)}

where (S) and (S) are the interior and the closure of a set S. 
We also set K^0_i K_i for the consistency of notations.
Let N_ν be the number of vertices contained in an element. We can construct a set of Lagrange bases {η^1_i,…, η^N_ν_i} of the element K_i∈𝒯^H.

On each K_i, we solve a local spectral problem:
find λ^j_i≥ 0, ϕ^j_i∈ℋ^1(K_i)
such that for all v∈ℋ^1(K_i),

    𝒜_(K_i)(ϕ^j_i,v)=λ^j_i s_i(ϕ^j_i,v),

where 

    s_i(w,v) = CH^-2∫_K_iκ_1 ||^2 wv ∫_K_iκwv
 and C is a constant that depends on the choice of basis {η_i} and for the local auxiliary space on a structured mesh such that |∇η_i|^2 ≤ CH^-2.
We take C = 24 since we will use the Lagrange polynomials as the basis.  For simplicity, we will omit the constant C in the analysis.
Arrange the eigenvalues {λ^j_i}^∞_j=0 in an ascending order.
Define the local auxiliary space V^aux_i{ϕ^1_i,…,ϕ^l_i_i} for some l_i. 
Let s(w,v) = ∑^N_i=1s_i(w,v) on V× V.
Denote 
w_s(ω)=√(∫_ωκw^2 x )
and 
w_s=√(s(w,w)).

Now, 
define the orthogonal projection π_i:L^2(K_i)→ V^aux_i by

    π_i(v)∑_j=0^l_is(ϕ^j_i,v)/s(ϕ^j_i,ϕ^j_i)ϕ^j_i.


Let V^aux⊕^N_i=1V^aux_i. Then s(·,·) and ·_s are an inner product and a norm on V^aux respectively. 
Also π∑^N_i=1π_i maps L^2(Ω) to V^aux.
Now, we can derive the following lemma <cit.>:

    For v∈ℋ^1(K_i),
    
    v-π_i(v)_s(K_i)^2≤v^2_a(K_i)/λ^l_i+1_i,

    
    π_i(v)_s(K_i)^2=v_s(K_i)^2
            -v-π_i(v)_s(K_i)^2≤v_s(K_i)^2.




  §.§.§ Multiscale basis functions



Although lemma <ref> has shown that the elliptic projection can approximate any vector v close enough given sufficient number of eigenfunctions used, this function in V^aux may not be continuous in Ω. For this, We now construct the multiscale local basis functions using the auxiliary space V^aux. 
Define

    ψ^j_i={ψ_𝒜^2+πψ-ϕ^j_i_s^2 ψ∈ V},


    ψ^j,m_i={ψ_𝒜^2+πψ-ϕ^j_i_s^2 ψ∈ V^m_i}.

where

    V^m_i{v∈ℋ^1(K^m_i)
    v=0  on Γ_D∩∂ K^m_i or Ω∩∂ K^m_i}.


With these definitions, we can obtain

    𝒜(ψ^j_i,v)+s(πψ^j_i,π v)=s(ϕ^j_i,π v),∀ v∈ V,


    𝒜(ψ^j,m_i,v)+s(πψ^j,m_i,π v)=s(ϕ^j_i,π v), ∀ v∈ V^m_i.


Denote V^glo_ms={ψ^j_i 0≤ j≤ l_i,1≤ i≤ N} and V^m_ms{ψ^j,m_i 0≤ j≤ l_i,1≤ i≤ N}.





It is worth-mentioning that the bilinear form 𝒜 is not symmetric, and therefore is not an inner product on V. However, the following lemma still holds, without stating the proof here for simplicity <cit.>.


    Let v∈ V^glo_ms. Then 𝒜(v,v^')=0 for any v^'∈ V with π v^' =0.
    If there exists v∈ V such that 𝒜(v,v^')=0 for any v^'∈ V^glo_ms, then π v=0. 
    






§ TIME-INDEPENDENT CONVECTION DIFFUSION BOUNDARY VALUE PROBLEMS


As the multiscale space V^m_ms is independent of t, the method is applicable to time independent problems. To illustrate the idea, consider
Find u∈ℋ^1(Ω) such that

    -∇·(A∇ u)+ ·∇ u = f,     in Ω,
    
        u=g,     on Γ_D,
    
        b u + · (A∇ u-  u)=q,     on Γ_N,

where f∈ L^2(Ω) is the source term independent of u.


 §.§ Derivation of the Method



The methods are the following steps:

    
  * Find 𝒟^m_i g∈ V^m_i and 𝒩^m_i q∈ V^m_i such that for all v∈ V^m_i,
    
    𝒜(𝒟^m_ig,v)+s(π𝒟^m_ig,π v)=𝒜_(K_i)(g,v),

    
    𝒜(𝒩_i^m q,v)+s(π𝒩_i^m q,π v)=∫_∂ K_i∩Γ_Nqv σ.

    Further denote 𝒟^mg=∑^N_i=1𝒟^m_ig and 
    𝒩^m q=∑^N_i=1𝒩_i^m q.

    
  * Construct the multiscale function space V^m_ms according to equation (<ref>).
    
  * Solve w^m∈ V^m_ms such that for all v∈ V^m_ms,
    
    𝒜(w^m,v)=(f,v)-𝒜(g,v)+∫_Γ_Nqv σ
            +𝒜(𝒟^mg,v)-𝒜(𝒩^m q,v).


    
  * Construct the numerical solution u^ms to approximate the actual solution u of equation (<ref>) by
    
    u^ms_0 = w^m-𝒟^mg+𝒩^m q  and   u^ms≈ u^ms_0+g.





 §.§ Analysis



We will use the following notation:
for w,v∈ V and ω⊂Ω,

    ℬ_(ω)(w,v) = 𝒜_(ω)(w,v)+s_(ω)(π w,π v)

and 
v_ℬ(ω)√(ℬ_(ω)(v,v)).
Before we give an account of the analysis, we will summarize all the quasi-norms in the following lemma,
 For any v∈ V,
    
    v_L^2≤ Hc_#v_ℬ,

    
    𝒜(w,v)≤Cw_𝒜v_ℬ,

    
    ℬ(w,v)≲Cw_ℬv_ℬ,

    where Λ =min_iλ^l_i+1_i, c_#β_0^-1κ_1^-1/2√(1+Λ^-1), C√(2)(1+Λ^-1/2) max{1, H/√(Cκ_0κ_1)} and ≲ is "not greater than up to a positive constant".
    


    First,
    
    v_L^2≤ Hβ_0^-1κ_1^-1/2v_s
            ≤ Hβ_0^-1κ_1^-1/2(1+Λ^-1)^1/2v_ℬ.

    On the other hand,
    
    𝒜(w,v)    = ∫A∇ w·∇ v + ∫·∇ w v + ∫_∂Ω (b-1/2·)wvσ
       ≤w_a v_a + H√(∫A∇ w ·∇ w)√(∫κv^2)
       ≤w_av_a + Hw_a(π v_s + v-π v_s)
       ≤w_av_a + Hw_a (π v_s + Λ^1/2v_a)
       ≤Cw_𝒜v_ℬ.

    Finally,
    
    ℬ(w,v)    = 𝒜(w,v)+ s(π w, π v)
       ≤Cw_𝒜v_ℬ+π w_s π v_s
       ≲Cw_ℬv_ℬ.



With the above lemmas, we can now start our error analysis.



  §.§.§ Global Approximations


We will approximate the error using the global approximation w^glo of the real solution via V^glo_ms.
Define 𝒟^glog=∑^N_i=1𝒟^glo_ig and 𝒩^glo q=∑^N_i=1𝒩^glo_i q where 𝒟^glo_ig∈ V^glo_ms satisfies that for all v∈ V,

    ℬ(𝒟^glo_ig,v) = 𝒜(g,v),

and 𝒩^glo_i q∈ V^glo_ms satisfies that for all v∈ V,

    ℬ(𝒩^glo_i q,v)=∫_∂ K_i∩Γ_N q v σ.


Also define w^glo∈ V^glo_ms such that,

    𝒜(w^glo,v) - 𝒜(𝒟^glog, v) + 𝒜(𝒩^glo q, v) = 𝒜(u_0,v)  for  v∈ V^glo_ms;


We first show the convergence of the elliptic projection defined using V^glo_ms.

    Let 𝒟^glo_ig, 𝒩^glo_i q and w^glo defined by equations (<ref>), (<ref>) and (<ref>). 
    Let u be the actual solution of the problem (<ref>). 
    Let u_0 =w^glo -𝒟^glog +𝒩^glo q and u = u_0+g. 
    Then
    
    u-u_𝒜≤Λ^-1/2κ_1^-1/2H(||^-1f_L^2(Ω)+∇ u_0_L^2(Ω)).

    



    Let e=u-u. By the definition of u, we have 𝒜(u-u,v)=𝒜(u_0-u_0,v)=0 for v∈ V^glo_ms, giving π e = π(u-u)=0 by equations (<ref>) and (<ref>). This leads to
    
    𝒜(𝒟^glog,e)=𝒜(g,e)  and 𝒜(𝒩^glo q,e)=∫_Γ_Nqeσ.

    Since w∈ V^glo_ms, then 𝒜(w,v) =0 for π v =0.
    
    e_𝒜^2 =𝒜(e,e) 
               = -𝒜(w,e) + 𝒜(𝒟^glog,e) - 𝒜(𝒩^glo q,e) + 𝒜(u_0,e)
       = 𝒜(𝒟^glog,e) - 𝒜(𝒩^glo q,e) + {∫_Ω fe- 𝒜(g,e)-∫_Ω·∇ u_0 e.
            .+∫_Γ_N qv σ}
       =  ∫_Ω fe - ∫_Ω·∇ u_0 e
       ≤κ^-1/2f_L^2(Ω)e_s
            +κ_1^-1/2H ∇ u_0_L^2(Ω)e_s
       ≤Λ^-1/2(κ^-1/2f_L^2(Ω)+κ_1^-1/2H ∇ u_0_L^2(Ω))e_𝒜.


    Hence, we obtain
    
    u-u_𝒜≤Λ^-1/2κ_1^-1/2H(||^-1f_L^2(Ω)+∇ u_0_L^2(Ω)).




  §.§.§ Abstract Problem


Following the approach in <cit.>, we summarise the analysis of CEM-GMsFEM by considering the following abstract problem:

    Let K_i∈𝒯^H and t_i∈ V^' such that <t_i,v>=0 for any v∈ V with supp(v)⊂Ω\ K_i.
    Define 𝒫_i:V^'→ V such that for all v∈ V,
    
    ℬ(,v)=<t_i,v>

    and 𝒫^m_i:V^'→ V^m_i with
    
    ℬ(,v)=<t_i,v>.


We aim to estimate 
    
    ∑^N_i=1-_ℬ^2=∑^N_i=1-_𝒜^2
        +∑^N_i=1-_s^2.



To solve this problem, we prepare ourselves with the following lemmas <ref>, <ref> and <ref>. 
To prove them, we need to define cutoff functions {χ_i^n,m}.
Let V^H be the Lagrange basis function space of 𝒯^H. For K_i∈𝒯^H, 
a cutoff function χ^n,m_i∈ V^H with n<m satisfies that:

    χ^n,m_i(x)=1 in K^n_i;


    χ^n,m_i(x)=0 in Ω\ K^m_i;


    0≤χ^n,m_i≤ 1 in K^m_i\ K^n_i.



    Let m≥ 1. Then there exists 0<θ<1 such that
    
    _ℬ(Ω\ K^m_i)^2
        ≤θ^m_ℬ^2

    where 
    θ=c_*/c_*+1
    and
    
    c_*(Λ,β_0)=
        max_x∈[0,π/2]((β_0^-1+κ_0^-1/2κ_1^-1/2H)cos(x)+sin(x))
        (Λ^-1/2cos(x)+sin(x)).

    


    
    Note that 1-≡ 0 in K^m-1_i
    and 1-≡ 1 in Ω\ K_i,
    implying that supp(1-)⊂Ω\ K_i^m-1.
    Put v(1-) in equation (<ref>), namely
    
    𝒜(,(1-))+ s(π,π(1-))    = 0.

    Then,
    
    _𝒜(Ω\ K^m_i)^2+π_s(Ω\ K^m_i)^2 
       =
            ∫_K^m_i\ K^m-1_i(-1)A∇·∇ x+∫_K^m_i\ K^m-1_iA∇·∇ x
          +∫_K^m_i\ K^m-1_i H^-2κ_1||^2 π()·π((-1))  x +∫_Γ_N\∂ K^m_i(-b+·) ^2 
          + ∫_K^m_i\ K^m-1_i·∇ (-1)
        I_1 + I_2 + I_3 + I_4 + I_5.

    Since -1≤ 0 in K^m_i\ K^m-1_i, then I_1≤ 0.
    Also, since · < b on Γ_N, we have I_4≤ 0.

    For I_2, by using the Cauchy Schwartz inequality,
    
    ∫_K^m_i\ K^m-1_iA∇·∇
       ≤√(∫_K^m_i\ K^m-1_iA∇·∇) √(∫_K^m_i\ K^m-1_i()^2 A∇·∇  x)
       ≤β_0^-1_a(K^m_i\ K^m-1_i)_s(K^m_i\ K^m-1_i).

    The last inequality is due to the assumptions on A and ,
    
    β_0^2∫_K^m_i\ K^m-1_i^2 A∇·∇≤∫_K^m_i\ K^m-1_i^2 ||^2 CH^-2κ_1
        = _s(K^m_i\ K^m-1_i)^2.

    For I_3,
    
    ∫_K^m_i\ K^m-1_iH^-2κ_1||^2 π()·π((-1))  x
       ≤π_s(K^m_i\ K^m-1_i)π((-1))_s(K^m_i\ K^m-1_i)
       ≤π_s(K^m_i\ K^m-1_i)(-1)_s(K^m_i\ K^m-1_i)
       ≤π_s(K^m_i\ K^m-1_i)_s(K^m_i\ K^m-1_i).

    For I_5,
    
    ∫_K^m_i\ K^m-1_i·∇ (-1)
       ≤κ_0^-1/2κ_1^-1/2H√(∫_K^m_i\ K^m-1_iA∇·∇)√(∫_K^m_i\ K^m-1_i||^2 H^-2κ_1 ||^2)
       ≤κ_0^-1/2κ_1^-1/2H _a(K^m_i\ K^m-1_i)_s(K^m_i\ K^m-1_i).

    By inequality (<ref>),
    
    _s(K^m_i\ K^m-1_i)   ≤-π_s(K^m_i\ K^m-1_i)+π_s(K^m_i\ K^m-1_i)
       ≤Λ^-1/2_a(K^m_i\ K^m-1_i)+π_s(K^m_i\ K^m-1_i).

    So, in summary,
    
    _ℬ(Ω\ K^m_i)^2
               ≤((β_0^-1+κ_0^-1/2κ_1^-1/2H)_a(K^m_i\ K^m-1_i)+π_s(K^m_i\ K^m-1_i))_s(K^m_i\ K^m-1_i)
       ≤ c_*(Λ,β_0)^2_ℬ(K^m_i\ K^m-1_i).

    The last inequality comes from considering
    
    cos(θ) = _𝒜(K^m_i\ K^m-1_i)/_ℬ(K^m_i\ K^m-1_i) and sin(θ)=π_s(K^m_i\ K^m-1_i)/_ℬ(K^m_i\ K^m-1_i)

    for some θ∈(0,π/2]. Now,
    
    ^2_ℬ(K^m-1_i)   = ^2_ℬ(K^m_i\ K^m-1_i)+^2_ℬ(Ω\ K^m_i)
       ≥(1+1/c_*)
            _ℬ(Ω\ K^m-1_i)^2.

    Then, iteratively we can obtain
    
    _ℬ(Ω\ K^m_i)^2
        ≤θ^m_ℬ^2.



    With the notations in lemma <ref>,
    then
    
    -_ℬ^2
        ≤C^2 c_⋆θ^m-1_ℬ^2,

    where
    
    c_⋆(Λ,β_0)
        max_x∈[0,π/2][(1+β^-1_0Λ^-1/2)cos(x)
        +β^-1_0sin(x)]^2
        +[Λ^-1/2cos(x)+sin(x)]^2.

    


    Let z_i- and decompose it as 
    
    z_i={(1-)}
        +{(-1)+ z_i} z^'_i+z^''_i.

    By definition, z^''_i∈ V^m_i, so by equations (<ref>) and (<ref>),
    ℬ(z_i,z_i^'')=0.
    Then,
    
    z_i_ℬ^2
            =ℬ(z_i,z_i^')
            ≤Cz_i_ℬz_i^'_ℬ≤C^2 z_i^'_ℬ^2.

    To compute z^'_i_ℬ, we investigate z^'_i_𝒜 and z_i^'_s.
    
    z_i^'_𝒜^2
               =(1-)_𝒜^2 ≲(1-)_a^2
       ≤∫_Ω\ K^m-1_i (1-)^2 A∇·∇ - 2∫_Ω\ K^m-1_i (1-)A∇·∇
          +∫_K^m_i\ K^m-1_i^2 A∇·∇ +∫_Γ_N∩∂ K^m_i (b-·) ^2 σ
       ≤β^-2_0_s(K^m_i\ K^m-1_i)^2
            +2β^-1_0_s(Ω\ K^m-1_i)_a(Ω\ K^m-1_i)
            +_a(Ω\ K^m-1_i)^2
       =(β^-1_0_s(K^m_i\ K^m-1_i)+
            _a(Ω\ K^m-1_i))^2
       =(β^-1_0_s(K^m_i\ K^m-1_i)+
            _𝒜(Ω\ K^m-1_i))^2.

    Again, by inequality (<ref>),    
    
    z_i^'_𝒜   ≤β^-1_0_s(K^m_i\ K^m-1_i)+
            _𝒜(Ω\ K^m-1_i)
       =(β^-1_0Λ^-1/2+1)_𝒜(Ω\ K^m-1_i)+β^-1_0π_s(Ω\ K^m-1_i).

    Also,
    
    π z_i^'_s^2   =
            π((1-))_s ≤(1-)_s
       ≤_s(Ω\ K^m-1_i)≤Λ^-1/2_𝒜(Ω\ K^m-1_i)
            +π_s(Ω\ K^m-1_i).

    Therefore, we can obtain
    
    z_i_𝒜^2+π z_i_s^2
               ≤C^2 z_i^'_ℬ^2
       ≤C^2{[(β^-1_0Λ^-1/2+1)_𝒜(Ω\ K^m-1_i)+β^-1_0π_s(Ω\ K^m-1_i)]^2 .
          +.(Λ^-1/2_𝒜(Ω\ K^m-1_i)
            +π_s(Ω\ K^m-1_i))^2}
       ≤C^2 c_⋆_ℬ(Ω\ K^m-1_i)^2.

    Thus by lemma <ref>,
    
    -_ℬ^2
        ≤C^2 c_⋆θ^m-1_ℬ^2.



    There exists a constant C_ol>0 such that for all K_i∈𝒯^H and m>0,
    
    #{K∈𝒯^H K⊂ K^m_i}≤ C_olm^d.




With the notations in lemmas <ref> and <ref>, then

    ∑^N_i=1-_ℬ^2≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1<t_i,>.




    Let z_i- and z∑^N_i=1z_i.
    Decompose z as
    
    z={(1-χ^m,m+1_i)z}
        +{χ^m,m+1_i z}=:z^'+z^''.

    Notice that supp(z^')⊂Ω\ K^m_i and thus supp(π z^')⊂Ω\ K^m_i. 
    Also, supp()⊂ cl(K^m_i) and thus supp(π)⊂ cl(K^m_i).
    Then, by equations (<ref>) and (<ref>),
    
    ℬ(,z^')=0  and ℬ(,z^')=0,

    granting us 
    ℬ(z_i,z^')=0.

    Now, similar to the proof of lemma <ref>,
    
    ℬ(z_i,z)     = ℬ(z_i,z^'') ≤Cz_i_ℬz_i^''_ℬ
       ≤C{((1+β_0^-1Λ^-1/2)z_𝒜(K^m+1_i)+β_0^-1π z_s(K^m+1_i))^2.
         . + (Λ^-1/2z_𝒜(K^m+1_i) + π z_s(K^m+1_i))^2}^1/2z_i_ℬ
       ≤C c_⋆z_ℬ(K^m+1_i)z_i_ℬ.

    Also, by the definition of C_ol,
    
    ∑^N_i=1z_𝒜(K^m+1_i)^2+π z_s(K^m+1_i)^2
        ≤ C_ol(m+1)^dz_ℬ^2,

    and recall by equation (<ref>),
    
    _𝒜^2+π_s^2=<t_i,>.

    Hence, by the Cauchy–Schwartz inequality and lemma <ref>,
    
    z_ℬ^2   =z_𝒜^2 + π z_s^2 =∑^N_i=1ℬ(z_i,z) ≤∑^N_i=1Cc_⋆z_ℬ(K^m+1_i)z_i_ℬ
       ≤Cc_⋆[∑^N_i=1z_ℬ(K^m+1_i)^2]^1/2[∑^N_i=1z_i_ℬ^2]^1/2
       ≤Cc_⋆[C_ol(m+1)^dz_ℬ(K^m+1_i)^2]^1/2[∑^N_i=1z_i_ℬ^2]^1/2
       ≤Cc_⋆[ C_ol(m+1)^dz_ℬ^2]^1/2[C^2 c_⋆θ^m-1∑^N_i=1<t_i,>]^1/2
       ≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1<t_i,>.




  §.§.§ Error Estimates for Boundary Correctors


We now directly apply these results to estimate 𝒟^glog -𝒟^mg and 𝒩^glo q -𝒩^m q.

    With the notations in lemmas <ref>, <ref> and <ref>, 
    
    𝒟^glog-𝒟^mg_ℬ^2
            ≤C^5 C_ol c_⋆^3 (m+1)^d θ^m-1g^2_𝒜  ;

    
    𝒩^glo q-𝒩^m q_ℬ^2
            ≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1 C^2_trq_L^2(Γ_N)^2,

    where 
    C_trsup_v∈ V,v≠ 0v_L^2(Γ_N)/v_𝒜.

    
    


    With lemma <ref>, it suffices to estimate
    
    ∑^N_i=1𝒜_(K_i)(g,𝒟^glo_ig) and ∑^N_i=1∫_Γ_N∩∂ K_iq 𝒩^glo_i q σ.

    Put v=𝒟^glo_ig in equation (<ref>) and since 𝒟^glo_ig∈ V^glo_ms,
    
    𝒟^glo_ig_ℬ^2 = 𝒜_(K_i)(g,𝒟^glo_ig) ≤Cg_𝒜(K_i)𝒟^glo_ig_ℬ≤C^2 g_𝒜(K_i)^2.

    Now, by lemma <ref>,
    
    𝒟^glog-𝒟^mg_ℬ^2   ≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1𝒜_(K_i)(g,𝒟^glo_ig)
       ≤C^5 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1g^2_𝒜(K_i)
       ≤C^5 C_ol c_⋆^3 (m+1)^d θ^m-1g^2_𝒜.

    Similarly we start from the equation (<ref>),
    
    𝒩^glo_i q_𝒜^2   ≤𝒩^glo_i q_𝒜^2 + π𝒩^glo_i q_s^2 =∫_∂ K_i∩Γ_Nq𝒩^glo_i q σ
       ≤q_L^2(∂ K_i∩Γ_N)𝒩^glo_i q_L^2(∂ K_i∩Γ_N)
       ≤ C_trq_L^2(∂ K_i∩Γ_N)𝒩^glo_i q_𝒜,

    which yields 𝒩^glo_i q_𝒜≤ C_trq_L^2(∂ K_i∩Γ_N).
    Then,
    
    ∑^N_i=1∫_∂ K_i∩Γ_Nq𝒩^glo_i q σ≤∑^N_i=1q_L^2(∂ K_i∩Γ_N)𝒩^glo_i q_L^2(Γ_N)
       ≤∑^N_i=1C_trq_L^2(∂ K_i∩Γ_N)𝒩^glo_i q_𝒜≤∑^N_i=1C_tr^2q_L^2(∂ K_i∩Γ_N)^2
       = C_tr^2q_L^2(Γ_N)^2.



To further analyse function spaces V^glo_ms and V^m_ms, 
we define operators 
ℛ^glo∑^N_i=1ℛ^glo_i
L^2(Ω)→ V^glo_ms and 
= ∑^N_i=1 L^2(Ω)→ V^m_ms
where

    𝒜(φ,v)+s(πφ,π v)= s(π_iφ,π v), for  v∈ V;


    𝒜(φ,v)+s(πφ,π v)=s(π_iφ,π v), for  v∈ V^m_i.


We also remark another lemma <cit.>.

    There exists a positive constant C_inv such that
    for any v∈ L^2(Ω), 
    there exists v∈ V with πv=π v 
    such that v_𝒜≤ C_invπ v_s.
    



  §.§.§ Error Estimate for the Multiscale Solution



We now state the main result.

    Let 𝒟^mg, 𝒩^m q and w^m be the numerical solutions as defined before, 
    w^glo defined in equation (<ref>),
    and Λ, θ, c_⋆, c_#, C_tr and C_inv be the constants defined in theorem <ref>, lemma <ref>, lemma <ref>,
    lemma <ref>, corollary <ref> and lemma <ref> respectively.
    Then
    
    w^m-𝒟^mg+𝒩^m q +g-u_𝒜
       ≤C{Λ^-1/2κ_1^-1/2H(||^-1f_L^2(Ω)+∇ u_0_L^2(Ω))}
       +C^2 √(C_ol) c_⋆^3/2 (m+1)^d/2θ^(m-1)/2{C^2max(C^2_inv,1)w^glo_ℬ..+Cg_𝒜
            +C_trq_L^2(Γ_N)}.

    
    Moreover, if we have C_invθ^(m-1)/2(m+1)^d/2=O(H^2),
    then
    w^m-𝒟^mg+𝒩^m q+g-u_𝒜 =O(H).



    By definition of ,  is surjective.
    Then there exists φ_*∈ L^2(Ω) such that w^glo=φ_*.
    Let w^m_*=φ_*.
    To estimate φ_* - φ_*, we use the similar argument in corollary <ref>.
    
    φ_* - φ_*_ℬ^2 ≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1
            s(π_iφ_*,πφ_*)
       ≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1π_iφ_*_sπφ_*_s ≤C^4 C_ol c_⋆^3 (m+1)^d θ^m-1∑^N_i=1π_iφ_*_s^2
       =C^4 C_ol c_⋆^3 (m+1)^d θ^m-1πφ_*^2_s.

    Then, by lemma <ref>,
    there exists φ_*∈ V such that 
    πφ_*=πφ_*
         and φ_*_𝒜≤ C_invπφ_*_s.

    Putting v=φ_* in equation (<ref>),
    
    πφ_*_s^2=ℬ(w^glo,φ_*)
            ≤Cw^glo_ℬφ_ℬ≤Cmax(C_inv,1)w^glo_ℬπφ_*_s.
    
    So,
    
    φ_*-φ_*_ℬ^2≤C^6 C_ol c_⋆^3 (m+1)^d θ^m-1max(C_inv^2,1)w^glo_ℬ^2.

    Now, let u^ms_0 w^m-𝒟^mg+𝒩^m q and u^glo_0 w^glo-𝒟^glog+𝒩^glo q. Then,
    
    u-u^ms_𝒜^2   = u_0-u^ms_0_𝒜^2 = 𝒜(u_0-u^ms_0,u_0-v)
       ≤Cu_0-u^ms_0_𝒜u_0-v_ℬ.

    Putting v=w^m_*-𝒟^mg+𝒩^m q,
    
    u_0-v_ℬ≤u_0-u_0_ℬ
            +u_0-v_ℬ
       ≤u_0-u_0_𝒜
            + w^glo-w^m_*_ℬ
            + 𝒟^glog-𝒟^mg_ℬ
            + 𝒩^glo q-𝒩^m q_ℬ.

    Altogether,
    
    u-u^ms_𝒜≤Cu_0-v_ℬ
       ≤C{u_0-u_0_𝒜
            + w^glo-w^m_*_ℬ
            + 𝒟^glog-𝒟^mg_ℬ
            + 𝒩^glo q-𝒩^m q_ℬ}
       ≤C{Λ^-1/2κ_1^-1/2H(||^-1f_L^2(Ω)+∇ u_0_L^2(Ω))}
          +C^2 √(C_ol) c_⋆^3/2 (m+1)^d/2θ^m-1/2{max(C_inv,1)w^glo_ℬ.
            .+Cg_𝒜
            +C_trq_L^2(Γ_N)} .

    Now, first recall that we have u_0-u_0_𝒜=O(1). Note that C=O(1).
    By the equations (<ref>) and (<ref>), we have 𝒟^glog-𝒟^mg_𝒜 = O(1) and 𝒩^glo q-𝒩^m q_𝒜 = O(1).
    Assume (m+1)^d/2θ^m-1/2C_inv = O(H^2). It suffices to estimate w^glo_𝒜 and π w^glo_s.
    By equation (<ref>),
    
    w^glo_𝒜^2=𝒜(u_0,w^glo)+𝒜(𝒟^glog,w^glo)-𝒜(𝒩^glo q,w^glo)

    giving us
    
    w^glo_𝒜≤u_0_𝒜+ 𝒟^glog_𝒜 + 𝒩^glo q_𝒜 =O(1).

    On the other hand,
    
    π w^glo_s ≤π u_0_s + π𝒟^glog_s + π𝒩^glo q_s ≤ O(H^-1)+ O(1).

    Therefore, we can obtain 
    u^ms-u_𝒜 = O(H).




 §.§ Numerical Experiments



In this section, we will demonstrate the method via several numerical examples in a high-contrast setting and verify the significance of the inflow condition. 
For simplicity, we take point-wise isotropic coefficients, A(x)=κ(x)I, the domain Ω = [0,1]×[0,1]. We will calculate the reference solutions on a 200×200 mesh with the bilinear Lagrange finite element method. The medium κ is presented in Figure <ref> and the source term in Figure <ref>. The experiments are each tested for coarse mesh H = 1/10, 1/20 and 1/40 with the fixed number l_m of eigenfunctions to generate the auxiliary space V^aux. By our experiments, we tested l_m=3 to be sufficient to verify our results. 
 



  §.§.§ Example 1

We first look at the Dirichlet condition, via considering the following problem:

    -∇·(κ(x_1,x_2) ∇ u)+(x_1,x_2)·∇ u=f     for  (x_1,x_2)∈Ω
    
            u(x_1,x_2)=g(x_1,x_2)=x_1^2 + e^x_1 x_2    for (x_1,x_2)∈{0,1}×[0,1]

where

    (x_1,x_2)=[cos(18π x_2)sin(18π x_1),
    -cos(18π x_1)sin(18π x_2)]^T.








To simplify notations, we denote the relative errors for the Dirichlet corrector

    D^m_a 𝒟^mg-𝒟^glog_𝒜/𝒟^glog_𝒜 and 
    D^m_L 𝒟^mg-𝒟^glog_L^2(Ω)/𝒟^glog_L^2;


For the Neumann condition corrector,

    N^m_a 𝒩^m q-𝒩^glo q_𝒜/𝒩^glo q_𝒜 and 
    N^m_L 𝒩^m q-𝒩^glo q_L^2(Ω)/𝒩^glo q_L^2;

to measure errors and Λ^' = max_iλ^l_m_i.
As for the error estimate,

    E^m_a u^ms-u_𝒜/u_𝒜 and 
    E^m_L u^ms-u_L^2/u_L^2.




As can be seen in Table <ref>, the error of the Dirichlet corrector decays exponentially as we increase the number of oversampling layers. From Table <ref>, the number of eigenfunctions does improve the results for both the overall multiscale solutions and the boundary correctors. Most importantly, from Table <ref>, we can see a second-order convergence when increasing the oversampling layers and decreasing the coarse mesh. This also echoes the idea that the convergence of u^ms depends on the oversampling layers N_ov. 





  §.§.§ Example 2


Another demonstration comes from the Neumann and Robin conditions. In this problem, we first consider the same velocity field :

    -∇·(κ(x_1,x_2) ∇ u)+(x_1,x_2)·∇ u=f     for  (x_1,x_2)∈Ω
    
            u(x_1,x_2)=g(x_1,x_2)=x_1^2 + e^x_1 x_2    for (x_1,x_2)∈{0,1}×[0,1]
    
            bu + ·(A∇ u - u) = -1     for  x_1 = 0  and  x_2 ∈ [0,1]
    
            bu + ·(A∇ u - u) = 1     for  x_1 = 1  and  x_2 ∈ [0,1]
    
            bu + ·(A∇ u - u) = 1     for  x_1 = (0.5,1]  and  x_2 =0
    
            bu + ·(A∇ u - u) = 0     for  x_1 = [0, 0.5]  and  x_2 = 0

where b(x_1,x_2)= κ(x_1,x_2) is the Robin coefficient.

For the following results, the numbers in brackets show the relative error for varying numbers of oversampling layers.
 




From Table <ref>, we also can observe the second-order convergence with respect to H. Moreover, we compare the effect of the inflow conditions using the following two examples. 



  §.§.§ Example 3


In this model, we consider the inflow condition on Γ_N.

    _in(x_1,x_2) =  + c_flow[1/2-x_1,x_1]^T.

Notice that _in·≤ 0 on Γ_N as shown in Figure <ref>. The constant c_flow is proportional to the magnitude of the velocity on the boundary Γ_N,





As can be seen in Tables <ref> and <ref>, similar observations of the results are shown, resembling our theoretical analysis and the numerical results in the examples. 



  §.§.§ Example 4


We consider the same setting as Example 3, but with the following velocity field 
_out = -_in
with c_flow>0.
Note that ∇·_out=0 on Ω but _out· > 0 on x_1=0 and x_1=1, shown in Figure <ref>.









In Table <ref>, second-order convergence in H is still observed when given a large enough number of oversampling layers and a fixed c_flow=3. However, as we increase the velocity field c_flow=4, we see that the problem is more demanding. Without a larger number of oversampling layers, reducing the coarse mesh H alone may not be enough to improve the approximation. However, when given a sufficient number of layers, the job can still be done. The comparison is more apparent in Figure <ref> and Figure <ref>, where the outflow case converges much slower in both ·_L^2 and ·_𝒜 compared to the inflow case.






§ TIME DEPENDENT CONVECTION-DIFFUSION INITIAL BOUNDARY VALUE PROBLEMS




 §.§ Derivation of the Method

We present the convection diffusion initial boundary value problem as follows:

    ∂_t u(x,t) -∇·(A(x) ∇ u(x,t))+ (x) ·∇ u(x,t) = f(x,t)     on Ω× (0,T]
    
        u(x,t) = g(x,t)     on Γ_D× (0,T]
    
        b(x) u(x,t) + · (A(x) ∇ u(x.t)- (x) u(x,t))= q(x,t)     on Γ_N× (0,T]
    
        u(x,0) = u_init(x)     on Ω.

Notice f is still independent of the solution u.
The variational formulation becomes: 
find u_0(·,t)∈ V such that for v∈ V,

    (∂_t u_0,v) + 𝒜(u_0,v)    = (f,v) + (q,v)_Γ_N - 𝒜(g,v) - (g_t,v)
    
        (u_0(·, 0),v)    = (u_init(x)-g(·,0),v)

and the actual solution would be u(·,t) = u_0(·,t)+g(·,t).
We will use the same auxiliary spaces V^aux and multiscale space V^m_ms as in the previous section, and thereby the same set of correctors 𝒟^m g and 𝒩^m q. 
In particular, for t∈ (0,T],

    
  * Find 𝒟^m_i g(·, t)∈ V^m_i such that v∈ V^m_i,
    
    (𝒟^m_i g_t, v) + ℬ(𝒟^m_i g,v) = (g_t, v)_(K_i) + 𝒜_(K_i)(g,v)
    ℬ(𝒟^m_i g(·, 0),v) = 𝒜_(K_i)(g(·,0),v).

    Denote 𝒟^m g = ∑_i=1^N𝒟^m_i g.
    
  * Find 𝒩^m_i q(·, t) ∈ V^m_i such that v∈ V^m_i,
    
    (𝒩^m_i q_t, v) + ℬ(𝒩^m_i q,v)= ∫_Γ_N∩∂ K_iqv σ
    ℬ(𝒩^m_i q(·, 0),v) = ∫_Γ_N∩∂ K_i q(·, 0)v σ.

    Denote 𝒩^m q = ∑^N_i=1𝒩^m_i q.
    

  * find w^m(·, t) ∈ V^m_ms such that for v∈V^m_ms

    (w^m_t,v) + 𝒜(w^m,v)    = (f,v) + (q,v)_Γ_N -𝒜(g,v)-(g_t,v) 
        + (𝒟^m g_t, v) + 𝒜(𝒟^m g, v) -(𝒩^m q_t,v) -𝒜(𝒩^m q, v) ,



    (w^m(·, 0), v) = (u_init-g(·,0)+𝒟^m g(·,0)-𝒩^m q(·,0),v) .
 

Then the multiscale approximation becomes 

    u^ms = u^ms_0 + g = w^m- 𝒟^m g + 𝒩^m q+g.



 §.§ Analysis


To test the performance of the oversampling layers, we define 𝒟^glog=∑^N_i=1𝒟^glo_ig and 𝒩^glo q=∑^N_i=1𝒩^glo_i q where 𝒟^glo_ig,𝒩^glo_i q∈ V^glo_ms satisfies that for all v∈ V,

    (𝒟^glo_i g_t, v) + ℬ(𝒟^glo_i g,v) = (g_t, v)_(K_i) + 𝒜_(K_i)(g,v)
    ℬ(𝒟^glo_i g(·, 0),v) = 𝒜_(K_i)(g(·,0),v),
,
    and 
    
    (𝒩^glo_i q_t, v) + ℬ(𝒩^glo_i q,v)= ∫_Γ_N∩∂ K_iqv σ
    ℬ(𝒩^glo_i q(·, 0),v) = ∫_Γ_N∩∂ K_i q(·, 0)v σ.



We will give an overview of the analysis.
Define for v∈ V,
v_ℰ^2 = v(·,T)_L^2^2 + ∫^T_0 v_ℬ^2.
Note that since ·_𝒜 is a quasi-norm, so are ·_ℬ and ·_ℰ. 
Suppose u(·,t) is the elliptic projection of the solution u, i.e.

    𝒜(u-u,v) =0  for  v∈ V^glo_ms.

Akin to the previous treatment, our strategy is to decompose u-v into two parts:

    u-u^ms = (u-u)+ (u-u^ms).

The error of the former term is computable while that of the latter term can be bounded by a specific choice v∈ V^m_ms.
In particular, for any v∈ V^m_ms,

    ∫^T_0[((u-u^ms)_t,u-u^ms) + 𝒜(u-u^ms,u-u^ms)]
       =∫^T_0[((u-u^ms)_t,u-v) + 𝒜(u-u^ms,u-v)]
       ≤.(u-u^ms,u-v)|^T_0
         -∫^T_0 ((u-v)_t,u-u^ms) + ∫^T_0 𝒜(u-u^ms,u-v)
       ≤(u-u^ms)(·,T)_L^2(u-v)(·,T)_L^2
         +(u-u^ms)(·,0)_L^2(u-v)(·,0)_L^2
       +√(∫^T_0(u-v)_t_L^2^2)√(∫^T_0u-u^ms_L^2^2)
         +3/4∫^T_0u-u^ms_𝒜^2
         +1/3C^2∫^T_0u-v_ℬ^2.


Hence, by the repeated use of Young's inequality and the Cauchy Schwartz inequality, 

    (u-u^ms)(·,T)_L^2^2+∫^T_0u-u^ms_𝒜^2
           ≤ 4(u-u^ms)(·,0)_L^2^2 
         + 2(u-v)(·,0)_L^2^2+ 4C^2u-v_ℰ^2
       +4√(∫^T_0(u-v)_t_L^2^2)√(∫^T_0u-u^ms_L^2^2)
       =: (i) + (ii) + (iii) + (iv).


Note that (i) has been investigated in the time-independent case. 
Now, define w^glo∈ V^glo_ms such that for any v∈ V,

    𝒜(w^glo,v) = 𝒜(u - 𝒟^glog + 𝒩^glo q,v).


Then, with the same set of ∑^N_i=1 and ∑^N_i=1 operators in the last section, there is φ(·,t) ∈ L^2(Ω) such that

    (φ_t,v)+ℬ(φ,v)=s(π_i φ, π v)    for v∈ V,
    
        (φ_t,v)+ℬ(φ,v)= s(π_i φ,π v)    for v∈ V^m_ms.

with initial conditions ℬ(φ(·,0),v)=s(π_iφ(·,0),π v) for v∈ V and ℬ(φ(·,0),v)=s(π_iφ(·,0),π v) for v∈ V^m_ms. 
Then, by the surjectivity of , we can find φ_*(·,t)∈ L^2(Ω) such that 

    φ_* = w^glo and define  w^m_*φ_*.


By putting v=w^m_* - 𝒟^m g + 𝒩^m q, we can further decompose 

    u - v = (u-u) + (w^glo-w^m_*)+ (𝒟^glog-𝒟^m g) + (𝒩^glo q-𝒩^m q).


The error analysis of these terms will suffice that of the remaining terms (ii), (iii), and (iv) in equation (<ref>). 
u-u is first dealt with
and the rest in another abstract problem.


  §.§.§ Elliptic Projection



We first give an error bound for the elliptic projection.


Let u_0∈ V^glo_ms be the elliptic projection of u_0 onto V^glo_ms, i.e.

    𝒜(u_0,v) = 𝒜(u_0,v)  for v∈ V^glo_ms.

Let u = u_0 + g. Then,
    
    u-u_𝒜≤Λ^-1/2κ^-1/2(f-u_t)_L^2.

    


    By the definition of u, 
    since 𝒜(u-u, v) = 0 for v∈ V^glo_ms,
    π (u-u) = 0.
    Then 𝒜(u_0, u-u) = 0.
    This grants us
    
    u-u_𝒜^2 
               = u_0-u_0_𝒜^2
       = 𝒜(u_0, u - u)
       = (f,u-u) + ((u_0)_t,u-u)
       = (f-u_t, u-u)
       ≤Λ^-1/2κ^-1/2(f-u_t)u-u_𝒜.



We also can obtain the L^2-norm of the global estimate u.
 With the notations in lemma <ref>, we have
    
    u-u_L^2≤C Hc_#Λ^-1/2κ^-1/2(f-u_t)_L^2.



    Let z∈ V and z∈ V^glo_ms such that 
    𝒜(z,v) = (u-u,v) 
            for v∈ V 
    𝒜(z,v) = (u-u,v)     for  v ∈ V^glo_ms.

    Note that 𝒜(z-z,v) = 0 for v ∈ V^glo_ms. This grants us 𝒜(z,z-z)=0 and 𝒜(z, u-u)=0.
    Then, 
    
    z-z_𝒜^2    = 𝒜(z-z, z-z)
       = 𝒜(z,z-z)
       = (u-u, z-z)
       ≤u-u_L^2z-z_L^2
       ≤ Hc_#u-u_L^2z-z_𝒜.

    So, 
    
    z-z_𝒜≤ Hc_#u-u_L^2.

    
    Now, 
    
    u-u_L^2^2 = 𝒜(z, u-u)
            = 𝒜(z-z,u-u)
            ≤Cz-z_𝒜u-u_𝒜.

    Combining the results, the lemma is proved.


    Following the similar lines of arguments, we can obtain
    
    (u-u)_t_L^2≤C Hc_#Λ^-1/2κ^-1/2(f_t-u_tt)_L^2.

    Also note that κ^-1/2=O(H). So the L^2-error is also second order with respect to H.




  §.§.§ Abstract Problem



We now move on to the analysis of the corrector. The main idea is that the error propagation has an exponential decay with respect to the oversampling layers, similar to the time-independent case.

    Let K_i∈𝒯^H and t_i(·,t), t_i^0∈ V^' such that <t_i,v>=0 and <t_i^0,v>=0 for any v∈ V with supp(v)⊂Ω\ K_i.
    Define 𝒫_i(·,t):V^'→ V such that for all v(·,t)∈ V,
    
    (_t,v)+ℬ(,v)=<t_i,v>
    ℬ((·,0),v)=<t_i^0,v>,

    and 𝒫^m_i:V^'→ V^m_i with
    
    (_t,v)+ℬ(,v)=<t_i,v>
    ℬ((·,0),v)=<t_i^0,v>.



We aim to estimate 
    
    ∑^N_i=1-_ℰ^2 =∑^N_i=1(-)(·,T)_L^2^2+ ∫^T_0∑^N_i=1-_ℬ^2  t.


One should note that the initial condition here is exactly the abstract problem in the previous section. The results are carried over here.
In addition, we will further define two norms for our analysis,

    t_i = max_v∈ V<t_i,v>/v_ℬ and t^0_i = max_v∈ V<t^0_i,v>/v_ℬ.



Let c_#β_0^-1κ_1^-1/2√(1+Λ^-1). Then
    
    (·,0)_L^2^2≤ H^2 c_#^2t^0_i^2,

    and
    
    _ℰ^2≤ H^2 c_#^2 t^0_i^2 + ∫^T_0 t_i^2.



    

    (·,0)_L^2^2   ≤ H^2c_#^2 (·,0)_ℬ^2 
       = H^2c_#^2<t^0_i, (·,0)>
       ≤ H^2c_#^2 t^0_i(·,0)_ℬ
       ≤ H^2 c_#^2 t^0_i^2.

Hence, 

    (·,0)_L^2≤ Hc_#t^0_i.

Also, putting v =,

    (_t,)+ℬ(,)   =<t_i,>
    ∫^T_01/2∂/∂ t_t_L^2^2
        +_ℬ^2  t    = ∫^T_0 <t_i,>
    1/2(·,T)_L^2^2 +∫^T_0 _ℬ^2   ≤1/2(·,0)^2 + 1/2∫^T_0 t_i^2 + 1/2∫^T_0 _ℬ^2.

Hence, by using equation (<ref>),

    _ℰ^2≤  H^2 c_#^2 t^0_i^2 + ∫^T_0 t_i^2.



With the same notations in lemma <ref>,

    _ℰ(Ω\ K^m_i)^2≤θ^m ((m+1)H^2c_#^2t^0_i^2+∫^T_0 t_i^2)
            .

    



Following the similar approach in the proof of lemma <ref>, putting v=(1-) into the equation (<ref>) and then integrating over t ∈ [0,T], we can obtain

    ∫^T_0(_t, )_(Ω\ K^m_i) + _ℬ(Ω\ K^m_i)^2
       = ∫^T_0(_t, (-1))_(K^m_i \ K^m-1_i)
        +∫^T_0 ℬ_(K^m_i \ K^m-1_i)(, (-1))
       ≤∫_K^m_i \ K^m-1_i (-1) ∫^T_0 1/2∂/∂ t_L^2(K^m_i \ K^m-1_i)^2 + 
        ∫^T_0 c_* _ℬ(K^m_i \ K^m-1_i)^2
       ≤1/2(·,0)_L^2(K^m_i \ K^m-1_i)^2+ 
        ∫^T_0 c_* _ℬ(K^m_i \ K^m-1_i)^2.

Then,

    _ℰ(Ω\ K^m-1_i)^2-1/2(·,0)_L^2(Ω\ K^m-1_i)^2
       ≥(1 + 1/c_*)(^2_ℰ(Ω\ K^m_i)-1/2(·,0)_L^2(Ω\ K^m_i)^2)
       -1/2c_*(·,0)_L^2(K^m_i \ K^m-1_i)^2
       ≥1/θ(^2_ℰ(Ω\ K^m_i)-1/2(·,0)_L^2(Ω\ K^m_i)^2)-1/2c_*(·,0)_L^2(K^m_i \ K^m-1_i)^2.

Along with lemma <ref>, this grants us 
    
    _ℰ(Ω\ K^m_i)^2
               ≤1/2(·,0)_L^2(Ω\ K^m_i)^2 + θ/2c_*(·,0)_L^2(K^m_i \ K^m-1_i)^2
            +θ_ℰ(Ω\ K^m-1_i)^2
       ≤1/2θ^m(·,0)_L^2^2 + θ/2c_*(·,0)_L^2(Ω\ K^m-1_i)^2
            +θ_ℰ(Ω\ K^m-1_i)^2
       ≤1/2θ^m(1+1/c_*)(·,0)_L^2^2 +θ_ℰ(Ω\ K^m-1_i)^2
       ≤1/2θ^m-1(·,0)_L^2^2 +θ_ℰ(Ω\ K^m-1_i)^2
       ≤θ^m(m (·,0)_L^2^2 + _ℰ^2)
       ≤θ^m(m H^2 c_#^2(·,0)_ℬ^2 + _ℰ^2)
       ≤θ^m(m H^2 c_#^2t_i^0^2 + _ℰ^2).

Note that combined with equation (<ref>), we obtain
    
    ^2_ℰ(Ω\ K^m_i)≤θ^m ((m+1)H^2c_#^2t^0_i^2+∫^T_0 t_i^2).



In parallel, we can have

    ^2_ℰ^2(Ω\ K^m_i)≤θ^m ((m+1)H^2c_#^2t^0_i^2+∫^T_0 t_i^2).



    With the same notations in lemma <ref>, we have
    
    -_ℰ^2≤ 2θ^m-1(1+2C^2 c_⋆) [H^2 c_#^2 (m+2)t_i^0^2+∫^T_0t_i^2].

    


    Recall that z_i = - and
    
    z_i = [(1-)]+[(-1)+ z_i] =: z_i^' + z_i^''.

    With equations (<ref>) and (<ref>),
    
    z_i(·,T)_ℰ(Ω\ K^m-1_i)^2≤θ^m-1(mH^2c_#^2t^0_i^2+∫^T_0 t_i^2).
    
    and by lemmas <ref> and <ref>,
    
    z_i(·,0)^2_L^2(Ω\ K^m-1_i)   ≤C^2 c_⋆ H^2c_#^2θ^m-1(·,0)_ℬ^2
       ≤C^2 c_⋆ H^2c_#^2θ^m-1t_i^0^2,

    and by lemma <ref>,
    
    (·,0)_L^2(Ω\ K^m-1)^2
            ≤C^2 c_⋆ H^2c_#^2θ^m-1t_i^0^2.

    Then,
    
    z_i^2_ℰ-1/2z_i(·,0)^2
       ≤∫^T_0((z_i)_t,z_i^')+∫^T_0ℬ(z_i,z_i^')
       ≤.(z_i,z_i^')|^T_0 - ∫^T_0 ((z^'_i)_t,z_i) + C∫^T_0 z_i_ℬ(Ω\ K^m-1_i)z_i^'_ℬ(Ω\ K^m-1_i)
       ≤z_i(·,T)_L^2(Ω\ K^m-1_i)z^'_i(·,T)_L^2(Ω\ K^m-1_i)
            + z_i(·,0)_L^2(Ω\ K^m-1_i)z^'_i(·,0)_L^2(Ω\ K^m-1_i)
            +∫^T_0 ℬ_(Ω\ K^m-1_i)(,(1-)z_i) + C∫^T_0z_i_ℬ(Ω\ K^m-1_i)z_i^'_ℬ(Ω\ K^m-1_i)
       ≤θ^m-1(mH^2c_#^2t^0_i^2+∫^T_0t_i^2)+(θ^m-1H^2c_#^2t^0_i^2)
            + C_ℰ(Ω\ K^m-1_i)(∫^T_0(1-)z_i^2_ℬ(Ω\ K^m-1_i))^1/2
            +C√(c_⋆)z_i_ℰ(Ω\ K^m-1_i)(∫^T_0_ℬ(Ω\ K^m-1_i)^2)^1/2
       ≤θ^m-1((m+1)H^2 c_#^2 t^0_i^2+∫^T_0 t_i^2)
             + 2C√(c_⋆)_ℰ(Ω\ K^m-1_i)z_i_ℰ(Ω\ K^m-1_i)
       ≤θ^m-1((m+1)H^2c_#^2t^0_i^2+∫^T_0t_i^2)+2C^2c_⋆_ℰ(Ω\ K^m-1_i)^2 + 1/2z_i_ℰ^2.

    Hence,
    
    z_i_ℰ^2≤ 2θ^m-1(1+2C^2 c_⋆) [H^2 c_#^2 (m+2)t_i^0^2+∫^T_0t_i^2].




    With the same notations in lemma <ref>, we have
    
    ∑^N_i=1-_ℰ^2
            ≤θ^m-1C(2+C^2 c_⋆) [H^2 c_#^2 (m+2)∑^N_i=1t_i^0^2+∫^T_0∑^N_i=1t_i^2].

    


    With the same notation, z = ∑^N_i z_i.
    First, with lemma <ref>,
    
    ∑^N_i=1z_i(·,T)_L^2^2≤(∑^N_i=1z_i(·,T)_L^2)^2
       ≤(∑^N_i=1θ^m-1/2(2+C^2 c_⋆)^1/2[H^2 c_#^2 (m+2)t_i^0^2+∫^T_0t_i^2]^1/2)^2
       ≤θ^m-1(2+C^2 c_⋆)(H^2 c_#^2 (m+2)∑^N_i=1t_i^0^2+∫^T_0∑^N_i=1t_i^2).

    Next, by the Cauchy Schwartz inequality, 
    
    ∫^T_0 z_ℬ^2 =∫^T_0 ∑_i,jℬ(z_i,z_j)≤C∫^T_0∑_i,jz_i_ℬz_j_ℬ≤C∑^N_i=1∫^T_0z_i_ℬ^2
       ≤θ^m-1C∑^N_i=1(2+C^2 c_⋆) [H^2 c_#^2 (m+2)t_i^0^2+∫^T_0t_i^2]
       ≤θ^m-1C(2+C^2 c_⋆) [H^2 c_#^2 (m+2)∑^N_i=1t_i^0^2+∫^T_0∑^N_i=1t_i^2].

    Now, together with all the terms, the proof is complete.



Suppose 𝒟^m g, 𝒩^m q, w^m, 𝒩^glo q, 𝒟^glog and w^glo are constructed by equations (<ref>), (<ref>), (<ref>), (<ref>), (<ref>), and (<ref>) respectively. Suppose C, θ, c_#, c_⋆ and C_inv are defined as in lemmas <ref>, <ref>, <ref> and <ref> respectively. Then, 
    
    𝒟^glog-𝒟^m g^2_ℰ≤θ^m-1C^2(2+C^2 c_⋆) [H^2 c_#^2 (m+2)g(·,0)_L^2^2+∫^T_0(g_𝒜^2+g_t_L^2^2)];

    
    𝒩^gloq-𝒩^m q^2_ℰ≤θ^m-1C_tr^2(2+C^2 c_⋆) [H^2 c_#^2 (m+2)q(·,0)_L^2(Γ_N)^2+∫^T_0q_L^2(Γ_N)^2];

    and
    
    w^glo-w^m_*^2_ℰ
       ≤θ^m-1C(2+C^2c_⋆)max(C_inv^2,1){H^2c_#^2[(m+2)w^glo(·,0)_ℬ^2+∫^T_0w^glo_t_L^2^2]+C^2∫^T_0w^glo_ℬ^2}.



    For the Dirichlet condition,
    
    <t_i,𝒟^glog>   =𝒜_(K_i)(g,𝒟^glog)+ (g_t,𝒟^glog)_(K_i)
       ≤(Cg_𝒜(K_i)+Hc_#g_t_L^2(K_i))𝒟^glog_ℬ.

    Hence, 
    ∑^N_i=1t_i^2   ≤∑^N_i=1(Cg_𝒜(K_i)+Hc_#g_t_L^2(K_i))^2
       =(C√(∑^N_i=1g^2_𝒜(K_i))+Hc_#√(∑^N_i=1g_t_L^2(K_i)^2))^2
       =(Cg_𝒜+Hc_#g_t_L^2)^2.

    Similarly, 
    
    ∑^N_i=1t^0_i^2≤∑^N_i=1C^2g(·,0)_L^2^2.

    Now for the Neumman corrector,
    
    ∑^N_i=1∫_Γ_N∩∂ K_iq𝒩^glo_i q σ   ≤∑^N_i=1q_L^2(Γ_N∩∂ K_i)𝒩^glo_i q_L^2(Γ_N)
       ≤∑^N_i=1 C_trq_L^2(Γ_N∩∂ K_i)𝒩^glo_i q_𝒜
       ≤ C_tr^2 q_L^2(Γ_N)^2.

    Finally, we deal with w^glo-w^m_*. Note that s(π_iφ_*,π v)≤π_iφ_*_sv_ℬ at any t>0 for v∈ V. To bound t_i, it suffices to bound π_iφ_*(·,t)_s.
    Akin to the time-independent case, for each t>0, there exists φ_*(·,t)∈ V such that πφ_*=πφ_* and φ_*_𝒜≤ C_invπφ_*_s.
    Then,
    
    πφ_s^2    = (w^glo_t,φ_*)+ℬ(w^glo,φ_*)
       ≤w^glo_t_L^2φ_*_L^2+Cw^glo_ℬφ_*_ℬ
       ≤(Hc_#w^glo_t_L^2+Cw^glo_ℬ)φ_*_ℬ
       ≤max(C_inv,1)(Hc_#w^glo_t_L^2+Cw^glo_ℬ)πφ_*_s.

    Hence, by assembling all the terms, one can obtain the desired result. 



If g_tt, q_t and u_tt exist in L^2(Ω, (0,T)), then

    (𝒟^glog-𝒟^m g)_t_ℰ^2
            ≤θ^m-1C^2(2+C^2 c_⋆) [H^2 c_#^2 (m+2)g_t(·,0)_L^2^2+∫^T_0(g_t_𝒜^2+g_tt_L^2^2)];
 

    (𝒩^glo q-𝒩^m q)_t_ℰ^2 
            ≤θ^m-1C_tr^2(2+C^2 c_⋆) [H^2 c_#^2 (m+2)q_t(·,0)_L^2(Γ_N)^2+∫^T_0q_t_L^2(Γ_N)^2];

and

    (w^glo-w^m_*)_t^2_ℰ
       ≤θ^m-1C(2+C^2c_⋆)max(C_inv^2,1){H^2c_#^2[(m+2)w^glo_t(·,0)_ℬ^2+∫^T_0w^glo_tt_L^2^2]+C^2∫^T_0w^glo_t_ℬ^2}.



    Now, by taking derivatives with respect to time, we can have another t_i^'(·,t), (t^0_i)^'∈ V^' such that 
    
    (_tt,v)+ℬ(_t,v)=<t_i^',v>
    ℬ(_t(·,0),v)=<(t^0_i)^',v>.

    Then, following the same lines of arguments of the abstract problems obtains the desired results. The existence of the time derivative of the corrector follows from the regularity of the boundary value functions.


    If furthermore C_invθ^(m-1)/2(m+2)^d/2=O(H^2), then
    
    𝒟^glog-𝒟^m g_ℰ   ≤ O(H^2+√(T)H),
    𝒩^glo q-𝒩^m q_ℰ   ≤ O(H^2+√(T)H),
    w^glo-w^m_*_ℰ   ≤ O(H^2+√(T)H).


The cases for 𝒟^mg and 𝒩^m q are clear. For w^glo,
    by utilizing equations (<ref>) and (<ref>), with the analysis in the time-independent case, it is easy to see ∫^T_0w^glo_ℬ^2 ≲∫^T_0 u_0_ℬ^2+∫^T_0 𝒟^glog_ℬ^2+∫^T_0 𝒩^gloq_ℬ^2 =O(T(H^-2+1)). 
    On the other hand, 
    
    ∫^T_0 𝒟^glog_t_L^2^2
               ≲ H^2 ∫^T_0 𝒟^glog_t_ℬ^2
       ≲ H^2{𝒟^glog_t(·,0)_L^2^2 + ∫^T_0g_t_𝒜^2+H^2∫^T_0g_tt_L^2^2}
       =O(H^2+TH^2 + TH^4).

    Similarly, ∫^T_0 𝒩^gloq_t_L^2^2=O(H^2+TH^2). Then, 
    
    ∫^T_0 w^glo_t_L^2^2≲∫^T_0 ∂_t u_0_L^2^2+∫^T_0 𝒟^glog_t_L^2^2+∫^T_0 𝒩^gloq_t_L^2^2=O(T+H^2+TH^2).
 
    Also, by the arguments in the time-independent case, w^glo(·,0)_ℬ=O(H^-1). Combining all the terms, we have w^glo-w^m_*_ℰ=O(H^2+√(T)H).



  §.§.§ Multiscale Solutions


We will first bound ∫^T_0u-u^ms_L^2^2.
 Suppose u^ms is constructed by equation (<ref>) and u is the actual solution of equation (<ref>). Then

    ∫^T_0 u-u^ms_L^2^2
           ≤4T(u-u^ms)(·,0)_L^2^2
       +2H^2c_#^2(1+T^2)
        ∫^T_0{κ^-1/2(f-u_t)^2_L^2
        +κ^-1/2(f_t-u_tt)^2_L^2}.



    Let u-u^ms = (u-u)+(u-u^ms)=: ϑ+ρ.
    Note that 
    ϑ_L^2≤ HΛ^-1/2Cc_#κ^-1/2(f-u_t)_L^2.

    For ρ,
    
    ((u-u^ms)_t,v) + 𝒜(u-u^ms,v)
            = (u_t,v)+𝒜(u,v)-(f,v)=((u-u)_t,v).

    Putting v=ρ,
    
    (ρ_t,ρ)+ρ_𝒜^2    = ((u-u)_t,ρ)
    ρ_L^2∂/∂ tρ_L^2=1/2∂/∂ tρ^2_L^2   ≤(u-u)_t_L^2ρ_L^2.

    Hence, for any t>0,
    
    ρ(·,t)_L^2≤ρ(·,0)_L^2+∫^t_0(u-u)_t(·,s)_L^2 s.

    All in all,
    
    ∫^T_0u-u^ms_L^2^2
            =2 ∫^T_0 ϑ(·,t)_L^2^2+ρ(·,t)_L^2^2
            
       ≤ 2H^2Λ^-1C^2c_#^2
            ∫^T_0 κ^-1/2(f-u_t)_L^2^2
            +2∫^T_0 {ρ(·,0)_L^2+∫^t_0(u-u)_t(·,s) s}^2  t
       ≤2H^2Λ^-1C^2c_#^2
            ∫^T_0 κ^-1/2(f-u_t)_L^2^2
            +4∫^T_0[ρ(·,0)_L^2^2
            +∫^t_0 t(u-u)_t(·,s)^2  s ] t
       ≤ 2H^2Λ^-1C^2c_#^2
            ∫^T_0 κ^-1/2(f-u_t)_L^2^2
            +4∫^T_0ρ(·,0)_L^2^2
            +4∫^T_0 t(T-t)(u-u)_t^2  t
       ≤ 2H^2Λ^-1C^2c_#^2
            ∫^T_0 κ^-1/2(f-u_t)_L^2^2
            +4∫^T_0ρ(·,0)_L^2^2
            +T^2∫^T_0(u-u)_t^2  t.

    The desired results are then obtained by using equation (<ref>).


    If furthermore C_invθ^(m-1)/2(m+2)^d/2=O(H^2), then
    
    ∫^T_0u-u^ms^2_L^2≤ O((T+T^3)H^4) and (u-u^ms)(·,T)_L^2=O(H^2+√(T)H^2).




We can now combine all the results to bound the terms in equation (<ref>).


    If g_tt, q_t and u_tt∈ L^2(Ω, (0,T)) exist on (0,T) and C_invθ^(m-1)/2(m+2)^d/2=O(H^2), then
    
    ∫^T_0u-u^ms_𝒜^2≤ O(H^4+(T+T^2) H^2)



    Recall from equation (<ref>), using the results from the time-independent problem, we take v=w^m_*-𝒟^mg+𝒩^m q+g. Then,
    
    1/4(i) = (u-u^ms)(·,0)_L^2^2
            ≤ O(H^4).

    
    1/2(ii)    = (u-v)(·,0)_L^2^2
        ≤ H^2c_#^2 (u-v)(·,0)_ℬ^2
       ≤ H^2c_#^2 {(u-u)(·,0)_𝒜+(w^glo-w^m_*)(·,0)_ℬ.
       +.(𝒟^glog-𝒟^m g)(·,0)_ℬ
            +(𝒩^glo q-𝒩^m q)(·,0)_ℬ}^2
       ≤ O(H^4).

    
    1/4C^2(iii)
            =u-v_ℰ^2
       ≤{u-u_ℰ
            +w^glo-w^m_*_ℰ
            +𝒟^glog-𝒟^m g_ℰ
            +𝒩^glo q-𝒩^m q_ℰ}^2
       ≤{
            O(H^2+√(T)H^2)+ O(H^2+√(T)H)
            +O(H^2+√(T)H)+O(H^2+√(T)H)
            }^2
       ≤ O(H^4+TH^2).

    
    1/4(iv)
               =√(∫^T_0u-u^ms_L^2^2)√(∫^T_0(u-v)_t_L^2^2)
       ≤ O(√(T+T^3)H^2){∫^T_0[
            (u-u)_t_L^2
            +(w^glo-w^m_*)_t_L^2
            +(𝒟^glog-𝒟^m g)_t_L^2..
       ..+(𝒩^glo q-𝒩^m q)_t_L^2]^2}^1/2
       ≤ O(√(T+T^3)H^2)O(H^3+√(T)H^2)
       = O((T+T^2)H^4).

    Altogether,
    
    (u-u^ms)(·,T)_L^2^2+∫^T_0u-u^ms_𝒜^2≤ O(H^4+(T+T^2)H^2).





 §.§ Temporal Discretization


We apply the Backward Euler method to the scheme. Explicitly, we let τ be the time step and U^n_ms u^ms(t_n).
Then, the variational formulation (<ref>) becomes

    (U^n_ms-U^n-1_ms/τ,v) + 𝒜(U^n_ms,v) = (f(t_n),v)     for  v∈ V^m_ms,
    
        U^0_ms = u^0_ms.



  §.§.§ Backward Euler Schemes


We compare two versions of the Backward Euler Scheme for convection diffusion, using the Dirichlet boundary condition as an illustration. The Neumann and Robin conditions follow similar treatments. Let τ be the timestep. 

    
  * 
    u^n+1-u^n/τ+ (·∇ u^n+1)=∇· (A ∇ u^n+1)

    
  * 
    u^n+1-u^n/τ+ (·∇ u^n)=∇· (A ∇ u^n+1).



Respectively, the application of the method becomes: given the multiscale solution u^n at step n,

    
  * Convection-Diffusion approach (CD-approach)
    
        
  * Find 𝒟^m_i g^n+1∈ V^m_ms such that for v∈ V^m_ms such that
        
    (𝒟^m_i g^n+1-𝒟^m_i g^n/τ,v) + ℬ(𝒟^m_i g^n+1,v)=𝒜_(K_i)(g^n+1,v)+(g_t^n+1,v)_(K_i).

        Then, set 𝒟^m g^n+1 = ∑^N_i=1𝒟^m_i g^n+1.
        
  * Find w^n+1∈ V^m_ms such that for v∈ V^m_ms, 
        
    (w^n+1-w^n/τ,v)+𝒜(w^n+1,v)
                       = (f^n+1,v)-𝒜(g^n+1,v)-(g^n+1_t,v)
       +𝒜(𝒟^m g^n+1,v)+(𝒟^m g^n+1_t,v).

        
  * Set u_ms^n+1=w^n+1-𝒟^m g^n+1+g^n+1.
    
    
  * Diffusion approach (D-approach)
    
        
  * Find 𝒟^m_i g^n+1∈ V^m_ms such that for v∈ V^m_ms such that
        
    (𝒟^m_i g^n+1-𝒟^m_i g^n/τ,v) + a(𝒟^m_i g^n+1,v)+s(π𝒟^m_i g^n+1,π v)=a_(K_i)(g^n+1,v)+(g_t^n+1,v)_(K_i).

        Then, set 𝒟^m g^n+1 = ∑^N_i=1𝒟^m_i g^n+1.
        
  * Find w^n+1∈ V^m_ms such that for v∈ V^m_ms,
        
    (w^n+1-w^n/τ,v)+a(w^n+1,v)
                       = (f^n+1,v)-a(g^n+1,v)-(g^n+1_t,v)
       -(·∇ u^n,v)+a(𝒟^m g^n+1,v)+(𝒟^m g^n+1_t,v).

        
  * Set u_ms^n+1=w^n+1-𝒟^m g^n+1+g^n+1.
    




 §.§ Nonlinear Convection Diffusion IBVPs


In this subsection, we are interested in the convection diffusion with a nonlinearity term f(u):

    ∂_t u + ·∇ u = f(u) + ∇·(Au),     in Ω×[0,T],
    
            u = g,     on Γ_D×[0,T],
    
            b u + ·(A∇ u - u) = q,     on Γ_N×[0,T],
    
            u(·,0) = u_init,     on Ω.



The traditional approach is to perform operator splitting. It is to decompose the convection-diffusion operator into two sub-problems, each targeting one operator <cit.>. 
However, with inhomogeneous boundary conditions, some operators are not left-invariant <cit.>. Therefore, a correction term η_n needs to be introduced.
On the other hand, with the current scheme on convection diffusion equations, the convection and diffusion operators can be considered at once. In other words, our ultimate goal is to construct the solution at the next step via:

    u^n+1 = (𝒮^τ/2_f-η_n∘𝒮^τ_CD+η_n∘𝒮^τ/2_f-η_n)(u^n)

where 𝒮^τ/2_f-η_n maps to the solution considering the adjusted nonlinearity term f-η_n for a stepsize τ/2 and 𝒮^τ_CD+η_n maps to the solution considering the convection diffusion equation with the source term η_n. 
Also, this corrector term η_n is to be carefully selected and get cancelled out in the process; otherwise can be accumulated over time and become a stiff term. The problem becomes even more apparent with time-variant boundary conditions. In light of this, we will decompose the nonlinearity into two parts, one dependent on the boundary and one independent. We will illustrate this idea using time-invariant Dirichlet boundary conditions. The time-variant version extends naturally by following a similar fashion.
The proof of this well-known method can be found in <cit.>. To summarise: at each step n,

    
  * Define z∈ L^2(Ω) such that .z|_∂Ω=g(·,t_n).
    
  * Find w(τ/2) by 
    ∂_t w = f(w+z) - f(z)
    
            w(0) = u^n-z.

    
  * Find v(τ) by 
    ∂_t v +𝒜v = f(z) - 𝒜z -∂_t z
    
            v(0) = w(τ/2).

    
    
  * Find w(τ/2) by 
    ∂_t w = f(w+z) - f(z)
    
            w(0) = v(τ).

    
  * Define z∈ L^2(Ω) such that .z|_∂Ω=g(·,t_n+1).
    
  * Define u^n+1 = w(τ/2) + z.


Above, the introduction of the corrector f(z) is independent of the solution in Step 3. Therefore, f(z)-𝒜z-∂_t z can be treated as a source term of a convection diffusion equation, which has been discussed already.
We note that for the first step, one has the flexibility to determine the interior value of z.
Aligned with our previous direction, we will use our choice of 𝒟^mg-g and 𝒩^m q for z, catering to the corresponding boundary conditions.




 §.§ Numerical Results


We demonstrate the numerical experiments for time-invariant and time-variant boundary conditions. Ω, κ, and  are taken as in section 3. For simplicity, we assume the source term vanishes f=0. We compare the CD-approach and D-approach. Experiments will be run on the coarse mesh H=1/10,1/20 and 1/40 with the number of eigenfunctions l_m=3 and a fixed timestep τ = 1/10. Reference solutions are generated on a 200×200 mesh with 1000 timesteps with the bilinear Lagrange finite element method. Without further mentioning, the error terms are recorded at the final time T=1.



  §.§.§ Time-invariant Boundary Conditions


We first consider time-invariant boundary conditions:

    u_t - ∇·(A∇ u) + ·∇ u = 0     in  [0,1]^2 × (0,1]
    
            u(x,t) = x_1^2+e^x_1x_2    for  (∂ [0,1]^2 )× (0,1]
    
            u(x,0) = x_1^2 + e^x_1 x_2    on  [0,1]^2.


Since the boundary condition is independent of time, 𝒟^m g_t and g_t vanish in the numerical method. The error analysis followed directly from the time-independent problem in section 3.



From Table <ref>, both second-order convergence in L^2-norm and at least first-order convergence in the energy norm can be observed for both D- and CD-approaches. Similar running times are recorded for both cases. However, since the convection term is dependent on the velocity field, the CD-approach outperforms D-approach in Table <ref>. Not only does it perform relatively poorly, the D-approach also fails to achieve the convergence in ·_𝒜 with respect to H.



  §.§.§ Time-variant Dirichlet Boundary Conditions


Consider the following:

    u_t - ∇·(κ∇ u) + ·∇ u = 0     in  [0,1]^2× [0,1]
    
            u(x,t) = (x_1^2+e^x_1 x_2)e^-t    for  (∂ [0,1]^2 )× [0,1]
    
            u(x,0) = x_1^2 + e^x_1 x_2    on  [0,1]^2.






 


As can be seen in Table <ref>, the D-approach is more efficient than the CD-approach, at the cost of a higher sensitivity to the Péclet number. CD-approach is more accurate at the cost of longer computational time. Even though both present spatial convergence but the CD-approach is more robust. The time-variant case also verifies our theoretical error estimates by showing second-order (or higher) convergence in L^2-norm and first-order convergence in the energy norm with respect to H. As in Table <ref>, the increase in the number of oversampling layers does allow decay in the corrector error, and therefore verifies our theoretical convergence in H as shown in Table <ref>. The solution profiles can be found in Figure <ref>.






  §.§.§ Nonlinear Example


We demonstrate combining CEM-GMsFEM with Strang Splitting to solve the following nonlinear convection-diffusion problem with a time-invariant Dirichlet boundary condition:

    ∂_t u-∇·(A(x)∇ u)+ (x) ·∇ u = u-u^3     on [0,1]^2× (0,1]
    
            u = x_1^2+e^x_1x_2    on (∂ [0,1]^2 )× (0,1]
    
            u(·,0) = x_1^2 + e^x_1x_2    on  [0,1]^2

    with = _in and c_flow=1/4.  

 The reference solutions are generated on a 200× 200 mesh with 1000 steps using the bilinear Lagrange finite element method and Backward Euler for the time discretization. Running tests on a combinations of stepsize τ∈{1/10,1/20,1/40}, mesh size H∈{1/10,1/20,1/40} and oversampling layers N_ov∈{7,8,9}. 



As shown in Table <ref>, the first order convergence in energy norm with respect to space and time are guaranteed. However, to achieve second order convergence in L^2-norm, higher oversampling layer is needed. The results could potentially be improved via adapting other temporal discretization scheme such as exponential integration <cit.>. 





§ CONCLUSIONS

In this paper, we propose an application of CEM-GMsFEM to solve convection diffusion equations under various types of inhomogeneous boundary conditions along with high-contrast coefficients. The method begins with constructing an auxiliary space and builds a multiscale space upon it. Boundary correctors are built upon this multiscale space by solving local oversampled cell problems. For both time independent and dependent problems, we provide convergence anaylsis and show second order convergence in L^2 and first order convergence in energy norm with respect to the coarse mesh, given sufficient oversampling, as agreed with numerical results. We also compare different time discretization strategies using the Backward Euler scheme. For nonlinear problems, we apply this modified method with Strang Splitting and demonstrate using a Dirichlet initial boundary value problem.



§ ACKNOWLEDGEMENTS

The research of Eric Chung is partially supported by the Hong Kong RGC General Research Fund (Projects: 14304021 and 14305423).

 

get arXiv to do 4 passes: Label(s) may have changed. Rerun

 Paper Title: Well-conditioned dipole-type method of fundamental solutions: derivation and its mathematical analysis.



§ ABSTRACT 
    In this paper, we examine the dipole-type method of fundamental solutions, which can be conceptualized as a discretization of the "singularity-removed" double-layer potential. 
    We present a method for removing the ill-conditionality, which was previously considered a significant challenge, and provide a mathematical analysis in the context of disk regions. 
    Moreover, we extend the proposed method to the general Jordan region using conformal mapping, demonstrating the efficacy of the proposed method through numerical experiments.

65N80, 65N35, 65N12, 35J05, 35J08

Flare Accelerated Electrons in Kappa-Distribution from X-Ray Spectra with
Warm-Target Model
    [
    
===========================================================================================







§ INTRODUCTION


The method of fundamental solutions (MFS, for short) is a mesh-free numerical solver for partial differential equations, and it has been actively used especially in the field of engineering.
Its idea is very simple.
Let us consider the boundary-value problem for the linear partial differential operator ℒ; more precisely,

    ℒu=0   in Ω,
    ℬg=f   on ∂Ω,

where Ω⊂ℝ^2 denotes a region such that ℝ^2∖Ω≠∅.
The second equation represents the linear boundary condition such as Dirichlet (ℬ=id), Neumann (ℬ=∂_ν), or Robin (ℬ=id+γ∂_ν).
Suppose that the partial differential operator ℒ has a fundamental solution E; that is, E satisfies

    ℒE=δ

in the distributional sense.
The MFS offers an approximate solution to the problem (<ref>) as in the following procedure.

    
  * Choose N∈ℕ, and arrange the singular points {y_k}_k=1^N⊂ℝ^2∖Ω “suitably”.
    
  * Seek an approximate solution u^(N) of the form
    
    u^(N)(x)=∑_k=1^NQ_kE(x-y_k),   x∈ℝ^2∖{y_1,y_2,…,y_N}.

    
  * Determine the coefficients {Q_k}_k=1^N by the collocation method; that is, choose P∈ℕ so that P≥ N, arrange the collocation points {x_j}_j=1^P⊂∂Ω “suitably”, and impose the collocation equations:
    
    u^(N)(x_j)=f(x_j),  
            j=1,2,…,P,

    which is equivalent to the linear system
    
    𝐆Q=f,

    where
    
    𝐆(ℬE(x_j-y_k))_j=1,2,…,P
    k=1,2,…,N∈ℝ^P× N,  Q(Q_k)_k=1^N∈ℝ^N,  f(f(x_j))_j=1^P∈ℝ^P.



As previously stated, in contrast to the finite element method and the finite difference method, the MFS does not necessitate the partitioning of the region into a mesh. 
An approximate solution can be derived by simply positioning points in a suitable manner on the boundary and exterior of the region. 
Consequently, the MFS can be regarded as a mesh-free numerical method.
It has been demonstrated in numerous prior studies that when the singular and collocation points are correctly positioned and the boundary data is real-analytic, the approximation error exhibits a noteworthy property of decaying exponentially with respect to the number of points <cit.>.
Nevertheless, the outcomes of these mathematical investigations remain inadequate. For instance, there is a paucity of findings pertaining to the existence of approximate solutions in multiply-connected regions in the plane, error estimation, and analysis in high-dimensional spaces.
Furthermore, the actual numerical computation with the MFS is subject to the issue of ill-conditionality. 
This refers to the fact that the condition number of the coefficient matrix 𝐆 of a linear system (<ref>) increases exponentially with respect to the number of points, indicating that the linear system (<ref>) is exceedingly challenging to solve. 
Numerous prior studies have devised MFS-specific pre-processing techniques based on numerical linear algebra (see, for instance, <cit.>). 
Nevertheless, the issue of ill-conditionality persists, and it has been identified as a significant impediment in actual numerical computation.

In light of these considerations, Antunes put forth an alternative approach in his paper <cit.>, namely, modifying the basis function. 
This concept was previously proposed in <cit.> within the context of the radial basis function method.

The infinite series expansion obtained by the Taylor series expansion of the basis function is truncated by a finite number of terms (e.g., up to the M-th order terms) to obtain

    Θ(r,θ)=𝐁𝐃𝐅(r,θ),

where the singular points are arranged as y_k=ε^-1(cosϕ_k,sinϕ_k)^⊤ and 

    𝐁   [        1   sinϕ_1   cosϕ_1        ⋯ sin Mϕ_1 cos Mϕ_1;        1   sinϕ_2   cosϕ_2        ⋯ sin Mϕ_2 cos Mϕ_2;        ⋮        ⋮        ⋮        ⋮        ⋮;        1   sinϕ_N   cosϕ_N        ⋯ sin Mϕ_N cos Mϕ_N ]∈ℝ^N×(2M+1),
    𝐃   (
            logε,ε,ε,…,ε^M/M,ε^M/M)∈ℝ^(2M+1)×(2M+1),
    F(r,θ)   [         1;     rsinθ;     rcosθ;         ⋮; r^Mcos Mθ; r^Msin Mθ ]∈ℝ^2M+1,

and M is chosen to satisfy 2M+1>N.
In this matrix representation, matrix 𝐁 represents a well condition, whereas the ill-conditionality mainly stems from matrix 𝐃. 
Accordingly, the matrix 𝐁 is decomposed via the QR algorithm to yield the representation 𝐁=𝐐𝐑. 
The N-th leading principal submatrix of 𝐃 is subsequently designated as 𝐃_N. 
The new basis function is then expressed as

    Ψ(r,θ)
        =(ψ_k(r,θ))_k=1^N
        𝐃_N^-1𝐐^⊤Θ(r,θ)
        =𝐃_N^-1𝐑𝐃F(r,θ).

Subsequently, he put forth the proposition of MFS-QR as a MFS-based system, founded upon this novel basis function. 
The approximate solution is expressed in the form of

    u^(N)(x)=u^(N)(r,θ)=∑_k=1^NQ_kψ_k(r,θ),    
        x=[ rcosθ; rsinθ ].

The coefficients of the linear combination are determined by the collocation method. 

In paper <cit.>, it was numerically shown that the ill-conditionality of the coefficient matrix can be completely eliminated for the disk domain, and that for other domains, the ill-conditionality cannot be completely eliminated, but can be reduced to some extent. 
Later, in paper <cit.>, a method based on singular value decomposition (MFS-SVD) instead of QR decomposition was developed, and it was numerically confirmed that the ill-conditionality can be removed when the domain is star-shaped. 
It should be noted that in both cases, the order of convergence is the same as that of the original MFS. 
In other words, MFS-QR and MFS-SVD achieve the exponential decay of errors, which is one of the major advantages of MFS, while eliminating one of the major problems of MFS, i.e., the problem of ill-conditionality. 
Therefore, it is expected that MFS-QR and MFS-SVD, rather than ordinary MFS, will become more important numerical methods in the future, but unfortunately, there are no theoretical guarantees to support the numerical experimental results obtained in previous studies.

It is also important to consider whether the construction of the approximate solution of the MFS is a sensible approach in the first place. 
In a sense, the approximate solution obtained through the MFS can be regarded as a "singularity-removed" discretization of the single-layer potential. 
However, as is well established in the theory of potential problems <cit.>, the single-layer potential should be employed for problems with Neumann boundary conditions, whereas the double-layer potential is more appropriate for problems with Dirichlet boundary conditions. 
In light of this awareness of the problem, the dipole-type method of fundamental solutions (also known as the dipole simulation method) was proposed. 
For further details, please refer to  <ref>.
Henceforth, the dipole-type method of fundamental solutions will be referred to as the DSM.  
In the DSM, the approximate solution is constructed using the normal derivative of the fundamental solution as the basis function in lieu of the fundamental solution. 
In the original theoretical analysis of the MFS, it was necessary to make some mathematically unnatural assumptions. 
However, with the DSM, a similar theoretical analysis can be performed without the need to make unnatural assumptions. 
Therefore, it is more natural to use the DSM when dealing with Dirichlet boundary conditions. Nevertheless, even in the DSM, the problem of ill-conditionality persists, and to the best of the author's knowledge, there is no known solution to this problem.

The objective of this paper is to put forth a novel DSM, the DSM-QR, which is founded upon the tenets of the MFS-QR and eliminates the problematic ill-conditionality. 
Additionally, this paper endeavors to conduct a fundamental mathematical analysis of the DSM-QR. 
With the intention of establishing a robust foundation for future mathematical analysis, this paper concentrates on the Dirichlet boundary value problem

    u=0   in 𝔹_ρ,
    
            u=f   on ∂𝔹_ρ,

for the Laplace equation in the disk domain, where 𝔹_ρ denotes the disk in the complex plane with radius ρ having the origin as its center; that is,

    𝔹_ρ{z∈ℂ||z|<ρ}.

The collocation points {x_j}_j=1^P and singular points {y_k}_k=1^N are to be placed as

    x_j   =ρexp(2πij/P),          j=1,2,…,P,

    y_k   =Rexp(2πik/N),          k=1,2,…,N,

where R>ρ and ℝ^2 and ℂ are identified here and hereafter.
Moreover, set

    κρ/R.


The following three theorems represent the core of this paper's theoretical framework.


    
    The DSM-QR gives a unique approximate solution.


Denote by _2(𝐀) the condition number of 𝐀 induced by the ℓ^2-norm.


    
    The condition number _2(𝐆) of the coefficient matrix 𝐆 is of O(1) as N→∞.
    More precisely, it holds that
    
    _2(𝐆)=2/1+κ^N-2,  
            N∈ℕ.




    
    
        
  * Suppose that the Fourier series ∑_n∈ℤf_ne^inθ of θ↦ f(ρe^iθ) is absolutely convergent.
        Then, u^(N) uniformly converges to u on 𝔹_ρ; that is,
        
    u-u^(N)_L^∞(𝔹)⟶0  (N→∞).

        
  * Suppose there exists an α>1 such that |f_n|=O(|n|^-α) (|n|→∞) holds.
        Then, there exists a positive constant C, independent of N, such that
        
    u-u^(N)_L^∞(𝔹_ρ)≤ CN^-α+1

        holds.
        
  * Suppose that there exists an a∈(0,1) such that |f_n|=O(a^|n|) holds as |n|→∞.
        Then, there exists a positive constant C, independent of N, such that
        
    u-u^(N)_L^∞(𝔹_ρ)≤ C×
                    a^N/2   if a>κ^2,
    
                    Nκ^N   if a=κ^2,
    κ^N   if a<κ^2.

    


The following is a description of the organization of this paper. 
In Section <ref>, we undertake a review of the MFS from the perspective of potential theory, and subsequently derive a tabular expression for the approximate solution by the DSM. 
In Section <ref>, we put forth the DSM-QR, which is based on the concept of the MFS-QR. 
Section <ref> presents a mathematical analysis of DSM-QR. 
First, we provide a detailed account of the basis functions utilized in the DSM-QR through the presentation of concrete calculations pertaining to QR decomposition. 
Subsequently, the eigenvalues of the coefficient matrix are obtained in Lemma <ref>, which also demonstrates that the condition number of the coefficient matrix is of O(1). 
Then, we provide a proof of the error estimate presented in Theorem <ref>. 
In Section <ref>, we present the results of numerical experiments. 
First, we consider the case of the disk region and demonstrate that the problematic ill-conditionality is entirely eliminated and that the error decays exponentially, as predicted by the main theorem of this paper. 
Next, we extend the DSM-QR to the case of Jordan regions, although no mathematical analysis is available. 
Specifically, DSM-QR is extended to the general Jordan region by mapping point configurations in the disk region to the Jordan region by a conformal mapping, and by viewing basis functions as perturbations of basis functions in the disk region. 
The utility of this approach is demonstrated through numerical experiments. 
Section <ref> provides a summary of the contents of this paper and suggests avenues for future research.




§ DIPOLE SIMULATION METHOD



In this section, we briefly explain the idea of the dipole simulation method (DSM).

Let Γ be a Jordan curve surrounding Ω; more precisely, Ω⊂Ω[Γ], where Ω[Γ] denotes a Jordan region with boundary Γ and Ω[Γ] denotes the interior of Ω[Γ].
Suppose that there exists a function Q defined on Γ such that the solution u to the Dirichlet boundary value problem (<ref>) can be expressed as

    u(z)=∫_Γ E(z-ζ)Q(ζ) dσ(ζ)SL[Q](z),

which can regarded as a “singularity-removed” single-layer potential.
Here, E=E(z)=-(2π)^-1log|z| is the fundamental solution to the Laplace operator -.
If we consider a linear combination of Dirac delta distributions Q^(N)(z)∑_k=1^NQ_kδ(ζ-ζ_k) with ζ_k∈Γ (k=1,2,…,N), we formally obtain that

    SL[Q^(N)](z)
        =∑_k=1^NQ_kE(z-ζ_k),

which is an approximate solution by the MFS.
Namely, the MFS can be regarded as a discretization of the “singularity-removed” single-layer potential representation.

However, it is natural to consider the double-layer potential, not single-layer potential, when we deal with the Dirichlet boundary condition (see, for instance, <cit.>).
Recall that the dipole-layer potential is given by replacing the integral kernel E(z-ζ) with its normal derivative as

    DL[Q](z)∫_Γ∂ E/∂ν(ζ)(z-ζ) dζ,

where ν(ζ) denotes the unit outward normal vector to Γ.
Then, we formally obtain

    DL[Q^(N)](z)
        =∑_k=1^NQ_k∂ E/∂ν(ζ_k)(z-ζ_k).

Therefore, it is natural to consider that a function of the form (<ref>) can also approximate the solution to the Dirichlet boundary value problem (<ref>).
This is the idea of the DSM.
More precisely, the algorithm of the DSM can be described as follows:

    
  * Choose N∈ℕ, and arrange the singular points {ζ_k}_k=1^N⊂ℂ∖Ω and the dipole moments {ν_k}_k=1^N⊂ S^1 suitably, where S^1{z∈ℂ||z|=1} denotes the unit circle.
    
  * Seek an approximate solution u^(N) of the form
    
    u^(N)(z)=∑_k=1^NQ_k∂ E/∂ν_k(z-ζ_k).

    
  * Determine the coefficients {Q_k}_k=1^N by the collocation method; that is, arrange the collocation points {z_j}_j=1^P⊂∂Ω suitably, and impose the collocation equations:
    
    u^(N)(z_j)=f(z_j),  
            j=1,2,…,P.





§ DSM-QR



In this section, we develop DSM-QR algorithm.
In particular, we replace the basis functions {(∂ E/∂ν_k)(·-ζ_k)}_k=1^N with new ones by a similar idea to the MFS-QR.

Let us arrange the singular points ζ_k on the circle with radius R uniformly as

    ζ_k Rω^k  (k=1,2,…,N),  ωexp2πi/N.

Regarding the circle |z|=R as Γ, it is natural to define dipole moments ν_k as

    ν_kζ_k/|ζ_k|=ω^k  (k=1,2,…,N).

Then, the basis function of the DSM can be computed as

    ∂ E/∂ν_k(z-ζ_k)
        =-1/2π(ν_k/z-ζ_k)
        =-1/2π(ω^k/z-Rω^k).


For z=re^iθ and ζ=Re^iϕ with 0≤ r≤ρ<R, we formally have

    -1/2π(ζ/|ζ|/z-ζ)
       =-1/2π(e^iϕ/re^iθ-Re^iϕ)
       =1/2π R[∑_n=0^∞(r/R)^ne^in(θ-ϕ)]
       =1/2π R[1+∑_n=1^∞(r/R)^ncos n(θ-ϕ)]
       =1/2π R[1+∑_n=1^∞(r/R)^n(cos nθcos nϕ+sin nθsin nϕ)]
       =1/2π R[     1  sinϕ  cosϕ sin2ϕ cos2ϕ     ⋯ ][   1;       κ;       κ;     κ^2;     κ^2;       ⋱ ][        1;    ssinθ;    scosθ; s^2sin2θ; s^2cos2θ;        ⋮ ],

where κρ/R∈(0,1) and s r/ρ∈[0,1].
Basis functions (<ref>) of the DSM can be obtained by taking ρ=ϕ_k2π k/N (k=1,2,…,N).
Namely, 

    [ ∂ E∂ν_1(z-ζ_1); ∂ E∂ν_2(z-ζ_2);              ⋮; ∂ E∂ν_N(z-ζ_N) ]
        =
        1/2π R[       1  sinϕ_1  cosϕ_1 sin2ϕ_1 cos2ϕ_1       ⋯;       1  sinϕ_2  cosϕ_2 sin2ϕ_2 cos2ϕ_2       ⋯;       ⋮       ⋮       ⋮       ⋮       ⋮;       1  sinϕ_N  cosϕ_N sin2ϕ_N cos2ϕ_N       ⋯ ]
    [   1;       κ;       κ;     κ^2;     κ^2;       ⋱ ][        1;    ssinθ;    scosθ; s^2sin2θ; s^2cos2θ;        ⋮ ]

Hereafter, we omit the constant factor 1/(2π R).

Let M∈ℕ satisfying 2M+1>N, and truncate the above infinite series expansion at the M-th order.
More precisely, set

    𝐁   [        1   sinϕ_1   cosϕ_1        ⋯ sin Mϕ_1 cos Mϕ_1;        1   sinϕ_2   cosϕ_2        ⋯ sin Mϕ_2 cos Mϕ_2;        ⋮        ⋮        ⋮        ⋮        ⋮;        1   sinϕ_N   cosϕ_N        ⋯ sin Mϕ_N cos Mϕ_N ]∈ℝ^N×(2M+1),
    𝐃   (1,κ,κ,…,κ^M,κ^M)∈ℝ^(2M+1)×(2M+1),
    F(s,θ)   [         1;     ssinθ;     scosθ;         ⋮; s^Msin Mθ; s^Mcos Mθ ]∈ℝ^2M+1.

We perform the QR decomposition of 𝐁 as 𝐁=𝐐𝐑, where 𝐐∈ℝ^N× N is an orthogonal matrix and 𝐑∈ℝ^N×(2M+1) is an upper triangular matrix.
Furthermore, let 𝐃_N be the N-th leading principal submatrix of 𝐃.
Then, we define new basis functions as

    Ψ(s,θ)
        =(ψ_k(s,θ))_k=1^N
        𝐃_N^-1𝐐^⊤𝐐𝐑𝐃F(s,θ)
        =𝐃_N^-1𝐑𝐃F(s,θ).


Hence, the DSM-QR gives an approximate solution to problem (<ref>) of the form

    u^(N)(z)
        =u^(N)(r,θ)
        =∑_k=1^NQ_kψ_k(r/ρ,θ),  
        z=re^iθ.

Coefficients {Q_k}_k=1^N are determined by the collocation method; that is,

    u^(N)(ρe^iθ_j)=f(ρe^iθ_j),  θ_j2π j/P,  
        j=1,2,…,P,

which is equivalent to the linear system

    𝐆Q=f,

where

    𝐆(ψ_k(1,θ_j))_j=1,2,…,P
    k=1,2,…,N∈ℝ^P× N,
          Q(Q_k)_k=1^N∈ℝ^N,  f(f(ρe^iθ_j))_j=1^P∈ℝ^P.




§ MATHEMATICAL ANALYSIS OF DSM-QR





 §.§ Concrete expression of basis functions of DSM-QR


Hereafter, we choose M=N-1 and suppose that N is odd and greater than 1 for the sake of simplicity.
We explicitly compute the QR decomposition of 𝐁.
To this end, define vectors c_j,s_j∈ℝ^N by

    c_j
        [ cos jϕ_1; cos jϕ_2;        ⋮; cos jϕ_N ],
          s_j
        [ sin jϕ_1; sin jϕ_2;        ⋮; sin jϕ_N ]

for j∈ℤ.
Then, the matrix 𝐁 can be represented as

    𝐁
        =[ c_0 s_1 c_1   ⋯ s_N c_N ].




    
    It holds that
    
           ⟨c_j,c_j'⟩_ℓ^2=⟨s_j,s_j'⟩_ℓ^2=0        for 0≤j,j'≤N-1/2 with j≠j',

           ⟨c_j,s_j'⟩_ℓ^2=0        for 0≤j,j'≤N-1/2,

           c_0_ℓ^2=√(N),

           c_j_ℓ^2=s_j_ℓ^2=√(N/2)        for 1≤j≤N-1/2.
    
    Furthermore, it holds that
    
    c_N-j=c_j,  s_N-j=-s_j,  
            1≤ j≤N-1/2.



Since the above lemma immediately follows from the elementary relation

    ∑_l=1^Nω^jl=
            0   if j≢0,
    
            N   otherwise,

we omit its proof.
Here and hereafter, the congruence ≡ is taken in modulo N.

Define

    q_j
           c_l/|c_l|   if j=2l, l=0,1,…,N-12,
    s_l/|s_l|   if j=2l-1, l=1,2,…,N-12
       =c_0/√(N)   if j=0,
    s_l/√(N/2)   if j=2l-1, l=1,2,…,N-12,
    c_l/√(N/2)   if j=2l, l=1,2,…,N-12.

Then, Lemma <ref> yields that

    𝐁   =[     c_0     s_1     c_1       ⋯ s_N-1/2 c_N-1/2 s_N+1/2 c_N+1/2       ⋯   s_N-1   c_N-1 ]
       =[      c_0      s_1      c_1        ⋯  s_N-1/2  c_N-1/2 -s_N-1/2  c_N-1/2        ⋯     -s_1      c_1 ]
       =[     c_0     s_1     c_1       ⋯ s_N-1/2 c_N-1/2 ][  1;     1 -1;     1  1;     ⋱  ⋱;     1 -1;     1  1 ]
       =[   q_0   q_1   q_2     ⋯ q_N-2 q_N-1 ]√(N/2)[ √(2);         1   -1;         1    1;         ⋱    ⋱;         1   -1;         1    1 ].

Therefore, setting

    Q   [   q_0   q_1   q_2     ⋯ q_N-2 q_N-1 ],
    𝐑   √(N)[ √(2);         1   -1;         1    1;         ⋱    ⋱;         1   -1;         1    1 ],

we have 𝐁=𝐐𝐑.
Since 𝐐 is an orthogonal matrix by Lemma <ref> and 𝐑 is an upper triangular matrix, this is the desired QR decomposition of 𝐁.
A direct computation shows that

    𝐃_N^-1𝐑𝐃=(r̃_ij)_i=1,2,…,N
    j=1,2,…,2N-1,
       r̃_ij=√(N)   if i=j=1,
    √(N/2)   if i=j=2,3,…,N,
    
            -√(N/2)κ^-⌊ i/2⌋+⌊ j/2⌋   if i=2l (l=1,2,…,N-12), j=2N-i,
    √(N/2)κ^-⌊ i/2⌋+⌊ j/2⌋   if i=2l+1 (l=1,2,…,N-12), j=2(N+1)-i.

Therefore, the basis functions Ψ=(ψ_k)_k=1^N of the DSM-QR, defined as (<ref>), can be explicitly given by

    ψ_k(s,θ)=√(N)   if k=1,
    √(N/2)[s^msin mθ-κ^N-2ms^N-msin(N-m)θ]   if k=2m, m=1,2,…,N-12,
    √(N/2)[s^mcos mθ+κ^N-2ms^N-mcos(N-m)θ]   if k=2m+1, m=1,2,…,N-12.




 §.§ Unique existence and condition number


In this subsection, we prove that the linear system (<ref>) is uniquely solvable.
We will establish mathematical theory for P=α N with α∈ℕ.
When α=1, the linear system (<ref>) is understood in a standard manner.
When α≥2, we seek a solution to the linear sytem (<ref>) in the least-squares sense; that is,

    Q∈_Q̃∈ℝ^N𝐆Q-f_ℓ^2^2,

which is equivalent to

    𝐆^⊤𝐆Q=𝐆^⊤f.

Hereafter, we focus on the case where α=1.
Other cases α≥2 can be shown in a similar manner.

We examine properties of the coefficient matrix 𝐆^⊤𝐆.


    
    The matrix 𝐆^⊤𝐆∈ℝ^N× N is diagonal and its diagonal elements are given by
    
    [𝐆^⊤𝐆]_kk=
                N^2   if k=1,
    N^2/4(1+κ^N-2m)^2   if k=2m or k=2m+1 with m=1,2,…,N-12.




    By the definition of 𝐆, we have
    
    𝐆^⊤𝐆=(∑_j=1^Nψ_k(1,θ_j)ψ_l(1,θ_j))_k,l∈ℝ^N× N,

    and
    
    ψ_k(1,θ_j)=√(N)   if k=1,
    √(N/2)[sin mθ_j-κ^N-2msin(N-m)θ_j]   if k=2m, m=1,2,…,N-12,
    √(N/2)[cos mθ_j+κ^N-2mcos(N-m)θ_j]   if k=2m+1, m=1,2,…,N-12.

    Using an elementary relation (<ref>), it can be easily shown that the following hold:
    
    ∑_j=1^Nsin mθ_j=0,   m∈ℤ,
       ∑_j=1^Ncos mθ_j=
                N   if m≡0,
    
                0   otherwise,
            
       ∑_j=1^Ncos mθ_jcos nθ_j=N/2   if (m-n≡0)⊻(m+n≡0),
    
                N   if (m-n≡0)(m+n≡0),
    
                0   otherwise,
            
       ∑_j=1^Nsin mθ_jsin nθ_j=N/2   if (m-n≡0)(m+n≢0),
    
                -N/2   if (m+n≡0)(m-n≢0),
    
                0   otherwise,
            
       ∑_j=1^Nsin mθ_jcos nθ_j=0,   m,n∈ℤ,

    where the symbol ⊻ denotes the exclusive or.

    Off-diagonal elements can be computed as in the following manner.
    (1,2n) and (1,2n+1) elements for n=1,2,…,(N-1)/2 are
    
    [𝐆^⊤𝐆]_1,2n   =N/√(2)∑_j=1^N[sin nθ_j-κ^N-2nsin(N-n)θ_j]=0,
    
            [𝐆^⊤𝐆]_1,2n+1   =N/√(2)∑_j=1^N[cos nθ_j+κ^N-2ncos(N-n)θ_j]=0

    by (<ref>) and (<ref>).
    (2m,2n) elements for m=1,2,…,(N-1)/2 and n=m+1,…,(N-1)/2 are
    
    [𝐆^⊤𝐆]_2m,2n   =N/2∑_j=1^N[sin mθ_jsin nθ_j-κ^N-2nsin mθ_jsin(N-n)θ_j.
       .-κ^N-2msin(N-m)θ_jsin nθ_j+κ^2(N-(m+n))sin(N-m)θ_jsin(N-n)θ_j]
       =0

    by (<ref>).
    (2m,2n+1) elements for m=1,2,…,(N-1)/2 and n=m,…,(N-1)/2 are
    
    [𝐆^⊤𝐆]_2m,2n+1   =N/2∑_j=1^N[sin mθ_jcos nθ_j+κ^N-2nsin mθ_jcos(N-n)θ_j.
       .-κ^N-2msin(N-m)θ_jcos nθ_j-κ^2(N-(m+n))sin(N-m)θ_jcos(N-n)θ_j]
       =0

    by (<ref>).
    (2m+1,2n+1) elements for m=1,2,…,(N-1)/2 and n=m+1,…,(N-1)/2 are
    
    [𝐆^⊤𝐆]_2m+1,2n+1   =N/2∑_j=1^N[cos mθ_jcos nθ_j+κ^N-2ncos mθ_jcos(N-n)θ_j.
       .+κ^N-2mcos(N-m)θ_jcos nθ_j+κ^2(N-(m+n))cos(N-m)θ_jcos(N-n)θ_j]

    by (<ref>).
    In the same way in computing (2m,2n+1) elements, we can show that (2m+1,2n) elements are equal to 0 for m=1,2,…,(N-1)/2 and n=m+1,…,(N-1)/2.
    Summarizing the above, we have shown that upper triangular elements of 𝐆^⊤𝐆 are equal to 0.
    Since 𝐆^⊤𝐆 is symmetric, it follows that 𝐆^⊤𝐆 is diagonal.

    Concerning diagonal elements, we have by (<ref>) and (<ref>) that
    
    [𝐆^⊤𝐆]_11   =∑_j=1^NN
            =N^2,
    
            [𝐆^⊤𝐆]_2m,2m   =N/2∑_j=1^N[sin^2mθ_j-2κ^N-2msin mθ_jsin(N-m)θ_j+κ^2(N-2m)sin^2(N-m)θ_j]
       =N^2/4(1+κ^N-2m)^2,
    
            [𝐆^⊤𝐆]_2m+1,2m+1   =N/2∑_j=1^N[cos^2mθ_j+2κ^N-2mcos mθ_jcos(N-m)θ_j+κ^2(N-2m)cos^2(N-m)θ_j]
       =N^2/4(1+κ^N-2m)^2

    for m=1,2,…,(N-1)/2.


The above lemma shows that 𝐆^⊤𝐆 is non-singular since all the eigenvalues of 𝐆^⊤𝐆 are positive.
Namely, Theorem <ref> has been proved.


    It is well known that the relation
    
    _2(𝐀)=√(λ_max(𝐀^⊤𝐀)/λ_min(𝐀^⊤𝐀))

    holds, where λ_max(𝐀^⊤𝐀) and λ_min(𝐀^⊤𝐀) are the maximal and minimal (by moduli) eigenvalues of 𝐀^⊤𝐀, respectively.
    Lemma <ref> shows that
    
    λ_max(𝐆^⊤𝐆)=N^2,  λ_min(𝐆^⊤𝐆)=N^2/4(1+κ^N-2)^2

    hold.
    Therefore, the relation (<ref>) yields that
    
    _2(𝐆)
            =√(λ_max(𝐆^⊤𝐆)/λ_min(𝐆^⊤𝐆))
            =2/1+κ^N-2⟶2  (N→∞).



The above corollary implies that the coefficient matrix 𝐆 in the collocation equation (<ref>) is well-conditioned, which is a quite difference compared with the original DSM.



 §.§ Error estimate



Since 𝐆^⊤𝐆 is diagonal by Lemma <ref>, the solution to (<ref>) is explicitly given by

    Q_k=1/[𝐆^⊤𝐆]_kk∑_j=1^Nψ_k(1,θ_j)f(ρe^iθ_j),  
        k=1,2,…,N.

To derive the error estimate, we write down the exact solution u to the problem (<ref>) using the Fourier series expansion as

    u(re^iθ)=∑_n∈ℤf_n(r/ρ)^|n|e^inθ,

where f_n denotes the n-th Fourier coefficient of the function θ↦ f(ρe^iθ).
We next represent u^(N) as an infinite series using {f_n}_n∈ℤ.
The concrete expression (<ref>) of coefficients {Q_k}_k=1^N yields that

    u^(N)(re^iθ)
           =∑_k=1^N[1/[𝐆^⊤𝐆]_k∑_j=1^Nψ_k(1,θ_j)f(ρe^iθ_j)]ψ_k(r/ρ,θ)
       =∑_k=1^N1/[𝐆^⊤𝐆]_k∑_j=1^Nψ_k(1,θ_j)(∑_n∈ℤf_ne^inθ_j)ψ_k(r/ρ,θ)
       =∑_n∈ℤf_n∑_k=1^N1/[𝐆^⊤𝐆]_k(∑_j=1^Nψ_k(1,θ_j)e^inθ_j)ψ_k(r/ρ,θ).



    
    It holds that
    
    φ_n^(N)(r/ρ,θ)
               ∑_k=1^N(1/[𝐆^⊤𝐆]_k∑_j=1^Nψ_k(1,θ_j)e^inθ_j)ψ_k(r/ρ,θ)
       =
                1   if n≡0,
    1/1+κ^N-2m(w^m+κ^N-2mw^N-m)   if n≡ m,
    1/1+κ^N-2m(w^m+κ^N-2mw^N-m)   if n≡-m

    with m=1,…,(N-1)/2, where we set w(r/ρ)e^iθ.



    We first compute ∑_j=1^Nψ_k(1,θ_j)e^inθ_j.
    The elementary relation (<ref>) and expression (<ref>) yield that
    
    ∑_j=1^Nψ_1(1,θ_j)
            =√(N)∑_j=1^Ne^inθ_j
            =
                N^3/2   if n≡0,
    
                0   otherwise

    for k=1,
    
    ∑_j=1^Nψ_2m(1,θ_j)e^inθ_j
       =√(N/2)∑_j=1^N(e^i(m+n)θ_j-e^-i(m-n)θ_j/2i-κ^N-2me^i(N-(m-n))θ_j-e^-i(N-(m+n))θ_j/2i)
       =i(N/2)^3/2×
                1+κ^N-2m   if n≡ m,
    
                -(1+κ^N-2m)   if n≡-m,
    
                0   otherwise

    for k=2m with m=1,…,(N-1)/2, and
    
    ∑_j=1^Nψ_2m+1(1,θ_j)e^inθ_j
       =√(N/2)∑_j=1^N(e^i(m+n)θ_j+e^-i(m-n)θ_j/2+κ^N-2me^i(N-(m-n))θ_j+e^-i(N-(m+n))θ_j/2)
       =(N/2)^3/2×
                1+κ^N-2m   if (n≡ m)(n≡-m),
    
                0   otherwise

    for k=2m+1 with m=1,…,(N-1)/2.
    Therefore, it follows from the above result and Lemma <ref> that
    
    ∑_k=1^N1/[𝐆^⊤𝐆]_k(∑_j=1^Nψ_k(1,θ_j)e^inθ_j)ψ_k(r/ρ,θ)
            =1/[𝐆^⊤𝐆]_1· N^3/2ψ_1(r/ρ,θ)
            =1

    for n≡0, 
    
    ∑_k=1^N1/[𝐆^⊤𝐆]_k(∑_j=1^Nψ_k(1,θ_j)e^inθ_j)ψ_k(r/ρ,θ)
       =(N/2)^3/2(1+κ^N-2m)(i/[𝐆^⊤𝐆]_2mψ_2m(r/ρ,θ)+1/[𝐆^⊤𝐆]_2m+1ψ_2m+1(r/ρ,θ))
       =1/1+κ^N-2m(w^m+κ^N-2mw^N-m)

    for n≡ m, and
    
    ∑_k=1^N1/[𝐆^⊤𝐆]_k(∑_j=1^Nψ_k(1,θ_j)e^inθ_j)ψ_k(r/ρ,θ)
       =(N/2)^3/2(1+κ^N-2m)(-i/[𝐆^⊤𝐆]_2mψ_2m(r/ρ,θ)+1/[𝐆^⊤𝐆]_2m+1ψ_2m+1(r/ρ,θ))
       =1/1+κ^N-2m(w^m+κ^N-2mw^N-m)

    for n≡-m.


Using the functions φ_n^(N) defined in Lemma <ref>, we can represent u^(N)(re^iθ) as

    u^(N)(re^iθ)=∑_n∈ℤf_nφ_n^(N)(r/ρ,θ).

Since u and u^(N) are harmonic in 𝔹_ρ and continuous on 𝔹_ρ, the maximum principle gives the following bound:

    u-u^(N)_L^∞(𝔹_ρ)   ≤∑_n∈ℤ|f_n|g_n^(N),

where

    g_n^(N)sup_θ∈ℝ|e^inθ-φ_n^(N)(1,θ)|.


To derive the desired error estimate, the local and global estimates on g_n^(N) are required.


    
    
        
  * For any n∈ℕ, it holds that
        
    g_n^(N)≤2.

        
  * It holds that
        
    g_0^(0)=0.

        Moreover, for n=1,…,(N-1)/2, it holds that
        
    g_n^(N)≤2κ^N-2n.

    



    (i)
    Lemma <ref> yields that
    
    φ_n^(N)(1,θ)=
                1   if n≡0,
    1/1+κ^N-2m(e^imθ+κ^N-2me^-i(N-m)θ)   if n≡ m,
    1/1+κ^N-2m(e^-imθ+κ^N-2me^i(N-m)θ)   if n≡-m

    with m=1,…,(N-1)/2.
    Therefore, the triangle inequality implies that φ_n^(N)(1,θ)≤1 for any n∈ℤ and θ∈ℝ.
    We thus obtain
    
    g_n^(N)≤sup_θ∈ℝ(|e^inθ|+|φ_n^(N)(1,θ)|)
            ≤2.


    (ii)
    Since φ_0^(N)(1,θ)=1 as shown in the above, we readily see that g_0^(N)=0 holds.
    For n=1,…,(N-1)/2, we have
    
    e^inθ-φ_n^(N)(1,θ)
               =e^inθ-1/1+κ^N-2n(e^inθ+κ^N-2ne^-i(N-n)θ)
       =κ^N-2n/1+κ^N-2n(e^inθ-e^-i(N-n)θ).

    Hence, the triangle inequality yields that
    
    |e^inθ-φ_n^(N)(1,θ)|
            ≤2κ^N-2n/1+κ^N-2n≤2κ^N-2n,

    which is the desired estimate.



    (i), (ii)
    Note that φ_-n^(N)=φ_n^(N) holds, which implies that g_-n^(N)=g_n^(N).
    Based on this relation, we decompose the right-hand side of (<ref>) as follows:
    
    ∑_n∈ℤ|f_n|g_n^(N)
            =|f_0|g_0^(N)+∑_n=1^⌊ N/3⌋-1(|f_n|+|f_-n|)g_n^(N)+∑_n=⌊ N/3⌋^∞(|f_n|+|f_-n|)g_n^(N)≡I+II+III.


    Since g_0^(N)=0 by Lemma <ref>(ii), we have I=0.
    Concerning II, since |f_n|≤f_L^∞(∂𝔹_ρ) holds, Lemma <ref>(ii) implies that
    
    II≤4f_L^∞(∂𝔹_ρ)∑_n=1^⌊ N/3⌋-1κ^N-2n≤4f_L^∞(∂𝔹_ρ)κ^N·κ^-2⌊ N/3⌋/κ^-2-1≤4f_L^∞(∂𝔹_ρ)/κ^-2-1κ^N/3,

    Lemma <ref>(i) yields that
    
    III≤2∑_n=⌊ N/3⌋^∞(|f_n|+|f_-n|)
            =o(1)   for case (i),
    O(N^-α+1)   for case (ii).


    Summarizing the above, we complete the proof of (i) and (ii).

    (iii)
    We decompose the right-hand side of (<ref>) as
    
    ∑_n∈ℤ|f_n|g_n^(N)
            =|f_0|g_0^(N)+∑_n=1^(N-1)/2(|f_n|+|f_-n|)g_n^(N)+∑_n=(N+1)/2^∞(|f_n|+|f_-n|)g_n^(N)≡I+II+III.


    Since g_0^(0)=0 by Lemma <ref>(ii), we have I=0.
    The assumption |f_n|=O(a^|n|) (|n|→∞) assures the existence of positive constant C satisfying |f_n|≤ Ca^|n| for all n∈ℤ.
    Combining this relation with Lemma <ref>(ii), we have
    
    II≤4C∑_n=1^(N-1)/2a^nκ^N-2n
            =4Cκ^N∑_n=1^(N-1)/2(aκ^-2)^n
            ≤
            4C×(aκ^-2)^1/2/aκ^-2-1a^N/2   if a>κ^2,
    
                Nκ^N   if a=κ^2,
    1/1-aκ^-2κ^N   if a<κ^2,

    where we have used
    
    ∑_n=1^mτ^n≤τ^m+1/τ-1   if τ>1,
    
                m   if τ=1,
    1/1-τ   if 0<τ<1.

    Concerning III, the assumption |f_n|=O(a^|n|) (n→∞) and Lemma <ref> imply that
    
    III≤4C∑_n=(N+1)/2^∞ a^n
            =4C/1-aa^(N+1)/2.


    Combining the above estimates, we complete the proof of (iii).




§ NUMERICAL EXPERIMENTS



In this section, we show several results of numerical experiments that compare the original MFS and MFS-QR.



 §.§ Disk


We first consider the problem (<ref>) with boundary value f(x,y)=x^2y^3.
We set parameters as ρ1 and R1.1 or R1.5.
We change N as N=2l+1 with l=1,2,…,l_max150, and compute condition numbers and approximation errors.

Figure <ref> shows the result of numerical experiments.
It can be observed that the L^∞-norms of approximation errors decay exponentially with respect to N for both the DSM and the DSM-QR.
The condition number increases exponentially for the DSM, while it is of O(1) for the DSM-QR.



 §.§ Jordan region


We next deal with the case where Ω is a Jordan region.
In the previous study <cit.>, it was pointed out that the MFS-QR reduces the ill-conditioning but cannot remove it completely when the problem region Ω is not a disk.
In this subsection, we develop an analogue of the MFS-QR for Jordan region with analytic boundary using conformal mappings.

Since Ω is a Jordan region, the Riemann mapping theorem assures the existence of a conformal map Ψ𝔹_1→Ω.
Then, the Osgood–Carathéodory theorem extends Ψ to a homeomorphism of Ω onto 𝔹_1, denoted by the same symbol Ψ.
Since ∂Ω is analytic, we can extend Ψ conformally to 𝔹_τ with some τ>1 by the Schwarz reflection principle.
As studied in <cit.>, for the MFS, if we arrange the collocation points {z_j}_j=1^N∈∂Ω and {ζ_k}_k=1^N⊂ℂ∖Ω by

    z_j   Ψ(ω^j),        j=1,2,…,N,

    ζ_k   Ψ(Rω^k),        k=1,2,…,N

with R∈(1,τ) and ωexp(2πi/N), an approximate solution uniquely exists for sufficiently large N, and an approximation error decays exponentially for analytic boundary data.
The essense of the proof is to decompose the basis function into the disk part and the perturbation part as follows:

    log|Ψ(re^iθ)-Ψ(Re^iϕ)|
        =log|Ψ(re^iθ)-Ψ(Re^iϕ)/re^iθ-Re^iϕ|+log|re^iθ-Re^iϕ|.

Concerning the DSM, the author developed mathematical theory in <cit.>. 
The similar idea plays an essential role, and the basis function is decomposed as

    -(ν(Ψ(Re^iϕ))/Ψ(re^iθ)-Ψ(Re^iϕ))
        =-(ν(Ψ(Re^iϕ))/Ψ(re^iθ)-Ψ(Re^iϕ)-ν(Re^iϕ)/re^iθ-Re^iϕ)-(ν(Re^iϕ)/re^iθ-Re^iϕ),

where ν(Re^iϕ) denotes the unit outward normal vector to the circle ∂𝔹_R at the point Re^iϕ, whereas ν(Ψ(Re^iϕ)) does the one to the Jordan curve Ψ(∂𝔹_R) at Ψ(Re^iϕ).
The first-term in the right-hand side can be regarded as a good perturbation in some sense.
The second-term in the right-hand side is the basis function for the case of disk, so it can be modified via the DSM-QR.
Hence, it is natural to imagine that the set of functions

    {-(ν(ζ_k)/Ψ(re^iθ)-ζ_k-ω^k/re^iθ-Rω^k)+ψ_k(r/ρ,θ)}_k=1^N

would be a new basis function, which removes the ill-conditioning completely.
Although there is no mathematical theory concerning the above idea, we below show several numerical experiments that exemplify the effectiveness of the above strategy.


    
    We first show the result where the conformal mapping is given by a polynomial:
    
    Ψ(z)4/5z+1/10z^5.

    We use the same boundary value f(x,y)=x^2y^3, and set parameters as R1.05, N=2l+1, and l_max500.
    
    
    Figure <ref> demonstrates results of numerical experiments.
    As in the case of disk, we can observe that the approximation error decays exponentially for both the DSM and the DSM-QR.
    Concerning the condition number, it increases exponentially for the DSM, while it is of O(1) for the DSM-QR.
    Namely, the aforementioned strategy works well.



    
    To strengthen the validity of the above idea, another numerical experiment will be performed.
    We consider the Joukowski-type transformation, defined as
    
    Ψ(z) w+1/2w,  
            w z-1/5+i/5.

    We use the same boundary value f(x,y)=x^2y^3, and set parameters as R1.1, N=2l+1, and l_max500.
    
    
    In this case, the curvature of boundary ∂Ω is very large at the right end, and it is generally difficult to obtain a highly accurate approximate solution. 
    However, as shown in Figure <ref>, DSM and DSM-QR show exponential decay of approximation error. 
    This indicates that DSM and DSM-QR are even more powerful in regions with complex geometry. 
    Moreover, even in this case, the ill-conditionality is completely eliminated in the DSM-QR, which truly demonstrates the high usefulness of the DSM-QR.





§ CONCLUDING REMARKS



This paper puts forth a new approach, designated as the DSM-QR, which draws inspiration from the MFS-QR. 
It then proceeds to undertake a comprehensive mathematical analysis of this proposed approach within the context of the disk region. 
Theorems <ref> and <ref> demonstrate that the approximate solution exists uniquely and that the approximation error decays in accordance with the expected order in the context of the MFS. 
A noteworthy outcome is the mathematical proof presented in Theorem <ref>, which demonstrates that the condition number is of O(1). 
Moreover, despite the absence of a mathematical analysis, we were able to extend the DSM-QR to the general Jordan region through the use of conformal mapping. 
In a previous study <cit.>, a method based on singular value decomposition was shown to be effective in removing ill-conditionality in star-shaped domains. 
However, the main feature of the method presented in this paper is that it is based on conformal mapping, which allows for domain shapes to be unrestricted.

Further research should be conducted in the following areas:
Firstly, the mathematical justification of the extension to Jordan regions should be considered. 
This can be achieved by applying the mathematical analysis methods of the DSM in the Jordan domain <cit.>. 
An additional avenue for exploration is the expansion of the problem region in terms of its dimensionality. 
This is a direction that has been pursued by the MFS-QR and the MFS-SVD as well, but thus far, these methods have only demonstrated the ability to remove ill-conditionality in planar regions. 
They have not yet succeeded in extending this approach to domains of greater dimensionality, such as three-dimensional regions. 
Moreover, it would be advantageous to consider extensions to problems in multiply-connected domains, even if they are in two-dimensional regions, for practical applications. 
Finally, since the MFS-QR has been developed for the Helmholtz equation <cit.>, it is crucial to investigate whether the DSM-QR can be extended to other equations as well.




  §.§.§ Acknowledgements


This work was supported by JSPS KAKENHI Grant Numbers JP18K13455, JP22K03425, JP22K18677, JP23H00086.




siam




 Paper Title: Sampling strategies for expectation values within the Herman--Kluk approximation.



Sampling strategies for expectation values within the Herman–Kluk approximation]Sampling strategies for expectation values within the Herman–Kluk approximation













1]Fabian Kröningerfabiankroninger@epfl.ch

2]Caroline Lasserclasser@ma.tum.de


1]Jiří J. L.  Vaníčekjiri.vanicek@epfl.ch


[1]Laboratory of Theoretical Physical Chemistry, Institut des Sciences et Ingénierie Chimiques, Ecole Polytechnique Fédérale de Lausanne (EPFL), CH-1015 Lausanne, Switzerland

[2] School of Computation, Information and Technology, Department of Mathematics, Technische Universität München, Germany 

When computing quantum-mechanical observables, the “curse of dimensionality” limits the naive approach that uses the quantum-mechanical wavefunction.
The semiclassical Herman–Kluk propagator mitigates this curse by employing a grid-free ansatz to evaluate the expectation values of these observables.
Here, we investigate quadrature techniques for this high-dimensional and highly oscillatory propagator.
In particular, we analyze Monte Carlo quadratures using three different initial sampling approaches. 
The first two, based either on the Husimi density or its square root, are independent of the observable whereas the third approach, which is new, incorporates the observable in the sampling to minimize the variance of the Monte Carlo integrand at the initial time. 
We prove sufficient conditions for the convergence of the Monte Carlo estimators and provide convergence error estimates.
The analytical results are validated by numerical experiments in various dimensions on a harmonic oscillator and on a Henon-Heiles potential with an increasing degree of anharmonicity.











[
    [
    August 5, 2024
==================





§ INTRODUCTION

Molecular quantum dynamics is a vibrant, interdisciplinary field of research dedicated to gaining insights into chemical and physical phenomena. To simulate it, one must solve the time-dependent Schrödinger equation 

    iϵ∂_t ψ(t,x) = (-ϵ^2/2Δ + V(x) ) ψ(t,x),   ψ(0,x) = ψ_0(x)∈ L^2(ℝ^D)

for nuclei on an electronic potential energy surface V that arises from the Born–Oppenheimer approximation <cit.>.
Here, 0<ϵ≪ 1 denotes the semiclassical parameter, Δ:= ∑_j=1^D ∂^2_x_j is the Laplacian and V:ℝ^D →ℝ is a smooth potential of sub-quadratic growth <cit.> such that the Hamiltonian Ĥ=-ϵ^2/2Δ + V is a self-adjoint operator when equipped with an appropriate domain. The unitary evolution operator

    Û_t := exp(-iĤt/ϵ)

then gives the solution to (<ref>) as ψ(t) = Û_tψ_0.

The numerical solution of the time-dependent Schrödinger equation is challenging.
In most applications, conventional grid-based integration methods are not feasible due to the large dimension D of the configuration space of a molecular system. 
In addition, the high oscillations induced by the small parameter ϵ exacerbate this curse of dimensionality.

To overcome these issues, semiclassical methods <cit.> have been developed using a priori knowledge of the solution's behaviour. 
Single-particle semiclassical methods, such as the thawed Gaussian approximation <cit.>, are limited in their applicability because some quantum effects, such as wavepacket splitting, cannot be described by these methods. 
Multi-particle methods often remain accurate and provide further qualitative insights. 
The Herman–Kluk approximation <cit.>, which refines Heller's frozen Gaussian approximation <cit.>, remains one of the most accurate multi-particle semiclassical methods.


Within the Herman–Kluk approximation, the solution of the time-dependent Schrödinger equation (<ref>) is approximated by a high-dimensional and highly oscillatory integral over the phase space ℝ^2D which has the general form

    ψ(t,x) ≈∫_ℝ^2D  a_t(z) e^iϕ_t(x,z)/ϵ dz.

The integrand is time-dependent and the complex-valued functions a_t and ϕ_t are obtained by solving ordinary differential equations.
Integral (<ref>) is motivated by the continuous superposition of time-dependent Gaussian wavepackets

    g_z(t)(x)=(Γ/π^Dϵ^D)^1/4exp(-1/2ϵ (x-q(t))^T Γ (x-q(t)) +i/ϵ  p(t)^T(x-q(t))  )
 that have a fixed, time-independent (“frozen”), symmetric, and positive-definite width matrix Γ∈ℝ^D× D, and are centered at a phase-space point z(t)=(q(t),p(t))∈ℝ^2D.
The time-dependent position and momentum parameters q(t) and p(t) obey Hamilton's equations of motion q̇=p and ṗ=-∇ V(q), whereas the lack of change in width Γ is compensated by a time-dependent prefactor.
Due to the frozen Gaussian ansatz, the function e^iϕ_t(·,z)/ϵ in (<ref>) will be of Schwartz class on the configuration space ℝ^D.


The use of the wavefunction per se is limited by the dimension of the configuration space ℝ^D.
In typical applications, where D≫ 1, further insights into a quantum system can
be obtained only by circumventing the evaluation of the wavefunction and, instead, by directly evaluating expectation values of quantum-mechanical observables. These expectation values provide information not only about position, momentum and kinetic, potential and total energies but also about the norm of the state.
For a normalized state ψ, the expectation value of a self-adjoint operator Â on L^2(ℝ^D) is given by the inner product

    ⟨ψ , Âψ⟩ = ∫_ℝ^Dψ(x) (Âψ)(x)  dx.

Conventionally, one must obtain a solution of the time-dependent Schrödinger equation (<ref>) and then compute the overlap (<ref>) using a numerical quadrature. 
Inserting approximation (<ref>) into the inner product (<ref>) leads to

    ⟨ψ(t), Âψ(t) ⟩≈∫_ℝ^4Da_t(y) a_t(z)⟨ e^iϕ_t(y)/ϵ, Â e^iϕ_t(z)/ϵ⟩ d(y,z).

This integral acts on the double phase space ℝ^2D⊗ℝ^2D≃ℝ^4D. 
As we shall see in <Ref>, the special form of ϕ_t due to the frozen Gaussian ansatz ensures the well-definedness of the inner product ⟨ e^iϕ_t(y)/ϵ, Â e^iϕ_t(z)/ϵ⟩ and significantly simplifies it. 
In particular, an additional numerical quadrature is often unnecessary, as explicit formulas are known. 
Instead of two separate Herman–Kluk integrals on ℝ^2D, the expectation value is interpreted as a weighted integral on a double phase space ℝ^4D with respect to a product measure and it can be computed with a single numerical quadrature.
The notion of double phase space also appears in other contexts of mathematical physics, for example, in the application of the Weyl propagator <cit.> or forward-backward semiclassical propagation <cit.>.



 §.§ Previous research

The Herman–Kluk propagator has been used extensively in theoretical chemistry. 
Several groups have applied further approximations to make the Herman–Kluk propagator computationally more feasible, see <cit.> for some exemplary contributions.
In <cit.>, the authors investigated sampling approaches for classical time autocorrelation functions and they proposed an observable-dependent sampling ensuring that the convergence of the estimator is independent of the dimensionality of the system and the underlying dynamics.

From a mathematical point of view, the Herman–Kluk propagator has been rigorously analyzed by several authors. 
Swart and Rousse <cit.> and Robert <cit.> proved that the Herman–Kluk propagator approximates the unitary evolution operator to the first-order of ϵ. 
An extensive overview of single- and multi-particle semiclassical methods (including the Herman–Kluk propagator) and their error analysis can be found in <cit.>.

In the mathematical literature, the Herman–Kluk approximation is sometimes referred to as the frozen Gaussian approximation.
Lu and Yang apply it to high-frequency wave propagation <cit.> and general linear strictly hyperbolic systems <cit.>.
Delgadillo et al. generalized the Herman–Kluk propagator to a gauge-invariant frozen Gaussian approximation for the Schrödinger equation with periodic potentials  <cit.>. 
In <cit.>, the numerical discretization of the Herman–Kluk wavefunction in both time and phase space was analyzed and the special form of the expectation value (<ref>) was introduced. 
Moreover, various wavefunction sampling approaches were investigated in <cit.>.
Xie and Zhou also investigated mesh-free discretization methods for wavefunctions and expectation values <cit.>. They introduced an estimator for expectation values that scales quadratically with the number of quadrature points, whereas our approach scales linearly.
Huang et al. combined the Herman–Kluk approximation with surface hopping to compute the wavefunction and expectation values in a nonadiabatic regime <cit.>.




 §.§ Main results

Owing to the frozen Gaussian ansatz, the Herman–Kluk expectation value (<ref>) can be perfectly combined with probabilistic discretization methods such as the Monte Carlo quadrature.
Common Monte Carlo approaches originate from the Husimi and “sqrt-Husimi” (absolute value of an inner product of a Gaussian wavepacket with the initial state ψ_0) densities. 
While the Husimi density is well-defined for all ψ_0 coming from L^2(ℝ^D), the sqrt-Husimi density requires a restriction on the domain of the initial states.
Because both of these densities act only on a single phase-space ℝ^2D, the quadrature points of the double phase-space
ℝ^4D are then generated by sampling both individual phase spaces either from the Husimi or sqrt-Husimi densities.
For the most important quantum-mechanical operators Â, we show that the Monte Carlo integrand based on the Husimi density has an unbounded variance while choosing the sqrt-Husimi density leads to finite values.
Moreover, we show examples in which the variance of the sqrt-Husimi approach has an exponential dependence on the spatial dimension D of the system.
The convergence of the estimators can be further improved by including operator Â in the sampling density.
We prove that such a choice minimizes the variance of the Monte Carlo integrand at the initial time.
Sampling from a probability density that incorporates the observable was also investigated for classical time autocorrelation functions <cit.>. The authors show that the convergence of the Monte Carlo estimator is independent of the system’s dimensionality and underlying dynamics. However, for Herman–Kluk expectation values we provide an example where the variance still has an exponential dependence on the dimension D. In fact, it is not uncommon for the convergence of Monte Carlo quadrature to depend on the dimension of the problem <cit.>.

Because of the inclusion of operator Â, the new sampling density may only be known up to a constant value, making the standard Monte Carlo estimator inaccurate.
To overcome this issue, we introduce self-normalizing estimators, which are based on a weighted average, whereas the standard Monte Carlo quadrature is non-weighted.
Specifically, we approximate the Herman–Kluk expectation value (<ref>) using an estimator in the form

    ∫_ℝ^4Da_t(y) a_t(z)⟨ e^iϕ_t(y)/ϵ, Â e^iϕ_t(z)/ϵ⟩ d(y,z) 
       
       ≈1/N∑_j=1^N[a_t(y_j) a_t(z_j)⟨ e^iϕ_t(y_j)/ϵ, Â e^iϕ_t(z_j)/ϵ⟩W(y_j,z_j)/ρ_1(y_j,z_j)] /∑_j=1^N W(y_j,z_j),

with a weight W=ρ_1/ρ_2, two probability densities ρ_1, ρ_2 and samples (y_j,z_j) that are independent and identically distributed with respect to ρ_2.
We show that this estimator converges to the expectation value (<ref>) as long as samples (y_j,z_j) can be generated from ρ_2. In particular, it is sufficient to know the density ρ_2 only up to a constant factor.
Moreover, we prove that the bias of the estimator vanishes as N→∞ for any choice of the weight W and that under certain conditions, the bias, variance, and mean squared error of the estimator are in the order of 𝒪(N^-1).
We investigate several choices of weight W based on the previously discussed sampling approaches. 

By combining Markov Chain Monte Carlo algorithms with self-normalizing estimators, we propose an advanced, novel algorithm for the approximation of expectation values with linear scaling improving on known approaches with quadratic scaling.



 §.§ Outline of the paper

The remainder of this paper is organized as follows. 
In the next section, we introduce the Herman–Kluk approximation and discuss its ability to compute the expectation values of observables Â. 
In <Ref>, we introduce the Monte Carlo quadrature and its crude Monte Carlo estimator as our main numerical tool for solving the high-dimensional and highly oscillatory integral (<ref>) obtained from the Herman–Kluk approximation. 
We provide sufficient conditions for the convergence of the estimator and investigate the different sampling approaches.  
In <Ref>, we extend the crude Monte Carlo estimator to a self-normalizing version and analyze its properties in detail.
The general algorithm that computes (<ref>) within the Herman–Kluk approximation is summarized in <Ref>. We complete the paper with numerical examples in several dimensions in <Ref> and a short discussion in <Ref>.



 §.§ Notation

Throughout this paper, we denote the space of Schwartz functions on the configuration space ℝ^D as 𝒮(ℝ^D).
These are smooth functions that decrease rapidly together with all their derivatives.
Moreover, we denote a point in the double phase space ℝ^4D by  w=(y,z) ∈ℝ^4D, consisting of y and z each originating from a single phase space ℝ^2D.

We restrict ourselves to the linear operators Â=Id (representing the norm squared), powers q̂^n and p̂^n for n∈ℕ of position and momentum, kinetic energy T(p̂), potential energy V(q̂) and total energy Ĥ=T(p̂) + V(q̂) as they are the most interesting operators in quantum mechanics. We denote the D-dimensional identity matrix by Id_D.

For a random variable X distributed with respect to probability density ρ, the expectation value and variance are denoted by

    𝔼[X]= ∫ xρ(x)  dx   and  Var[X] = ∫| x - 𝔼[X] |^2 ρ(x)  dx

and for an estimator 𝕏 of 𝔼[X], the bias is defined as

    Bias(𝕏) = |𝔼[𝕏] - 𝔼[X] |.





§ THE HERMAN–KLUK PROPAGATOR

Similar to Heller <cit.>, Herman and Kluk <cit.> argued that a quantum system cannot be described accurately by a single Gaussian wavepacket. Instead, they proposed using multiple Gaussians to solve the time-dependent Schrödinger equation (<ref>).



 §.§ Integral representation

The inversion formula of the Fourier-Bros-Iagolnitzer (FBI) transform provides a tool to represent a wavefunction in terms of a phase-space integral of Gaussians.

For z∈ℝ^2D, the FBI transform <cit.> is defined as

    𝒯: 𝒮(ℝ^D) →𝒮(ℝ^2D),    (𝒯ψ)(z) := (2πϵ)^-D/2⟨ g_z, ψ⟩,

where the Gaussian g_z is parameterized as in (<ref>).
The FBI transform maps 𝒮(ℝ^D) continuously into 𝒮(ℝ^2D) (see <cit.>)
and there is the inversion formula <cit.>
    
    ψ(x) = (2πϵ)^-D∫_ℝ^2D⟨ g_z, ψ⟩ g_z(x)  dz

for all ψ∈𝒮(ℝ^D). The FBI transform can be extended to an isometry from L^2(ℝ^D) to L^2(ℝ^2D) (see <cit.>) and, because the Schwartz space 𝒮 is dense in L^2, the inversion formula (<ref>) extends to square-integrable functions.

Based on the continuous superposition (<ref>), one writes 

    (Û_tψ_0)(x) = (2πϵ)^-D∫_ℝ^2D⟨ g_z, ψ_0 ⟩ (Û_t g_z)(x) dz,

which motivates the Herman–Kluk propagator

    (Û_t^ HKψ_0)(x)    :=(2πϵ)^-D∫_ℝ^2D⟨ g_z, ψ_0 ⟩  R_t(z) e^iS_t(z)/ϵ  g_z(t)(x)   dz,

which is in the form of (<ref>) with


    a_t(z)   = (2πϵ)^-DR_t(z) ⟨ g_z,ψ_0 ⟩
    and  ϕ_t(x,z)    = i/2 (x-q(t))^T Γ (x-q(t)) + p(t)^T(x-q(t)) + S_t(z).

In particular, since Γ is real symmetric and positive-definite, we have e^iϕ_t(·, z)∈ L^2(ℝ^D) for any fixed z∈ℝ^2D.
Here, z(t)=(q(t),p(t))∈ℝ^2D is a classical trajectory in the phase space
associated with the Hamiltonian function h(q,p)= | p |^2/2+V(q) and S_t(z)∈ℝ is the classical action along this trajectory. 
The Herman–Kluk prefactor R_t(z)∈ℂ is defined as

    R_t(z) :=2^-D/2( M_qq + Γ^-1  M_ppΓ -i M_qpΓ + i Γ^-1 M_pq)^1/2,

where the D× D matrices M_αβ, with α, β∈{q,p},  are the blocks of the stability matrix M(t) := ∂ z(t)/∂ z, and the branch of the square root of (<ref>) is chosen continuously with respect to time.
For all phase-space points z∈ℝ^2D, the parameters z(t)=(q(t),p(t)), S_t, and M(t) obey the ordinary differential equations (ODEs)

    ż(t)    = J ·∇ h(z(t)), 
    Ṡ_t(z)    = 1/2| p(t) |^2 - V(q(t)), 
    Ṁ(t)     = J ·Hess  h(z(t)) · M(t)

with the symplectic matrix J: = [     0  Id_D; -Id_D     0 ]∈ℝ^2D, the Hamiltonian function

    h: ℝ^2D→ℝ, (q,p) ↦1/2| p |^2 + V(q)
 and initial conditions z(0)=z,  S_0(z) = 0 and M(0) = Id_D. 

    In the mathematical literature, spherical Gaussians (i. e., Gaussians with Γ=_D) are often considered <cit.>. 
    In chemical physics simulations, the width matrix Γ is usually chosen such that the Gaussian is an eigenstate of the Hamiltonian Ĥ=T+Ṽ, where the potential Ṽ is a harmonic approximation of the real potential V at a reference position q_0 (i. e. Ṽ is equal to the second order Taylor expansion of V around q_0) <cit.>. 
    Hence, we keep width matrices real symmetric and positive-definite to have flexible initial data for chemical applications.


Swart and Rousse <cit.>, and Robert <cit.> rigorously justified the Herman–Kluk propagator (see also <cit.>). More recently, discretizations with respect to time and phase space for numerical calculations have been analyzed <cit.>.
In particular, the Herman–Kluk propagator has the following properties:


  * For a sub-quadratic potential V, the Herman–Kluk propagator Û_t^ HK approximates the unitary evolution operator (<ref>)
with an error of the order of ϵ <cit.>, that is,

    sup_t∈[0,τ]||Û_t - Û_t^ HK||_L^2→ L^2≤ C(τ) ϵ,

where τ>0 is a fixed time, and the constant C(τ) is independent of ϵ and depends only on τ. 
    
  * For potentials V that are at most quadratic polynomials, the Herman–Kluk approximation is exact <cit.>.
    
  * For a sub-quadratic potential V, the Herman–Kluk prefactor R_t is bounded from above and from below <cit.>. For a fixed time τ>0, there exist constants c_1(τ)> 0 and c_2(τ)>0 such that
    
    c_1(τ) ≤| R_t(z) |≤ c_2(τ)

    for all z∈ℝ^2D and t∈[0,τ].
 



 §.§ The representation for expectation values

In practice, due to dimension D≫ 1, one is often more interested in the expectation values of quantum-mechanical observables than the wavefunction itself.
These expectation values provide information about the norm and measurable quantities such as the positions, momenta, and energies, and can be used for comparison with reference solutions (if they are available).

The expectation value of a self-adjoint operator Â on L^2(ℝ^D) (or some subspace of it) in the normalized state ψ is given by the inner product

    ⟨ψ, Âψ⟩ = ∫_ℝ^Dψ(x) (Âψ)(x) dx.


    Let ψ(t) be the solution to the time-dependent Schrödinger equation (<ref>) or some approximation of it and let ρ̂(t) = |ψ(t) ⟩⟨ψ(t)| (in Dirac notation, see <cit.>) be the orthogonal projection onto the span of ψ(t). The expectation value of a quantum mechanical operator Â can be written as (see <cit.>)
    
    ⟨ψ(t), Âψ(t) ⟩ = Tr[ρ̂(t) Â].

    In the chemical literature, often the more general time-correlation function
    
    C_B̂Â(t) = Tr[B̂(t)Â]

    of operators Â and B̂(t) is of interest <cit.>. To simplify the exposition, our focus in this paper is on expectation values (<ref>), but we believe that our results can be extended to the more general case (<ref>).


A conventional calculation of the expectation value (<ref>) requires finding an (approximate) solution ψ(t) of the time-dependent Schrödinger equation (<ref>) and evaluating the inner product using an additional numerical quadrature. 
This is clearly limited to low-dimensional situations.
However, the Herman–Kluk propagator provides a convenient way of computing the expectation values without evaluating the full Herman–Kluk wavefunction Û_t^ HKψ_0 for some normalized ψ_0∈ L^2(ℝ^D). 
By considering a double phase space ℝ^2D⊗ℝ^2D≃ℝ^4D, expectation values of an operator Â within the Herman–Kluk setting can be reformulated as <cit.>

    ⟨Â⟩_t    :=  ∫_ℝ^4D  f_0(w) Φ_t(w)  O_t[Â](w) dw

with a new variable w=(y,z)∈ℝ^4D consisting of y,z∈ℝ^2D, each coming from a single phase space ℝ^2D. 
Here, the integrand is split into an initial-state-dependent and operator- and time-independent factor

    f_0:ℝ^4D→ℂ,   (y,z)    ↦ (2πϵ)^-2D⟨ψ_0 , g_y ⟩⟨  g_z, ψ_0 ⟩,

which is identical to f_0(y,z) = (2πϵ)^ - D(𝒯ψ_0)(y) (𝒯ψ_0)(z),
a time-dependent, operator- and  initial-state-independent factor 

    Φ_t: ℝ^4D→ℂ,    (y,z) ↦R_t(y) R_t(z) exp[i(S_t(z)-S_t(y))/ϵ],

and a time- and operator-dependent and initial-state-independent factor

    O_t[Â]: ℝ^4D→ℂ,    (y,z) ↦⟨ g_y(t) , Â  g_z(t)⟩.

To improve readability, the notation omits the ϵ-dependence of the integrand. 
We introduce the following 

    Let ψ_0 be a normalized initial state coming from  𝒮(ℝ^D).

This assumption is not too restrictive as in most applications the initial state ψ_0 is a coherent ground state of a harmonic oscillator or a Hermite function that belong to 𝒮(ℝ^D). Therefore, we take <Ref> from now on for granted. 

In particular, Schwartz functions are contained in the domains of operators Â we will consider, and (due to the assumed smoothness and growth conditions on the potential function V), for all times t∈ℝ the time-evolved state ψ(t) =Û^ HK_tψ_0 is a Schwartz function <cit.>.

We explore the characteristics of the integrand in (<ref>), which will be useful in later sections.
 
Let τ>0 be a fixed time and let Â=Id, q̂^n_j, p̂^n_j, or Ĥ for  n ∈ℕ, j ∈{1, … , D}.


    
  * If <Ref> is true, then f_0∈𝒮(ℝ^4D).
    
  * There are τ-dependent constants c_1(τ), c_2(τ)>0 such that
    
    c_1(τ) ≤|Φ_t(w)|≤ c_2(τ)

    for all t∈ [0,τ] and w∈ℝ^4D.
    
  * For all t∈[0,τ], the absolute square | O_t[Â](y,z)|^2 of the operator-dependent part
    grows at most polynomially as | y(t) |, | z(t) |→∞.


  


    
  * If ψ_0∈𝒮(ℝ^D), then the FBI transform satisfies 𝒯ψ_0 ∈𝒮(ℝ^2D) and f_0∈𝒮(ℝ^4D) as a tensor product of two Schwartz functions.
 
  * Since S_t∈ℝ, it holds
    
    |Φ_t(y,z) |  = | R_t(z) R_t(y)|.

    The Herman–Kluk prefactor R_t is bounded from below and from above. Hence, (<ref>) holds true.

  * By the Cauchy-Schwarz inequality, it holds that
   
    | O_t[Â](y,z)|^2 = |⟨ g_y(t), Â g_z(t)⟩|^2 ≤||Âg_z(t)||^2.


    
  * For Â= Id and Â=q̂_j^n, we obtain the bound

    | O_t[Â](y,z)|^2    ≤∫_ℝ^D x^2n_j exp(-1/ϵ(x-q(t))^T Γ (x-q(t)) )  dx
        
        = Pol(q(t)),

because the moments of a Gaussian depend polynomially on its mean. See Appendix <ref> for a discussion of the moments of Gaussians.

  * For a sub-quadratic potential V, there is a constant c>0 such that
    | V(x) |≤| V(0) |  + |∇ V(0)^T x | + c ∑_j,k=1^D | x_j x_k |.

Hence, there is a polynomial Pol: ℝ^D →ℝ such that V(x)^2 ≤Pol(x) for all x∈ℝ^D. We obtain

    | O_t[Â](y,z)|^2    ≤∫_ℝ^D V(x)^2 exp(-1/ϵ(x-q(t))^T Γ (x-q(t)) )  dx
        
       ≤∫_ℝ^DPol(x) exp(-1/ϵ(x-q(t))^T Γ (x-q(t)) )  dx.

The right-hand side of the inequality depends polynomially on the mean q(t). We refer again to Appendix <ref> for a discussion of the moments of a Gaussian. 

  * By changing to the momentum space, similar to the n-th order position operator q̂_j^n, | O_t[Â](y,z)|^2 grows at most polynomially for Â=p̂_j^n as | y(t) || z(t) |→∞. In particular, this is true for the kinetic energy operator Â=T.

  * Using the triangle inequality, the result follows for the Hamiltonian Ĥ=T+V.




For single-particle Gaussian-based methods, the error bounds for both wavefunctions and observables were established (see <cit.> for the semiclassical thawed Gaussian approximation and <cit.> for the variational Gaussian approximation). 
For both methods, the accuracy of the expectation values was higher than the wavefunction accuracy measured by the standard L^2 Hilbert space norm.
However, for the Herman–Kluk propagator, error bounds for observables have not yet been analyzed in detail. 

We transfer the propagator's norm accuracy to the expectation values in a straightforward manner.
 
    Let ψ(t) = Û_tψ_0 be the solution to the time-dependent Schrödinger equation (<ref>) and let <Ref> hold.
    For a bounded, self-adjoint operator Â, the integral in (<ref>) approximates the expectation value ⟨ψ(t) , Âψ(t) ⟩ in the order of ϵ. That is, for a fixed τ>0 there is a constant c(τ) such that
    
    sup_t∈[0,τ]|⟨Â⟩_t -  ⟨ψ(t), Âψ(t) ⟩|≤ c(τ) ϵ.

     In the case of at most quadratic polynomial potentials V, the Herman–Kluk approximation becomes exact, i. e.,  ⟨Â⟩_t = ⟨ψ(t), Âψ(t) ⟩


Let u(t) = Û^ HK_t ψ_0 be the Herman–Kluk wavefunction (<ref>) and recall that ⟨Â⟩_t = ⟨ u(t), Âu(t) ⟩. Using the triangle inequality, we obtain

    |⟨Â⟩_t - ⟨ψ(t), Âψ(t) ⟩|   = |⟨ u(t) - ψ(t), Â u(t) ⟩ + ⟨Âψ(t), u(t) - ψ(t) ⟩|
       ≤|| u(t) - ψ(t) ||·||Â||· (|| u(t)|| + ||ψ(t)||)
            
       ≤ c(τ) ϵ,

where the last step follows from the norm accuracy of the wavefunction, the boundedness of Â and the norm bound of the Herman–Kluk wavefunction (see <cit.>).


The error bound in <Ref> might not be sharp. The proof presented here is naive since averaging effects of Gaussian wavepackets, used in the error bounds of single-particle Gaussian-based methods <cit.>, are not employed. It is an open question whether these beneficial averaging effects carry over to the Herman–Kluk approximation.



 §.§ Computational tasks

The inner product (<ref>) that defines the expectation value is interpreted as an integral on the double phase space ℝ^2D⊗ℝ^2D≃ℝ^4D and it can be computed with a single numerical quadrature. 
The operator Â appears in the inner product (<ref>) of two frozen Gaussians with different centres, which either simplifies the numerical quadrature or even makes it dispensable if explicit formulas are known.

To evaluate the double phase-space integral (<ref>) numerically, we have the following two tasks:

    
  * The integral over ℝ^4D has to be discretized with some quadrature rule.
    
  * The integrand must be obtained from the ordinary differential equations (<ref>), (<ref>), and (<ref>).
 
The ODEs can be solved using a symplectic integration method to preserve the symplectic structure of the underlying classical Hamiltonian system <cit.>. 
The computational bottleneck is the discretization of the double phase space ℝ^4D, whose dimension grows linearly with the number of dimensions D. 
Because of the high dimensionality and because the frozen Gaussian wavepackets can be combined with probabilistic sampling techniques, standard approaches resort to Monte-Carlo or quasi Monte-Carlo quadrature <cit.>. 
In <cit.>, the authors considered Monte Carlo estimators for Herman–Kluk expectation values that scale quadratically with the number of quadrature points.
However, it is possible to construct an estimator that scales linearly with the number of quadrature points due to the interpretation ℝ^2D⊗ℝ^2D≃ℝ^4D as we will show here.





§ PROBABILISTIC SPACE DISCRETIZATION

We aim to approximate the integral 

    ⟨Â⟩_t = ∫_ℝ^4Df_0(w) Φ_t(w) O_t[Â](w)  dw ≈ A_N(t)

by employing a probabilistic discretization technique <cit.> involving N quadrature points.
In the spirit of importance sampling, we introduce an arbitrary probability density ρ > 0 on ℝ^4D. Although the density ρ may vanish on Lebesgue null sets, we omit to mention this for simplicity in the present and upcoming sections.

For a probability density ρ>0 on ℝ^4D, we call
    
    A_N(t) = 1/N∑_j=1^N  f_0(w_j) /ρ(w_j)Φ_t(w_j)O_t[Â](w_j)

the crude Monte Carlo estimator of ⟨Â⟩_t at time t for which the samples w_j∈ℝ^4D are independent and identically distributed with respect to ρ. 

This estimator is unbiased, and the strong law of large numbers <cit.> provides the almost sure convergence 

    A_N(t) →⟨Â⟩_t,    N→∞,

whenever the function f_0Φ_t O_t[Â]: ℝ^4D↦ℂ is integrable.

In particular, the convergence is independent of the chosen probability density ρ. 

    Let n∈ℕ, j ∈{1,…, D} and Â= Id, q̂^n_j, p̂^n_j or Ĥ. 
    Let τ>0 be a fixed time. 
    For all times t∈ [0,τ], <Ref> is sufficient for the almost sure convergence of the crude Monte Carlo estimator A_N(t), as defined in (<ref>).



    Recall f_0, Φ_t and O_t[Â] as defined in (<ref>), (<ref>) and (<ref>). 
    By <Ref>, f_0∈𝒮(ℝ^4D) and there is a τ-dependent constant c(τ)>0 such that 
    
    |Φ_t(y,z) | = | R_t(y) R_t(z) |≤ c(τ)

    for all (y,z)∈ℝ^4D and t∈[0,τ], and
    
    | O_t[Â](y,z)| =|⟨ g_y(t), Âg_z(t)⟩|

    grows at most polynomially as | y(t) |, | z(t) |→∞. We deduce
    
    ∫_ℝ^4D| f_0(w) Φ_t(w) O_t[Â](w)  | dw ≤ c(τ)∫_ℝ^4D| f_0(w) O_t[Â](w)| dw < ∞,

    since f_0 ∈𝒮(ℝ^4D).


The accuracy of the crude Monte Carlo estimator (<ref>) is given by the mean squared error

    𝔼[| A_N(t) - ⟨Â⟩_t |^2] = 𝕍_t[Â, ψ_0, ρ]/N,

whenever the variance <cit.>

    𝕍_t[Â, ψ_0, ρ]    = ∫_ℝ^4D|f_0(w) Φ_t(w)  O_t[Â](w)/ρ(w) - ⟨Â⟩_t |^2 ρ(w) dw
         
       =∫_ℝ^4D1/ρ(w)| f_0(w) Φ_t(w)  O_t[Â](w) |^2  dw - |⟨Â⟩_t |^2

is well-defined. 
To minimize the computational cost, i. e., to obtain the most accurate result with a given number of samples N, one should choose the probability density ρ such that the above variance is minimized. 
To do so, one could try to minimize the variance at the initial time t=0 when the auxiliary function Φ_t defined in (<ref>) satisfies Φ_0 =1. 

In the remainder of this section, we analyse several sampling approaches in terms of their variance.



 §.§ Husimi and sqrt-Husimi sampling

There are an infinite number of possibilities for choosing the sampling density ρ.
In this section, we introduce the Husimi and sqrt-Husimi sampling approaches. 
The Husimi function is a commonly used sampling method in the context of the Herman–Kluk approximation <cit.>, because for time-correlation functions of the form (<ref>) and for wavepacket autocorrelation functions ⟨ψ_0, ψ(t) ⟩ the integrand is quadratic in the initial state ψ_0. For the time propagation of the wavefunction (<ref>) itself, the sqrt-Husimi approach was originally used in <cit.>, first analyzed in <cit.>, and a systematic comparison with the Husimi approach is given in <cit.>.


The Husimi function ρ_H of an initial state ψ_0 is defined as <cit.> 

    ρ_ H(z)     := |  (𝒯ψ_0)(z)|^2
        
       =(2πϵ)^-D|⟨ g_z , ψ_0 ⟩|^2.

  Under <Ref>, we call
    
    ρ_(z)    := (2πϵ)^-D/κ_|⟨ψ_0 , g_z ⟩|,

the sqrt-Husimi density with normalizing constant 

    κ_ :=  (2πϵ)^-D∫_ℝ^2D|⟨ψ_0 , g_ζ⟩| dζ.
  

By construction, ρ_ H is a probability density on a single phase space ℝ^2D for all initial data ψ_0 coming from L^2(ℝ^D) while <Ref> is sufficient for ρ_ to be well-defined. 
Using the Husimi density on each phase space leads to the crude Monte Carlo estimator

    A_N(t) = 1/N∑_j=1^N  f_0(w_j) /ρ^ dbl_ H(w_j)Φ_t(w_j)O_t[Â](w_j) Case H

with the shorthand notation ρ^ dbl_ H:= ρ_ H⊗ρ_ H and with independent w_j∈ℝ^4D distributed with respect to ρ^ dbl_ H.
By the definition f_0(w)= (2πϵ)^-2D⟨ψ_0, g_y ⟩⟨ g_z , ψ_0 ⟩ (see (<ref>)) and the Husimi function (<ref>), the variance (<ref>) of the estimator (<ref>) reduces to

    𝕍_t[Â, ψ_0, ρ^ dbl_ H  ]    =∫_ℝ^4D| f_0(w) |^2/ρ^ dbl_ H(w)|Φ_t(w)  O_t[Â](w) |^2  dw - |⟨Â⟩_t |^2
        
       =(2πϵ)^-2D∫_ℝ^4D|Φ_t(w)  O_t[Â](w) |^2  dw - |⟨Â⟩_t |^2.


Employing ρ_ instead of ρ_ H on each phase space, we obtain the estimator

    A_N(t) = 1/N∑_j=1^N  f_0(w_j) /ρ^ dbl_(w_j)Φ_t(w_j)O_t[Â](w_j) Case sqrt-H

where ρ^ dbl_:= ρ_⊗ρ_ and independent w_j∼ρ^ dbl_.
Here, the variance (<ref>) of the estimator (<ref>) reduces to

    𝕍_t[Â, ψ_0, ρ^ dbl_]    =  ∫_ℝ^4D| f_0(w) |^2/ρ^ dbl_(w)|Φ_t(w)  O_t[Â](w) |^2  dw - |⟨Â⟩_t |^2
        
       =κ_^2 ∫_ℝ^4D|  f_0(w)||Φ_t(w) |^2 | O_t[Â](w)|^2  dw - |⟨Â⟩_t |^2.


We obtain

Let τ>0 be a fixed time, n∈ℕ, j ∈{1,…, D} and Â=Id, q̂^n_j, p̂^n_j, T, V or Ĥ. For all times t∈[0,τ]

    
  * the variance (<ref>) of the Husimi approach is finite if and only if Â=0; and
    
  * the variance (<ref>) of the sqrt-Husimi approach is finite if <Ref> is satisfied.
  




    
  * For the Husimi approach, it is enough to analyze the integral

    ∫_ℝ^4D|Φ_t(w) O_t[Â](w) |^2   dw

since |⟨Â⟩_t|^2 < ∞. By <Ref> there are τ-dependent constants c_1, c_2>0 such that
    
    c_1 ∫_ℝ^4D| O_t[Â](w) |^2  dw ≤∫_ℝ^4D|Φ_t(w) O_t[Â](w) |^2  dw ≤ c_2 ∫_ℝ^4D| O_t[Â](w) |^2  dw.

Therefore, the variance 𝕍_t[Â, ψ_0, ρ_ H^ dbl] is finite if and only if | O_t[Â](w)|^2 is integrable.
Because the phase-space points y(t) and z(t) are solutions to Hamiltonian systems, the simplecticity of the classical flows reduces the time-dependent integral to a time-independent one, that is,

    ∫_ℝ^4D| O_t[Â](y,z) |^2  d(y,z)    = ∫_ℝ^4D|⟨ g_y(t), Â g_z(t)⟩|^2  d(y,z) 
        
       = ∫_ℝ^4D|⟨ g_y, Â g_z⟩|^2  d(y,z)
        
        = ∫_ℝ^2D||Âg_z ||^2_L^2(ℝ^D) dz,

where we also used the isometry of the FBI transform in the last step.

    
  * If Â=0, the integrals vanish and, therefore, the variance (<ref>).

  * Let Â=Id or q̂_j^n. In that case, |Â g_z |^2 is independent of the phase-space variable p and by the non-negativity, Fubini's theorem leads to 

    ∫_ℝ^4D| O_t[Â](y,z) |^2  d(y,z) 
        
       =(Γ/π^Dϵ^D)^1/2∫_ℝ^D 1 dp ∫_ℝ^D x_j^2n∫_ℝ^Dexp(-1/ϵ(x-q)^T Γ (x-q)  ) dq dx 
          
        = ∫_ℝ^D 1 dp ∫_ℝ^D  x_j^2n dx.

Hence, the variance is unbounded.

  * With a similar argument, | O_t[Â](w)|^2 is not integrable for Â=V when V≠ 0. 

  * By changing to the momentum representation, | O_t[Â](y,z) |^2 is not integrable for Â=p̂_j^n. In particular, it is not integrable for the kinetic energy operator Â=T.

  * For the Hamiltonian Ĥ, we first consider the Hessian of g_z, parameterized as in (<ref>), with respect to the spatial coordinate x∈ℝ^D; 

    Hess g_z(x) = -g_z(x)/ϵ^2[ ϵΓ - (Γ (x-q) - ip)(Γ (x-q) - ip)^T ].

For the Hamiltonian Ĥ=-ϵ^2Δ/2 + V applied to the Gaussian g_z we then obtain

    Ĥg_z(x)    = V(x) g_z(x) - ϵ^2/2Tr(Hess g_z(x))
        
        = g_z(x) [ V(x)  + ϵ/2Tr(Γ) - 1/2( Γ (x-q) - ip )^T( Γ (x-q) - ip ) ].

Using the estimate | a+ib|^2 ≥ b^2, a,b ∈ℝ, we obtain the lower bound

    ∫_ℝ^2D||Ĥg_z ||^2_L^2 dz 
        
       ≥(Γ/π^D ϵ^D)^1/2∫_ℝ^3D (p^T Γ (x-q))^2 exp( -1/ϵ (x-q)^T Γ (x-q) ) d(x,q,p)
        
        = (Γ/π^D ϵ^D)^1/2∫_ℝ^3D (p^T Γ x)^2 exp( -1/ϵ x^T Γ x ) d(x,q,p)

where the right-hand side is infinite for any potential V.

This proves the statement for the variance (<ref>) of the Husimi approach.

  * The proof of the finite variance (<ref>) of the sqrt-Husimi approach analogously follows the proof of <Ref>.


We give some simple examples that illustrate <Ref>.


    
  * For the Husimi approach, we recall equality (<ref>) and consider Â=. The integrals reduce to 
    
    ∫_ℝ^4D|⟨ g_y, g_z⟩|^2  d(y,z)    = ∫_ℝ^4Dexp(-1/2ϵ (z-y)^T  [    Γ    0;    0 Γ^-1 ]  (z-y) ) d(y,z)
            
        = (2πϵ)^2D∫_ℝ^2D 1  dy,

     which clearly diverges.
     
  * For z_0=(q_0,p_0)∈ℝ^2D, let ψ_0=g_z_0.
Then ρ_ is a Gaussian 

    ρ_(z) = (4πϵ)^-Dexp(-1/4ϵ (z-z_0)^T  Σ_0  (z-z_0))

on a single phase space ℝ^2D with width matrix Σ_0 = diag(Γ, Γ^-1)∈ℝ^2D × 2D.

    
  * For the identity operator Â=Id, the variance is given by

    𝕍_0[, g_z_0, ρ^ dbl_]    = 4^D   ∫_ℝ^4D| f_0(w) || O_0[](w)|^2  dw - |⟨⟩_0 |^2
        
        = (16/5)^D -1.


  * Considering the position operator Â = q̂_j for some j∈{1,2,…, D} and assuming that the initial width matrix is diagonal, i. e., Γ = diag(γ_1, γ_2, … , γ_D), γ_j>0, it holds that

    𝕍_0[q̂_j, g_z_0, ρ^ dbl_]
           = 1/4(16/5)^D( 4q_0,j^2 + 24/5ϵγ^-1_j  ) - q_0,j^2,

where the subscript j denotes the j-th component of a vector.

 The derivations can be found in Appendix <ref>.




The Husimi approach has the disadvantage of an unbounded variance (<ref>) of the estimator (<ref>). In contrast, <Ref> provides a sufficient condition to have a finite variance (<ref>) for the sqrt-Husimi approach.

<Ref> shows an unfortunate exponential dependence of the variance of the sqrt-Husimi approach on the dimension D. 
Moreover, for the position operator, the exponential D-dependent part of the variance is multiplied by a polynomial depending on the initial position, the semiclassical parameter ϵ, and the width Γ. However, it is robust in the semiclassical limit ϵ→ 0. For applications with small initial values of γ_j, the included γ_j^-1 could lead to extensive calculations.

We conclude this section by stating that even if the Husimi approach is applicable (i. e., one has the existence of the first moment), one can and should directly use the sqrt-Husimi approach, as the finite variance will result in a faster convergence compared to the Husimi approach. This was also observed for the Herman–Kluk wavefunction in <cit.> and our numerical examples confirm the theoretical results for the expectation values.



 §.§ Incorporating the operator and a variational problem

In the previous section, we have shown that the Monte Carlo integrand of the sqrt-Husimi approach leads to finite variances. However, there can be an exponential dependence of the variance on the dimension D leading to expensive calculations when realistic molecular systems are investigated.
This calls for improvements in the approach.
In this section, we present a generalized approach that includes the operator Â in the sampling density.

Let <Ref> be true and let Â≠ 0.
The optimal sampling density that minimizes the variance (<ref>) at the initial time t=0 is given by

    ρ_ opt(w)=1/κ_ opt| f_0(w) O_0[Â](w) |

with  κ_ opt =∫_ℝ^4D| f_0(ξ) O_0[Â](ξ) |  dξ ensuring the normalization.
At the initial time, the variance satisfies

    𝕍_0[Â, ψ_0, ρ_ opt] = κ_ opt^2 - |⟨Â⟩_0 |^2.



First, if Â = 0 then 𝕍_0[Â, ψ_0, ρ]=0 for any density ρ.
Therefore, let Â≠ 0. Then, ρ_ opt≠ 0 and by <Ref> and <Ref>, the density is well-defined.
We show that ρ_ opt minimizes the integral

    F(ρ)    = ∫_ℝ^4D f(w,ρ(w)) dw,
        
     f(w,ρ(w))    = |  f_0(w) O_0[Â](w) |^2/ρ(w) - |⟨Â⟩_0 |^2ρ(w)

under the constraints ρ>0 and ∫_ℝ^4Dρ(w) dw=1.
Let f=f+λ g with g(ρ)=ρ and λ∈ℝ. Then, ρ_ opt is a solution to the differential equation

    0    = ∂/∂ρf(ρ) 
            = -|  f_0 O_0[Â] |^2/ρ^2 - |⟨Â⟩_0 |^2+ λ

when λ = |⟨Â⟩_0 |^2+κ_ opt^2 >0.
Moreover, for a,b ≥ 0 and λ>0 the real maps

    ℝ_>0   →ℝ,    x  ↦a/x-bx,   and
    ℝ_>0   →ℝ_>0,    x  ↦λ x,

are convex. Then, by <cit.>, ρ_ opt minimizes F(ρ) under the constraints ρ>0 and ∫_ℝ^4Dρ(w) dw=1.
The variance 𝕍_0[Â, ψ_0, ρ_ opt] immediately follows  by inserting ρ_ opt into (<ref>).

Now, with ρ_ opt defined in (<ref>), the crude Monte Carlo estimator for ⟨Â⟩_t reads

    A_N(t) = 1/N∑_j=1^N  f_0(w_j) /ρ_ opt(w_j)Φ_t(w_j)O_t[Â](w_j) Case opt

with independent samples w_j∼ρ_ opt. 
Sampling from a general ρ_ opt can be done using a Metropolis–Hastings algorithm such as the Hamiltonian Monte Carlo algorithm described in Appendix <ref>.
The time dependence of the variance of the estimator (<ref>) is given by

    𝕍_t[Â, ψ_0, ρ_ opt] = κ_ opt∫_ℝ^4D| f_0(w) ||Φ_t(w) |^2 | O_t[Â](w) |^2/| O_0[Â](w) | dw - |⟨Â⟩_t |^2.

Compared with the variances (<ref>) and (<ref>) of the previous two approaches, the variance (<ref>) now includes | O_0[Â]| in the denominator, resulting in a reduced value.
Because of the normalization constant κ_ opt, the variance may depend on the dimension D and various parameters such as the initial position, momentum, and width.
In a simple example, we compare the variance (<ref>) of the optimal approach with that of the sqrt-Husimi approach (<ref>).

Let ψ_0 = g_z_0 be a Gaussian wavepacket with an initial phase-space point z_0=(q_0,p_0)∈ℝ^2D and let Â=Id. Then ρ_ opt is a Gaussian 

    ρ_ opt(y,z) = (2πϵ)^2D(3/4)^D exp(-1/4ϵ[  y-z_0; z- z_0 ]^T [  2Σ_0  -Σ_0; - Σ_0  2Σ_0 ][  y-z_0; z- z_0 ])

on the double phase space ℝ^4D with Σ_0=diag(Γ, Γ^-1)∈ℝ^2D × 2D, κ_ opt = (4/3)^D and 

    𝕍_0[, g_z_0, ρ_ opt]= (4/3)^2D-1 =(16/9)^D-1 < 2^D-1.

The sqrt-Husimi approach provided 𝕍_0[, g_z_0, ρ_^dbl ] = (16/5 )^D-1 > 3^D-1.

In conclusion, for a given number of quadrature points, the most accurate approximation of the integral (<ref>) can be obtained by including the operator Â in the sampling density. The density ρ_ opt is the optimal choice that minimizes the variance (<ref>) at the initial time t=0. However, for most choices of Â, the density ρ_ opt is only known up to a constant and the crude Monte Carlo estimator (<ref>) will not be applicable as the next example shows.

    The normalization constant of ρ_ opt is given by
    
    κ_ opt   = (2πϵ)^-2D∫_ℝ^4D|⟨ψ_0, g_z ⟩⟨ g_y, ψ_0 ⟩||⟨ g_y, Âg_z ⟩| d(y,z).

    In the case of Â=V with a general potential V and a more complex initial state ψ_0, the explicit value of κ_ opt may not be derivable through a straightforward calculation.
    Therefore, ρ_ opt is only known up to a constant.




§ WEIGHTED IMPORTANCE SAMPLING

To use the optimal sampling approach, we extend the initial crude Monte Carlo estimator (<ref>) to a self-normalizing version based on the general idea of including an additional weight  (see, for example, <cit.> or <cit.>).
We will first motivate the estimator in a general setting, then analyze some of its properties, and afterwards compare different weighting approaches.

Let ρ_1, ρ_2 > 0 be two densities on the double phase space ℝ^4D and define the weight W:=ρ_1/ρ_2 > 0. 
The expectation value of an operator Â within the Herman–Kluk approximation defined in (<ref>) can be written as the quotient

    ⟨Â⟩_t    = ∫_ℝ^4D f_0(w) Φ_t(w) O_t[Â](w)  dw
        
        = .∫_ℝ^4Df_0(w)/ρ_1(w)Φ_t(w) O_t[Â](w) W(w) ρ_2(w)  dw/∫_ℝ^4D W(w)ρ_2(w)  dw,

since

    ∫_ℝ^4D W(w) ρ_2(w)  dw = ∫_ℝ^4Dρ_1(w)  dw = 1.

By linearity, the weight W can be replaced by κ W for some constant κ≠ 0.


Let ρ_1>0 be a known density and let ρ_2>0 be the desired sampling density, both acting on the double phase space ℝ^4D.
Let w_1,w_2, … , w_N∈ℝ^4D be independent and identically distributed with respect to ρ_2. 
We define the weighted importance sampling estimator of ⟨Â⟩_t as

    A_N^ W(t)= ∑_j=1^N g_t(w_j)W(w_j)/∑_j=1^N W(w_j)

with the weight W=ρ_1/ρ_2 > 0 and the function

    g_t(w)=  f_0(w)/ρ_1(w)Φ_t(w) O_t[Â](w) .
 



One can directly see, in the case of ρ_1=ρ_2, the weighted importance sampling estimator (<ref>) reduces to the crude Monte Carlo estimator (<ref>).
Moreover, the features of the estimator (<ref>) are provided by 
 
Let w_1, … , w_N∈ℝ^4D be independent and identically distributed with respect to ρ_2 and let τ>0 be a fixed time.
The weighted importance sampling estimator 
    
    A_N^ W(t)= ∑_j=1^N g_t(w_j)W(w_j)/∑_j=1^N W(w_j)

    as defined in (<ref>) has the following properties:
    
    
  * The estimator A_N^ W(t)
    converges in mean to ⟨Â⟩_t for all times t∈[0, τ] as N→∞, that is,
    
    𝔼[| A_N^ W(t) - ⟨Â⟩_t |]  0.

    
  * The estimator A_N^ W is biased 
    and the bias vanishes for N→∞.
    
  * If 𝔼[| g_tW|^2] and 𝔼[| W|^2] are finite, then the bias, the variance, and the mean squared error of the estimator A_N^ W are in the order of 𝒪(N^-1).
    




 §.§ Proof of <Ref>

In the following, we provide the proof of <Ref>. We first examine the convergence, that is guaranteed by the finite time interval [0,τ], of the averages that appear in the numerator and denominator of (<ref>). Then, we employ Taylor expansions of the estimator A_N^ W to assess the bounds for the convergence error, bias, variance, and mean squared error which depend on the individual sample averages.
We emphasize that <Ref> is taken for granted.

We start with two helpful results:


    Let τ>0 be a fixed time, let ρ_1, ρ_2>0 be arbitrary densities on the double phase space ℝ^4D and let w_1,… ,w_N∈ℝ^4D be samples independent and identically distributed with respect to ρ_2.
    The individual sample averages
    
    G_N(t):= 1/N∑_j=1^N g_t(w_j)W(w_j)

    and 
    
    W_N:= 1/N∑_j=1^N W(w_j)

    converge almost surely to the means 𝔼[G_N(t)]=⟨Â⟩_t and 𝔼[W_N]=1 for all times t∈ [0,τ] as N→∞. Moreover, they also converge in L^1.


    Because the samples w_1,… ,w_N∈ℝ^4D are independent and identically distributed with respect to ρ_2, it holds that
    
    𝔼[G_N(t)]   =1/N𝔼[∑_j=1^N g_t(w_j)W(w_j)]
            =∫_ℝ^4D g_t(w)W(w) ρ_2(w)  dw
            = ⟨Â⟩_t

    as well as
    
    𝔼[W_N]=1/N𝔼[∑_j=1^N W(w_j)]= ∫_ℝ^4D W(w) ρ_2(w) dw = 1.

    Therefore, W_N and G_N are unbiased estimators with means 𝔼[W_N]=1 and 𝔼[G_N(t)]=⟨Â⟩_t.
    Moreover, since ρ_1 and ρ_2 are two densities on the double phase space ℝ^4D, we obtain
    
    𝔼[| W |]    = ∫_ℝ^4D| W(w) |ρ_2(w)  dw 
            = ∫_ℝ^4Dρ_1(w)/ρ_2(w)ρ_2(w)  dw=1

     and, by <Ref> and <Ref>, it follows 
    
    𝔼[| g_tW |] = ∫_ℝ^4D| g_t(w) W(w) |ρ_2(w) dw = ∫_ℝ^4D| f_0(w) Φ_t(w) O_t[Â](w) | dw < ∞.

    Hence, applying the strong law of large numbers shows almost sure convergence of the estimators G_N(t) and W_N for all times t∈ [0, τ] as N→∞. The convergence in L^1 also follows from the strong law of large numbers <cit.>.

To assess the bounds for the convergence error, bias, variance, and mean squared error we use
 
    Let the samples w_1, … , w_N∈ℝ^4D be independent and identically distributed with respect to ρ_2.
    Then, the zeroth- and first-order Taylor expansions of A_N^ W, as a function from ℂ×ℝ to ℂ, around the vector μ=(⟨Â⟩_t, 1 ) ∈ℂ×ℝ_>0 are given by
    
    A_N^ W   =⟨Â⟩_t + R_0(G_N(t),W_N)

    and
    
    A_N^ W   =⟨Â⟩_t + (G_N(t)-⟨Â⟩_t) - ⟨Â⟩_t(W_N-1)   + 1/2R_1(G_N(t),W_N),

    with the complex-valued remainders R_0 and R_1 acting on ℂ×ℝ.
Moreover, with probability one, there is N_0∈ℕ and a constant c>0 depending on N_0 such that 
    
    | R_0(G_N(t), W_N) |≤ c(N_0)(| G_N(t)-⟨Â⟩_t | + | W_N-1| ),
            
       and
       | R_1(G_N(t), W_N) |≤ c(N_0)(| G_N(t)-⟨Â⟩_t || W_N-1| + | W_N-1|^2 )

    for all N≥ N_0.




   Consider the function 
   
    f: ℂ×ℝ_>0→ℂ,    (a, b) ↦a/b.

    First, by the definition of A_N^ W, one can see that f(G_N(t), W_N)=A_N^ W. Then, the Taylor expansions of A_N^ W follow directly from Taylor's theorem applied to f with the remainders
    
    R_0(G_N(t), W_N)    =  (G_N(t)-⟨Â⟩_t)∫_0^1 1/1+s(W_N-1) ds
        
       - (W_N-1)∫_0^1 ⟨Â⟩_t + s(G_N(t) -⟨Â⟩_t )/[1+s(W_N-1)]^2 ds,

and

    R_1(G_N(t),W_N)    = 2(W_N-1)^2∫_0^1 (1-s)⟨Â⟩_t + s(G_N(t) -⟨Â⟩_t )/[1+s(W_N-1)]^3 ds 
         
        - (G_N(t)-⟨Â⟩_t)(W_N-1) ∫_0^1 (1-s)1/[1+s(W_N-1)]^2 ds.


   Second, by <Ref>, G_N(t) and W_N converge almost surely to ⟨Â⟩_t and 1 as N→∞. 
    With probability one, there is 0<δ<1 and N_0 ∈ℕ such that 
    
    (G_N(t),W_N) ∈ B_δ(⟨Â⟩_t) × [1-δ, 1+δ]

    for all N> N_0 with the closed ball B_δ(⟨Â⟩_t) = {a∈ℂ: | a-⟨Â⟩_t |≤δ} . 
    Therefore, by the extreme value theorem, there is a constant c_0>0 such that
    
    | R_0(G_N(t),W_N) |   ≤| G_N(t)-⟨Â⟩_t  |∫_0^1  |∂_af(μ+s((G_N(t),W_N)-μ)) | ds
        
       +  | W_N-1 |∫_0^1  |∂_bf(μ+s((G_N(t),W_N)-μ)) | ds
        
       ≤ c_0 (| G_N(t)- ⟨Â⟩_t | + | W_N- 1 | )

    for all N large enough. 
    Similarly, one obtains the bound 
    
    | R_1(G_N(t), W_N) |≤ c_1(| G_N(t)-⟨Â⟩_t || W_N-1| + | W_N-1|^2 )

    with a constant c_1>0.
    The choice of c=max{c_0,c_1} proves the lemma.



Combining <Ref> and <Ref> we obtain the proof of <Ref>.


 1. <Ref> implies that
    
    𝔼[| A_N^ W - ⟨Â⟩_t |]    = 𝔼[|⟨Â⟩_t + R_0(G_N(t),W_N) - ⟨Â⟩_t |]
            
       =𝔼[| R_0(G_N(t), W_N) | ]
            
       ≤ c (𝔼[| G_N(t) - ⟨Â⟩_t |] +𝔼[| W_N - 1 |] )

    for some constant c>0 and for N sufficiently large. 
    By <Ref>, the right-hand side vanishes as N→∞. Therefore, the estimator is consistent.


2. By <Ref>, the bias of the estimator satisfies the equality

    Bias(A_N^ W(t)) 
         = |𝔼[R_0(G_N(t), W_N)] |.

For N large enough, the remainder can be bounded by <Ref> and we obtain

    Bias(A_N^ W(t)) ≤ c (𝔼[| G_N(t) - ⟨Â⟩_t |] +𝔼[| W_N - 1 |] ),

for some constant c>0. The bias vanishes as N→∞ by <Ref>.


3. If 𝔼[| g_tW |^2], 𝔼[| W |^2]< ∞, the bias obeys

    Bias(A_N^ W(t)) 
            
       = |𝔼[ ⟨Â⟩_t + (G_N(t)-⟨Â⟩_t) - ⟨Â⟩_t(W_N-1)   + 1/2R_1(G_N(t),W_N)  ] - ⟨Â⟩_t |
        = 1/2|𝔼[R_1(G_N(t), W_N)]  |,

by <Ref> and <Ref>. 
Moreover, by <Ref>, the remainder can be bounded for N large enough and the Cauchy-Schwarz inequality leads to

    Bias(A_N^ W(t))    ≤c/2𝔼[| G_N(t)-⟨Â⟩_t || W_N-1| + | W_N-1|^2] 
        
       ≤c/2N ([W] + √([W][g_tW]))

for some constant c>0.
Assumptions 𝔼[| g_tW |^2] and 𝔼[| W |^2]< ∞ imply that Bias(A_N^ W(t))=𝒪(N^-1). Using  similar arguments, we find that the variance of the estimator satisfies

    [A_N^ W(t)]    = [R_0(G_N(t), W_N)]  
    
       ≤𝔼[| R_0(G_N(t), W_N) |^2]
    
       ≤ c^2 𝔼[(| G_N(t)-⟨Â⟩_t | + | W_N-1 | )^2 ]
    
        = c^2 𝔼[| G_N(t)-⟨Â⟩_t |^2 + | W_N-1 | ^2 + 2 | G_N(t)- ⟨Â⟩_t || W_N -1 |]

for N large enough. Therefore, Var[A_N^ W(t)] = 𝒪(N^-1).
Finally, the mean squared error is given by
   
    𝔼[| A_N^ W(t) - ⟨Â⟩_t  |^2]     = 𝔼[| A_N^ W(t) - 𝔼[A_N^ W(t)] - ⟨Â⟩_t +𝔼[A_N^ W(t)]|^2]
                
        = Var[A_N^ W(t)] +  Bias(A_N^ W(t)) ^2
                
        = 𝒪(N^-1),

    as the variance and bias are of order 𝒪(N^-1).


From a computational point of view, ρ_2 might only be known up to a constant κ≠ 0, that is, ρ_2/κ is known. Assume that the samples w_j∼ρ_2 can still be generated. 
By the linearity, one can then replace W=ρ_1/ρ_2 in the estimators G_N(t), W_N and A_N^ W with W=κρ_1/ρ_2. 
Because this is a constant factor, the properties of <Ref> remain the same.   
Without loss of generality and for the sake of readability, we omit to mention any such constant κ here and in the following sections.


    The analysis of the weighted importance sampling estimator (<ref>) is a generalization of the analysis of the crude Monte Carlo estimator (<ref>). 
    If one sets ρ_1=ρ_2, all stated formulas for the bias, variance, and mean squared error reduce to the corresponding formulas for the crude Monte Carlo estimator (<ref>).


    Assuming the existence of fourth moments 𝔼[| g_tW|^4] and 𝔼[| W|^4]< ∞, one can prove the estimate
    
    [A_N^ W(t)]= [g_tW-⟨Â⟩_t W]/N + 𝒪(N^-2)

    by using a second-order Taylor expansion of A_N^ W around the vector μ=(⟨Â⟩_t, 1).




 §.§ Choice of the weight W

By <Ref>, for any choice of the weight W, the estimator is consistent and the bias vanishes for N→∞. 
Moreover, if 𝔼[| g_tW|^2],𝔼[| W|^2]< ∞, the bias, variance, and mean squared error of the estimator are in the order of 𝒪(N^-1). 
In the following, we briefly discuss various choices for the weight W. 

First, we provide an example in which an explicit formula for the variance (<ref>) of the weighted importance sampling estimator (<ref>) is known.

    Consider Â=Id at initial time t=0 with an initial Gaussian wavepacket ψ_0 = g_z_0 for some initial phase-space point z_0∈ℝ^2D. For ρ_1=  ρ^ dbl_ and ρ_2 = ρ^ dbl_, finite fourth moments of g_0W and W exist.
    Then,
    
    Var[g_0W-W] = 𝔼[| g_0W|^2 + | W |^2 - 2 Re(g_0W^2)]+ 𝒪(N^-2)

     and the first two terms are given by
    
    𝔼[| g_0W|^2] = ∫_ℝ^4D| f_0(w) O_0[Id](w) |^2/ρ_2(w)   = 4^D ∫_ℝ^4D| f_0(w) || O_0[Id](w) |^2 dw  = ( 16/5)^D,
        
    and   𝔼[| W|^2]    = ∫_ℝ^4Dρ_1(w)^2/ρ_2(w) dw = (16/9)^D.

    Finally, by applying several Fourier transforms of Gaussians, we find that
    
    𝔼[g_0W^2] = ∫_ℝ^4D f_0(w) O_0[Id](w) ρ_1(w)/ρ_2(w)   =(16/9)^D

    and we obtain the variance of the estimator
    
    [A_N^ W(0)] = ((16/5 )^D - (16/9 )^D)/N + 𝒪(N^-2).

    In the case of D=10, the term (16/5)^D-(16/9)^D in the variance (<ref>) is approximately 1.1× 10^6.


There is also an example with an unbounded second moment of the weight W and, therefore, not all properties stated in <Ref> will be applicable.
[Diverging second moment]
Let  Â=Id, ρ_1 =ρ^ dbl_ and ρ_2 = ρ_ opt with initial Gaussian wavepacket g_z_0 and an initial phase-space point z_0∈ℝ^2D. Recall that ρ_ opt is proportional to

    ρ_ opt(w) ∝|| f_0(w) ||⟨ g_y , g_z ⟩|.

Then,

    |ρ^ dbl_|^2/|ρ_ opt|∝exp[-1/8ħ[ z-z_0; y-z_0 ]^T [ 0 Γ; Γ 0 ][ z-z_0; y-z_0 ]].

The width matrix in (<ref>) is not positive-definite. Hence, the second moment 𝔼[| W |^2] of the weight W=ρ_1/ρ_2 does not exist. 



In <Ref>, we consider numerical examples with several choices of the weight W.
It turns out that, the choice ρ_1 = ρ^ dbl_ H and ρ_2=ρ_ opt has the best performance.
This suggests the following weighted importance sampling estimator

    A_N^ W(t) = ∑_j=1^N f_0(w_j) Φ_t(w_j) O_t[Â](w_j) / ρ_ opt(w_j)/∑_j=1^N ρ^ dbl_ H(w_j)/ρ_ opt(w_j), Case WIS opt

with independent samples w_j ∼ρ_ opt.




 §.§ Algorithm


We have analysed several sampling strategies for Herman–Kluk expectation values based on the Husimi, sqrt-Husimi (Sec. <ref>), and the optimal approach  (Sec. <ref>) that either use the crude Monte Carlo  (<ref>) or the weighted importance sampling estimator (<ref>). 
Hence, we propose the following general <Ref> to compute the expectation value of an operator Â at a given time t within the Herman–Kluk approximation.

For the Markov-Chain-Monte-Carlo algorithm in <Ref>, we use the Hamiltonian Monte Carlo algorithm described in Appendix <ref>.
With a fictitious, random momentum variable and a carefully chosen Hamiltonian, it uses a symplectic, time-reversible integrator that propagates an initial double phase-space variable w_0∈ℝ^4D according to Hamiltonian's equations of motion and accepts or rejects generated points w_j ∈ℝ^4D. 

For the Hamiltonian Monte Carlo Algorithm <ref> (see Appendix <ref>), we use a second-order Størmer-Verlet integrator for the propagation. 
Recently analyzed multi-stage integration schemes <cit.> provide a way to improve the efficiency of the Hamiltonian Monte Carlo algorithm by reducing the expensive evaluation of the gradient of the carefully chosen potential.
However, in <Ref>, the generation of the samples in step (<ref>) is inexpensive compared to the remaining steps (<ref>) and (<ref>).
Therefore, when using splitting schemes for the time integrator of the Hamiltonian Monte Carlo Algorithm <ref>, we did not observe any noticeable improvement in <Ref> over the standard Størmer-Verlet integrator. 




§ NUMERICAL EXAMPLES

This section complements the theoretical results with numerical examples for the three different sampling approaches and the most important operators. 

In all our tests, we assume that the initial state ψ_0 is a spherical Gaussian wavepacket ψ_0 = g_z_0 with Γ=Id_D,  centred at a phase-space point z_0∈ℝ^2D. 
For all examples, we used the crude Monte Carlo estimators (<ref>) for the Husimi approach and (<ref>) for the sqrt-Husimi approach.
For the optimal approach, when we compute the norm, we used the crude Monte Carlo estimator (<ref>) and for other observables, we used the weighted importance sampling estimator (<ref>) in combination with the Hamiltonian Monte Carlo <Ref>.

To choose a good weight W for the optimal approach with the weighted importance sampling estimator, we first investigate the initial sampling error for several choices of the weight W in one and six dimensions.
Subsequently, we compare the performance at the initial time for low and high dimensions of the Husimi, sqrt-Husimi, and optimal approaches. 
In the case of a five-dimensional harmonic oscillator, we compare numerical results with analytical solutions, showing that the optimal approach converges faster.
Finally, we examine a six-dimensional Henon-Heiles potential for various initial conditions, going from regions of low to high anharmonicity.
We show that, whenever the Herman–Kluk prefactor blows up on average, the choice of the density is of minor importance.





 §.§ Weight W
 





To determine the choice of the weight W, we provide the following numerical example.
Consider a Gaussian initial state ψ_0 = g_z_0 with unit width, q_0=(1,…,1)∈ℝ^D and p_0=(1,…,1)∈ℝ^D and let Â=Id or q̂_1.
<Ref> displays the mean squared error of the weighted importance sampling estimator in one and six dimensions for several choices of the weight W.
The value obtained analytically in <Ref> matches our numerical result.
Moreover, in both dimensions and for both operators, for large enough N, the choice W=ρ^ dbl_ H/ρ_ opt for the weight provided the highest accuracy even though, for the position operator, the weight W might have a diverging second moment.
Finally, for small N, one can see a higher error due to the bias of the estimator.

This short numerical example suggests using the weighted importance sampling estimator (<ref>) with the weight W=ρ^ dbl_ H/ρ_ opt.



 §.§ Initial time


We start by considering the initial sampling error for the norm, position, and momentum expectations, that is, Â=Id, q̂_j and p̂_j,
with  ϵ = 1, q_0=(1, … , 1), and p_0=(1, … ,1) for up to 10 dimensions. 
<Ref> shows the absolute errors

    | A_N(0) - ⟨Â⟩_0 |

and

    | A_N^ W(0) - ⟨Â⟩_0 |

between the Monte Carlo approximations A_N(0) and A_N^W(0) and the exact value at time t=0 as functions of the number of Monte Carlo quadrature points N in one (first column) and ten (second column) dimensions as well as for a fixed N=2^20 as a function of the dimension D (third column). Each panel was produced by averaging the error over 100 independent simulations.

Due to the unbounded variance, the Husimi approach performs the worst, and the sqrt-Husimi and optimal approaches confirm our theoretical error estimations derived in <Ref>, <Ref> and Appendix <ref>.
For a small number of trajectories, the bias of the weighted importance sampling estimator causes a significant error that rapidly decays.

For D=1 and large N, there is almost no difference between the optimal and the sqrt-Husimi approaches.
However, in high dimensions, starting from approximately N=10^2 Monte Carlo points, both the correlation of the chain and the bias of the estimator are negligible, and we obtain an improvement with the optimal approach over both the Husimi and sqrt-Husimi approaches. If we compare different methods “horizontally”, to reach the same accuracy with the optimal approach, one might need up to a factor of approximately 10^2 fewer trajectories compared with the sqrt-Husimi approach.




 §.§ Harmonic Potential


To analyse the time dependence of the variance, we consider a harmonic potential V(x)=| x |^2/2 in five spatial dimensions for one full oscillation period. 
We consider the same initial values as in the previous numerical example and compute the norm, position, momentum, potential, kinetic, and total energies and compare them to the corresponding expectation values ⟨Â⟩_t obtained from the exact solution of the time-dependent Schrödinger equation <cit.>. In this numerical example, the exact solutions are given by the constants ⟨Id⟩_t = 1, ⟨Ĥ⟩_t = 6D/4 and the time-dependent values

    2⟨q̂_j⟩_t       = cost + sint,
      -.5[0pt][0pt]for all j∈{1,… , D},
    ⟨p̂_j⟩_t    = cost - sint, 
    ⟨ V ⟩_t    = D/4( 1 + 2(cost + sint))^2 ), 
    ⟨ T ⟩_t       = D/4( 1 + 2(cost - sint))^2 ).



<Ref> shows the behaviour of the absolute error between our Monte Carlo approximations with N=2^20  quadrature points and the exact solution as a function of time t.
The trajectories for the optimal approach were sampled using the Hamiltonian Monte Carlo <Ref>. 
The error of the total energy was produced by taking the absolute value of the difference between the exact total energy and the sum of the approximations for the potential and kinetic energy. 
In all the examples, we ran 100 individual simulations and then plotted the square root of the mean of the squares of the errors. 
We can see that, in all cases, the Husimi approach gives the highest error, whereas the optimal approach provides the most accurate results. 
The Husimi approach needs approximately 100 times more trajectories to achieve the same accuracy as the sqrt-Husimi approach and the sqrt-Husimi approach requires approximately 10 times more trajectories to reach the same accuracy as the optimal approach.
Moreover, the result for the square root Husimi approach almost perfectly coincides with the explicitly predicted errors (see Appendix <ref>).




 §.§ Henon-Heiles Potential

Finally, we consider the modified Henon-Heiles potential

    V(x) = ∑_j=1^D x_j^2/2 +∑_j=1^D-1[σ( x_j x_j+1^2 - x_j^3/3) +σ^2/16( x_j^2 + x_j+1^2 )^2],   σ≥ 0,

as in <cit.> in six spatial dimensions. This potential differs from the standard Henon-Heiles model <cit.> by an additional quartic term that makes the system bound.
We take ϵ=0.01, σ=1/√(80) and p_0=(0, …,0) based on <cit.> and let the initial displacement q_0 of the Gaussian wavepacket vary to be in regions of the potential with low to high anharmonicity. For the time propagation, we use a second-order Størmer-Verlet scheme with a stepsize τ = 0.2.



<Ref> shows the behaviour over time of the averaged Herman–Kluk prefactors

    1/N∑_j=1^N | R_t(y_j) R_t(z_j) |,

where (y_j,z_j) are sampled from ρ^ dbl_ H, ρ^ dbl_ or ρ_ opt. 
We considered three different initial conditions, with  q_0=k · (1, … ,1) for k∈{1,2,2.3}. 
Note that for any initial displacement, the three sampling approaches result in a similar behaviour of the averaged Herman–Kluk prefactors.  
Moreover, for each  k∈{1,2,2.3}, we display the total energy E_tot of the systems that can be obtained from formulas stated in Appendix <ref>.
Small changes in the total energy of the system lead to large changes in the growth of the Herman–Kluk prefactor, showing that the potential (<ref>) has very anharmonic regions.


In the following, we will analyze the time evolution of the intrinsic error 

    | A_N(t)-A_2N(t)|
 for the different displacements.
<Ref> shows the evolution up to the final time T=40 for a fixed number N=2^19≈ 5× 10^5 of quadrature points. 
For a small q_0=(1,…, 1), the optimal approach has a significantly lower error than the square root Husimi approach, and the Husimi approach performs even worse because of the unbounded variance of its estimator (<ref>). 
For q_0=(2, …, 2), the Husimi approach continues to have a significantly larger error than the other two approaches. 
The square root Husimi and optimal approaches behave similarly. 
The optimal approach still produces a smaller error, but it is less significant and only for times up to about T=20.
For a large q_0=(2.3, …, 2.3), the square root Husimi and optimal approaches exhibit a similar performance. 
However, the Husimi approach does not necessarily increase the error, and if it does, then only for small times up to approximately T=20.
We can conclude that for small anharmonicity, where the Herman–Kluk prefactor behaves moderately, the choice of the density significantly improves the accuracy. For increasing anharmonicity, and hence for a Herman–Kluk prefactor that blows up, the choice of the density is less important up to the point where even an estimator with unbounded variance has a similar accuracy.





Finally, we consider error estimations of the intrinsic error (<ref>) for the square root Husimi and optimal approaches. 
From (<ref>) it follows that

    𝔼[| A_N(t)-A_2N(t)|^2] = 3/2N𝕍_t[Â, ψ_0, ρ^ dbl_ ],

where the variance can be simultaneously evaluated with another Monte Carlo estimator and without substantial additional effort  (see Appendix <ref>). 
Even though we know for the weighted importance sampling estimator at most the order of the mean squared error by <Ref>, we can still estimate 𝕍_t[Â, ψ_0, ρ_ opt] and compare the mean squared error with


    3/2N𝕍_t[Â, ψ_0, ρ_ opt].

More details on (<ref>) and the estimators of the variances can be found in Appendix <ref>. 
<Ref> shows the same time evolution of the numerical intrinsic error (<ref>) as in <Ref>, but only for the square root Husimi and optimal approaches. 
Additionally, in <Ref> we also display the probabilistic error estimation (<ref>) for the sqrt-Husimi approach and the approximate error (<ref>) for the optimal approach. 
Interestingly, we observe that in most cases, the weighted importance sampling estimator approximately follows (<ref>).




§ CONCLUSION, OUTLOOK AND FURTHER APPLICATIONS

We presented an analysis of Monte Carlo quadrature techniques for evaluating expectation values of quantum-mechanical observables Â within the Herman–Kluk approximation.
After introducing a sufficient condition for the convergence of the crude Monte Carlo estimator, we investigated three sampling approaches.
For the initial double phase-space sampling, we used one of three different strategies, based on the Husimi density (<ref>), its square root (<ref>) or the new optimal method (<ref>). 
While the generation of quadrature points with the first two approaches is independent of Â, the optimal approach incorporates the quantum-mechanical observable in its sampling.
The Husimi approach has a Monte Carlo integrand with an unbounded second moment, and the square root Husimi approach produces a favourable finite variance.
The optimal approach minimizes the variance of the crude Monte Carlo integrand at the initial time but it still has an exponential dependence on the spatial dimension D (see also <cit.> for a discussion of the dimensionality dependence of Monte Carlo quadrature).
To take full advantage of the new approach, we extended the crude Monte Carlo estimator to a weighted, self-normalizing version.

The numerical experiments for the harmonic oscillator and the Henon-Heiles potential confirm that the infinite second moment leads to slower convergence of the Husimi approach and that the optimal approach has the fastest convergence. 
For dynamical regions with high anharmonicities, where the Herman–Kluk propagator is known to be inaccurate due to the drastic growth of the Herman–Kluk prefactor, the differences in convergence become smaller up to the point, where the choice of the sampling approach plays a minor role.

The new approach presented in this paper has applications in several fields.
Theoretical chemists make ubiquitous use of the Herman–Kluk approximation. 
They apply further approximations such as time-averaging <cit.>, Filinov filtering <cit.>, hybrid dynamics <cit.> and many others to improve Monte Carlo estimators by reducing the required number of quadrature points. 
Our idea can be combined with these techniques to make the Herman–Kluk approximation even more computationally attractive.
Moreover, expectation values are a special case of time-correlation functions <cit.>. 
The idea of including observables in the sampling can be extended to time-correlation functions. It has been explored for classical correlation functions in <cit.> and will be explored for evaluating time-correlation functions within the Herman–Kluk approximation in the future.
Finally, the analysis at the initial time t=0 presented here might be of general interest for all Gaussian-based methods.





§ ACKNOWLEDGMENTS

C. Lasser acknowledges financial support by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)  TRR 352 – Project-ID 470903074.
F. Kröninger and J. Vaníček acknowledge financial support from the European Research Council (ERC) under the European Union's Horizon 2020 Research and Innovation Programme (Grant Agreement No. 683069-MOLEQULE).

abbrvnat







§ BOUND FOR INTRINSIC MEASURES

When an exact solution is not available, intrinsic measures can be used as indicators of the goodness of an approximation.
Assume we have an unbiased estimator A_N(t) = 1/N∑_j h_t(w_j) of ⟨Â⟩_t with existing second moment. 
One can consider the distance between A_N and A_2N since

    𝔼[| A_N - A_2N|^2] 
        
        = 𝔼[ (| A_N - ⟨Â⟩_t) + (⟨Â⟩_t - A_2N) |^2 ]
        
        = 𝔼[| A_N - ⟨Â⟩_t |^2] + 𝔼[| A_2N- ⟨Â⟩_t |^2] - 2Re(𝔼[A_N - ⟨Â⟩_t]^* 𝔼[ A_2N - ⟨Â⟩_t ]) 
        
       = 𝕍_t/N  + 𝕍_t/2N - 0 
        
        = 3/2N𝕍_t,

where we assumed that the samples of A_N and A_2N were independent. Let us recall the formulas (<ref>) and (<ref>) for the variances

    𝕍_t[Â, ψ_0, ρ^ dbl_ ] = κ_^2 ∫_ℝ^4D| f_0(w) ||Φ_t(w) |^2 | O_t[Â](w) |^2  dw - |⟨Â⟩_t|^2

and

    𝕍_t[Â, ψ_0, ρ_ opt ] = κ_ opt∫_ℝ^4D| f_0(w) ||Φ_t(w) |^2 | O_t[Â](w) |^2/| O_0[Â](w) | dw - |⟨Â⟩_t|^2.

In the calculation, we can simultaneously evaluate the appearing integrals without any additional work using another Monte Carlo estimator. 
Assuming we know κ_, we then have

    ∫_ℝ^4D| f_0(w) ||Φ_t(w) |^2 | O_t[Â](w) |^2  dw ≈κ^2_/N∑_j=1^N  |Φ_t(y_j,z_j) |^2 | O_t[Â](y_j,z_j) |^2,

for all operators Â and with y_j,z_j ∼ρ_ independent.
For the optimal approach, whenever κ_ opt is known, we can use

    ∫_ℝ^4D| f_0(w) ||Φ_t(w) |^2 | O_t[Â](w) |^2/| O_0[Â](w) | dw ≈κ_ opt/N∑_j=1^N  |Φ_t(w_j) |^2 | O_t[Â](w_j) |^2/| O_0[Â](w_j) |^2,

where w_j∼ρ_ opt are independent.
If κ_ opt is unknown, a slight change in the weighted importance sampling estimator leads to

    κ_ opt∫_ℝ^4D| f_0(w) ||Φ_t(w) |^2 | O_t[Â](w) |^2/| O_0[Â](w) | dw
        
       ≈1/N∑_j=1^N  W(w_j)| f_0(w_j)||Φ_t(w_j) |^2 | O_t[Â](w_j) |^2/(ρ^ dbl_ H(w_j))| O_0[Â](w_j) |)/1/N^2(∑_j=1^N W(w_j))^2,

with the weight W=ρ^ dbl_ H /ρ_ opt and independent samples w_j ∼ρ_ opt.





§ MOMENTS OF GAUSSIAN RANDOM VARIABLES
 
Consider a multivariate Gaussian

    G^ M(x)= (2πϵ)^-D/2Ξ^-1/2exp( -1/2ϵ (x-μ)^T Ξ^-1 (x-μ) )

with mean vector μ∈ℝ^D and symmetric, positive-definite covariance matrix Ξ∈ℝ^D × D.
Let Ξ = L L^T be its Cholesky decomposition with a lower triangular matrix L∈ℝ^D × D and let Z∼ N(μ, Ξ). 
Moreover, let Pol: ℝ^D →ℝ be a multivariate polynomial. Then,

    𝔼[Pol(Z)]  = 𝔼[Pol(LX+μ)]

with the D-dimensional random vector X=(X_1, …, X_D) ∈ℝ^D, where X_1,…, X_D are independent and identically distributed with respect to N(0,ϵ). Since the composition of two polynomials is a polynomial and since X_1, …, X_D are independent and identically distributed, 𝔼[Pol(LX+μ)] depends polynomially on ϵ, μ and Ξ <cit.>. 




§ FORMULAS FOR INNER PRODUCTS
 
In the following, we provide analytical formulas for the overlaps of the form
    ⟨ g_y , Â  g_z ⟩
for important examples of Â 
 with a Gaussian wavepacket

    g_z(x)= ( Γ/π^D ϵ^D)^1/4exp[-1/2ϵ (x-q)^T  Γ  (x-q) +i/ϵp^T (x-q) ] ,    z=(q,p).


 
[Â=Id]

Using the substitution x ↦ x +(q_y+q_z)/2 and the Fourier transform, we obtain

    ⟨ g_y  , g_z ⟩    = ( Γ/π^D ϵ^D)^1/2∫_ℝ^Dexp[-1/2ϵ (x-q_y/2+q_z/2)^T  Γ  (x-q_y/2+q_z/2)]
       ×exp[-1/2ϵ (x-q_z/2+q_y/2)^T  Γ  (x-q_z/2+q_y/2)]
       ×exp[-i/ϵ p_y^T   (x-q_y/2+q_z/2) +i/ϵp_z^T  (x-q_z/2+q_y/2) ] dx
        
        = exp{[ -1/4 (y-z)^T  Σ_0  (y-z) + i/2 (p_y+p_z)^T  (q_y-q_z) ]/ϵ},

with Σ_0=diag(Γ, Γ^-1)∈ℝ^2D.

[Â=q̂_j] 
Similarly, for the position operator it follows

    ⟨ g_y  , q̂_j  g_z ⟩    
         = 1/2( q_y +q_z -i Γ^-1  p_y+ i Γ^-1  p_z )_j ⟨ g_y , g_z ⟩.

 


[Â=q̂^2_j]
For the second moment of the position operator it holds

    ⟨ g_y  , q̂_j^2  g_z ⟩ 
           = [ 2ϵΓ^-1 + (q_y+q_z-iΓ^-1p_y + iΓ^-1p_z)^2]_j ⟨ g_y , g_z ⟩/4.




[Â = p̂_j]
Changing to the momentum representation and applying the same techniques as before, we obtain

    ⟨ g_y , p̂_j g_z ⟩ = 1/2( p_y+p_z+iΓ q_y - i Γ q_z )_j ⟨ g_y , g_z ⟩.




[Â = p̂^2_j]
For the kinetic energy it holds

    ⟨ g_y , p̂^2_j  g_z ⟩ = [ 2ϵΓ + (p_y+p_z+iΓ q_y - iΓ q_z)^2]_j ⟨ g_y , g_z ⟩/4.




[Henon-Heiles potential (<ref>)]
By linearity, the overlap is given by

    ⟨ g_y , V g_z ⟩ = 1/2∑_j=1^D  ⟨ g_y, q̂_j^2 g_z ⟩_=: I_1 + σ∑_j=1^D-1⟨ g_y,  ( q̂_j q̂_j+1^2 - q̂_j^3/3) g_z ⟩_=:I_2 + σ^2/16∑_j=1^D-1⟨ g_y,   ( q̂_j^2 + q̂_j+1^2 )^2 g_z ⟩_=:I_3.

We obtain

    I_1 = ⟨ g_y , g_z ⟩/8∑_j=1^D [ 2ϵΓ^-1 + (q+iΓ^-1p)^2]_j ,


    I_2 = σ⟨ g_y , g_z ⟩∑_j=1^D-1   { (q+iΓ^-1p)_j/2[ 2ϵΓ^-1 + (q+iΓ^-1p)^2]_j+1/4.
            
       .-[6ϵΓ^-1q + 6iϵΓ^-2p + (q + iΓ^-1p)^3 ]_j/24}

and

    I_3 
               = σ^2 ⟨ g_y , g_z ⟩/16∑_j=1^D-1{[ 2ϵΓ^-1 + (q+iΓ^-1p)^2]_j[ 2ϵΓ^-1 + (q+iΓ^-1p)^2]_j+1/8.
            
       .+ [12ϵΓ^-1q^2+24ia^3qs-12a^4s^2+12ϵ^2Γ^-2 + (q +iΓ^-1p)^4 ]_j/16.
            
       .+ [12ϵΓ^-1q^2+24ia^3qs-12a^4s^2+12ϵ^2Γ^-2 + (q +iΓ^-1p)^4 ]_j+1/16}

with the abbreviations q := q_y+q_z, p  := p_z - p_y,  a:= ϵ^1/2Γ^-1/2 and s:= (ϵΓ)^-1/2p. 





§ VARIANCES FOR THE SQUARE ROOT HUSIMI APPROACH
 
We provide analytical formulas for the variance (<ref>) with a Gaussian initial state ψ_0 = g_z_0. For notational simplicity, we assume Γ=Id_D and employ the notations w=(y,z) and w_0=(z_0,z_0). To obtain the formulas for general Γ, one may invoke the transformation 

    (w-w_0) ↦[ Σ_0^-1/2        0;        0 Σ_0^-1/2 ] (w-w_0),

where Σ_0=diag(Γ,Γ^-1)∈ℝ^2D× 2D. Moreover, whenever we consider 4D× 4D matrices of the form [ a b; c d ], a,b,c,d∈ℝ, we understand [ aId_2D bId_2D; cId_2D dId_2D ].


 §.§ Initial time

[Â = Id]
    
    𝕍_0[  , g_z_0, ρ^ dbl_ ] 
        
        = (πϵ)^-2D∫_ℝ^4Dexp[ -1/4ϵ (w-w_0)^T  [  3 -2; -2  3 ]  (w-w_0) ] dw -1
        
        =  (16/5)^D-1.


[Â = q̂_j]
Employing the transformation 
    [ z-z_0; y-z_0 ]↦1/2[  1 -1;  1  1 ][ z-z_0; y-z_0 ],
 we obtain 

    𝕍_0[q̂_j , g_z_0, ρ^ dbl_] 
            = (πϵ)^-2D/4∫_ℝ^4D| q_z,j+q_y,j+i p_z,j - i p_y,j|^2
        
       ×exp[ -1/4ϵ[ z-z_0; y-z_0 ]^T  [  3 -2; -2  3 ][ z-z_0; y-z_0 ]] d(z,y) -q_0,j^2
        
        =  1/4(16/5)^D[ 4q_0,j^2 + 24/5ϵ]-q_0,j^2

where the j-th component of vectors is denoted by a subscript.


[Â = p̂_j]
    Similarly to the position operator, for the momentum we obtain

    𝕍_0[p̂_j , g_z_0, ρ^ dbl_] 
        =  1/4(16/5)^D[ 4p_0,j^2 + 24/5ϵ ]-p_0,j^2.
 

[Â = ∑_j=1^Dq̂_j^2/2] 
For simplicity, we assume the initial position q_0=c_1(1,…, 1) and initial momentum p_0=c_2(1, …, 1) for some constants c_1,c_2∈ℝ, i. e. the same parameters in each dimension. Then, the variance is given by

    𝕍_0[Â , g_z_0, ρ^ dbl_] 
         
        = 1/16( 16/5)^D [ D^2 ( 2c_1^2 + 13    /5ϵ)^2 + D 96/25( 3ϵ + 5c_1^2) ]- D^2/4(ϵ+2c_1^2)^2.


[Â =  ∑_j=1^Dp̂_j^2/2]
Similarly to the harmonic potential, for the kinetic energy we obtain

    𝕍_0[Â , g_z_0, ρ^ dbl_] 
        
        = 1/16( 16/5)^D [ D^2 ( 2c_2^2 + 13    /5ϵ)^2 + D 96/25( 3ϵ + 5c_2^2) ]- D^2/4(ϵ+2c_2^2)^2.
 

[Â =  ∑_j=1^D(p̂_j^2 + q̂_j^2)/2]

    𝕍_0[Â , g_z_0, ρ^ dbl_] 
        
        = 1/4( 16/5)^D [ D^2  ( (c_1^2 + c_2^2) + 13    /5ϵ)^2 + D 24/5( 6/5ϵ + c_1^2+c_2^2) ]
        
       - D^2(ϵ+c_1^2+c_2^2)^2.
 




 §.§ Harmonic potential

We assume a Gaussian initial state with Γ = _D. For a harmonic potential V(x)=| x|^2/2, there exist explicit solutions to the position, momentum, and Herman–Kluk prefactor. In particular, for all z=(q,p)∈ℝ^2D,

    q(t)=q cos(t)+p sin(t),
        
        p(t)=pcos(t)-qsin(t),
        
       | R_t(z)|^2=1.

Moreover, the Herman–Kluk approximation itself is exact. Hence, we can calculate the behaviour of the variances over time.

[Â=Id]

    𝕍_t[ , g_z_0, ρ^ dbl_] 
         = ( 16/5)^D-1.



[Â=q̂_j]
    
    𝕍_t[q̂_j, g_z_0, ρ^ dbl_] = 1/4(16/5)^D [ 4q_0,j(t)^2+24/5ϵ] - q_0,j(t).




[Â=p̂_j]
    
    𝕍_t[p̂_j, g_z_0, ρ^ dbl_] = 1/4(16/5)^D [ 4p_0,j(t)^2+24/5ϵ] - p_0,j(t).




[Â=∑_j=1^Dq̂^2_j/2] Under the same assumption as in <Ref>, i. e. q_0=c_1(0)(1, … ,1) and p_0=c_2(0)(1, … ,1), c_1(0),c_2(0) ∈ℝ, let
c_1(t) = c_1(0)cos(t)+c_2(0)sin(t)
and c_2(t) = c_1(0)cos(t)-c_2(0)sin(t).
Then
    
    𝕍_t[Â , g_z_0, ρ^ dbl_] 
             = 1/16( 16/5)^D [ D^2 ( 2c_1(t)^2 + 13    /5ϵ)^2 + D 96/25( 3ϵ + 5c_1(t)^2) ]
         
       - D^2/4(ϵ+2c_1(t)^2)^2.




[Â=∑_j=1^Dp̂^2_j/2]
Similarly to the harmonic potential, it holds
    
    𝕍_t[Â , g_z_0, ρ^ dbl_] 
            = 1/16( 16/5)^D [ D^2 ( 2c_2(t)^2 + 13    /5ϵ)^2 + D 96/25( 3ϵ + 5c_2(t)^2) ]
        
       - D^2/4(ϵ+2c_2(t)^2)^2.
 



[Â=∑_j=1^D(q̂^2_j+p̂^2_j)/2]
    
    𝕍_t[Â , g_z_0, ρ^ dbl_] 
        
        = 1/4( 16/5)^D [ D^2  ( (c_1(t)^2 + c_2(t)^2) + 13    /5ϵ)^2 + D 24/5( 6/5ϵ + c_1(t)^2+c_2(t)^2) ]
        
       - D^2(ϵ+c_1(t)^2+c_2(t)^2)^2.






§ HAMILTONIAN MONTE CARLO ALGORITHM

The Hamiltonian Monte Carlo algorithm <cit.> belongs to the class of Metropolis–Hastings algorithms whose aim is to generate samples of probability densities ρ where sampling directly from ρ on ℝ^D is not viable either because the expression of ρ is too complicated or because it is only known up to a multiplicative constant. 
Using a fictitious momentum variable, it generates a Markov chain {ξ^0, ξ^1, …} = {ξ^n}_n∈ℕ which has ρ as its invariant density. 
Introducing the potential energy U(ξ)= - log(ρ(ξ)), the kinetic energy K(Ξ)=Ξ^T  M^-1Ξ/2 with mass matrix M, and the Hamiltonian H(ξ,Ξ) = U(ξ) + K(Ξ), one simulates a Markov chain according to <Ref>.

This algorithm has a few parameters, namely, the initial point ζ_0 and the mass matrix M as well as the step size Δ t and final time T for the numerical integrator. 
These must be chosen carefully to obtain a valuable sequence {ζ^n}. 



 §.§ Gradients for Hamiltonian Monte Carlo

In the following, we provide a method to derive gradients for the Hamiltonian Monte Carlo algorithm to generate a Markov chain distributed with respect to ρ_ opt for specific examples of Â. We assume a Gaussian initial state ψ_0 = g_z_0, z_0∈ℝ^2D, and a unit width matrix Γ=Id_D. In our examples, we have the polynomial decomposition

    ρ_ opt(w)   ∝|⟨ g_z , ψ_0 ⟩⟨ g_y , ψ_0 ⟩⟨ g_y , Â g_z⟩|
        =|(y,z) ||⟨ g_z , ψ_0 ⟩⟨ g_y , ψ_0 ⟩⟨ g_y ,  g_z⟩|

and hence the potential for the Hamiltonian Monte Carlo algorithm is given by

    U[Â](y,z)  = -log|⟨ g_z , ψ_0 ⟩| - log|⟨ g_y , ψ_0 ⟩| - log|⟨ g_y , g_z⟩| - log|(y,z) |.
 
[Â= Id]
In this case, (y,z)=1 and the potential reduces to

    U[](y,z) 
             = 1/4ϵ[(z-z_0)^T  (z-z_0) + (y-z_0)^T   (y-z_0)+ (z-y)^T   (z-y)].

Hence, the gradient is given by

    ∇ U[](y,z) = 1/2ϵ[ (y-z_0) -   (z-y);  (z-z_0) +  (z-y) ].



By the polynomial decomposition (<ref>) and the Potential (<ref>), for an operator Â it follows 

    U[Â](y,z) = U[](y,z) - log|(y,z) |

and 

    ∇ U[Â](y,z) 
        = ∇ U[](y,z)  
     
       +1/|(y,z) |^2[ ((y,z)) ∇((y,z)) + ((y,z))  ∇((y,z)) ] .

Hence to derive the gradients for further examples of Â, one only needs polynomial terms. Those can be taken from Appendix <ref>.





 Paper Title: On the mean-field limit of the Cucker-Smale model with Random Batch Method.




    
        
        
     
        a]Yuelin Wang sjtu_wyl@sjtu.edu.cn
        a]Yiwen Lin linyiwen@sjtu.edu.cn
[a]School of Mathematical Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai, 200240, P.R.China 


On the mean-field limit of the Cucker-Smale model with Random Batch Method
    [
    
==========================================================================



§ ABSTRACT 
    In this work, we focus on the mean-field limit of the Random Batch Method (RBM) for the Cucker-Smale model. Different from the classical mean-field limit analysis, the chaos in this model is imposed at discrete time and is propagated to discrete time flux. We approach separately the limits of the number of particles N→∞ and the discrete time interval τ→ 0 with respect to the RBM, by using the flocking property of the Cucker-Smale model and the observation in combinatorics. The Wasserstein distance is used to quantify the difference between the approximation limit and the original mean-field limit. Also, we combine the RBM with generalized Polynomial Chaos (gPC) expansion and proposed the RBM-gPC method to approximate stochastic mean-field equations, which conserves positivity and momentum of the mean-field limit with random inputs.

    
    Keywords: Random Batch Method, mean-field limit, Cucker-Samle model, stochastic Galerkin.






§ INTRODUCTION


    Collective behaviors of many-body systems are ubiquitous in the real world, like  flocking of birds <cit.>, swarming of fishes <cit.>, synchronicity of fireflies <cit.> and pacemaker cells <cit.>. We use the jargon “flocking" to describe the phenomenon in which self-propelled particles organize into an ordered motion using only limited environmental information and simple rules <cit.>. Since there is a great deal of literature on collective behaviors and related models, we refer the readers to <cit.> and the references therein.
    
     
    The Cucker-Smale model, proposed by Cucker and Smale <cit.>, is a famous model of collective behaviors that models phenomenologically  the flocking phenomenon. It is in the form of an N-body second-order system of ordinary differential equations or particle systems  for position and velocity, which resemble Newton's equations.  
    

     In addition, the mean-field limit, as the number of particles N →∞,  is also a prevalent tool for simplifying large particle systems, which makes use of continuum models derived within the framework of mathematical kinetic theory, to approximate collective behaviors. In these years, there has been extensive research on the mean-field limit of particle systems and we refer to <cit.> for survey articles.
     
    As a classical type of N-body systems, in the Cucker-Smale model, each particle interacts with N-1 particles, and thus the computational cost to solve it is 𝒪(N^2) per time step. An efficient algorithm to reduce computational complexity to 𝒪(N) is the Random Batch Method (RBM) proposed by Jin et al. <cit.> in 2020. The RBM indeed constructs a randomly decoupled system with subsystems of interaction between p particles, where the constant p≪ N. In this system, at each time step, the interaction only occurs within small batches with p particles. The choice of batches is random and thus the time-average effect makes it  a good approximation of the original system <cit.>.
    
     
    
    
    An interesting problem is to understand the mean-field limit of the RBM. In 2022, Jin and Li considered the mean-field limit of the RBM on first-order systems with Gaussian noise in <cit.>. 
        
        
        
        
    Inspired by <cit.>, in this paper, we investigate the mean-field limit of the RBM on the Cucker-Smale model as the number of particles tends to infinity. Unlike the first-order system considered in <cit.>, the Cucker-Smale model is a second-order system with interaction coupled with both velocity and position. Technically, moment control needs to be  derived by the asymptotic flock estimate instead of using the contraction property in first-order systems in <cit.>. Specifically, we mainly analyze the path from (b) to (c), and briefly discuss the path from (a) to (d) in the following figure (<ref>). On the upper left corner, Ha et al. give an estimate on (a) in <cit.> and a proof of the existence of mean-field limit on (d) in <cit.>. Their analysis on (d) is uniform in time but implicit in N. In later work <cit.>, an explicit estimate on N is given but only works on finite time. The four models in (<ref>) are represented in Section <ref>.
    
    
    Cucker-Smale system (<ref>) @ > (d)  N→∞ >> mean-field limit (<ref>)
    
             @A (a) τ→ 0 AA  @AA (c) τ→ 0 A
    Random Batch Cucker-Smale model (<ref>) @>> (b)  N→∞ > mean-field limit of RBM (<ref>)


    
   With the justification of (b) and (c),   we employ the RBM, which is an efficient particle-based numerical scheme for the approximation of stochastic mean-field equations of the Cucker-Smale model, inspired by <cit.>. We consider such an application in the stochastic setting, where the interacting force depends on a random variable modeling uncertainties.  This is an important problem in Uncertainty Quantification <cit.>. The method combines the RBM with a generalized Polynomial Chaos (gPC) expansion in the random space and thus we call it the RBM-gPC. It avoids loss of positivity like MCgPC in <cit.>, and moreover preserves the mean velocity during evolution.

    
    The rest of the paper is organized as follows. In Section <ref>, we give a concise introduction to the Cucker-Smale model including notations, assumptions and prerequisites. Section <ref> presents our main results on the analysis of limits (b) and (c) in (<ref>) with helpful discussions on the limits from (a) to (d). Section <ref> and Section <ref> provide the proof details. In Section <ref>, we show some numerical experiments with the RBM-gPC. Finally, Section <ref> is devoted to a summary of our main results and some remaining issues to be explored in the future.
    
    
    
    


  




§ PRELIMINARIES

In this section, we introduce the main models in (<ref>) with assumptions and notations. Also, some basic properties of the models are given.



 §.§ The Cucker-Smale model

Let X_i and V_i be the position and velocity of the i-th particle with unit mass, and ψ(| X_j -X_i|) be the communication weight between the j-th and i-th particles. The Cucker-Smale model reads as the following

    {d/dt X_i(t) =    V_i(t),   i=1,⋯, N,
    d/dt V_i(t) =   κ/N-1∑_j=1^N 
            ψ(|X_j(t)-X_i(t)|)(V_j(t)-V_i(t)),
        .

where κ is the nonnegative coupling strength and ψ satisfies  positivity, boundedness, Lipschitz continuity and mononticity conditions, i.e., there exist positive constants ψ_0, ψ_M  > 0  such that 

    0< ψ_0 ≤ψ(r) ≤ψ_M, ∀ r≥0;   ψ_Lip<∞;   (ψ(r_1)-ψ(r_2))(r_1-r_2)≤ 0,  r_1,r_2 ∈ℝ_+.

Without loss of generality, we set ψ_M = 1 in this paper for convenience. 
Under the condition (<ref>) of ψ, it is observed in <cit.> that the total momentum is conserved as a constant and the total energy is nonincreasing along the Cucker-Smale flow. Actually,

    supposing that {(X_i,V_i)} is the solution of  system (<ref>), then for any t>0, one has
    
    d/dt∑_i=1^N V_i (t) = 0,   d/dt∑_i=1^N |V_i(t)|^2 = -κ/N-1∑_i,jψ(|X_j(t) - X_i(t)|) |V_j (t) - V_i (t) |^2.




A key observation of the Cucker-Smale model, given by Ha et al. in <cit.>, is that the standard deviations of particle phase-space positions are dominated by SDDI (the system of dissipative differential inequalities):

    | d𝒳/dt| ≤𝒱,   d𝒱/dt≤ -ϕ (𝒳) 𝒱,

where (𝒳,𝒱) are nonnegative functions and ϕ is a nonnegtive measurable function. This is a useful tool to analyze the Cucker-Smale model. In this paper, we use a variant  of (<ref>) (Lemma <ref>) in the proof of our main result. One can find its proof in Lemma 3.1 in <cit.>.

    Suppose that two nonnegative Lipschitz functions 𝒳 and 𝒱 satisfy the coupled differential inequalities:
    
    | d𝒳/dt| ≤𝒱,   d𝒱/dt≤ -α𝒱 + γ e^-α t𝒳,    a.e.  t>0,

    where α and γ are positive constants. Then, 𝒳 and 𝒱 satisfy the uniform bound and decay estimates:
    
    
    𝒳(t) ≤2M/α (𝒳(0) + 𝒱(0)),   𝒱(t) ≤ M (𝒳(0) + 𝒱(0)) e^-α t/2,    t≥ 0,

    where M is given by
    
    M:= max{1,2γ/α e} + 8γ/α^3 e^3.





 §.§ The mean-field limit of the Cucker-Smale model


The corresponding evolution for the distribution function f(x, v, t) was first derived by Tadmor, etc. in <cit.> by the BBGKY hierarchy (e.g. <cit.>), which follows

    ∂_t f + v·∇_x f+ ∇_v · [ξ[f]f]=0,

where the operator ξ is defined by

    ξ[f](x,v,t) = κ∫∫ψ(|x-y|)(w-v)f(y,w,t)  dwdy.

It is clear that 

    ∂/∂ t∫∫ vfdxdv =0.

Similar with energy-decreasing in (<ref>) in the microscopic scale, the mean-field limit has the following property: 
 
Let f_0 be compactly supported in ℝ^2d with x = ∫ x f_0 (x,v) dxdv and v = ∫ v f_0 (x,v) dxdv. Define C_x0:=inf{X≥0  |  supp(f_0)⊂ B(x,X) ×ℝ^d } and C_v:=inf{V≥0  |  supp(f_0)⊂ℝ^d × B(v,V)}, where B(x,X) designates the ball of center x and radius X in ℝ^d. Let f_t be the unique solution of  (<ref>) with f(0) = f_0. If the initial data satisfies

    C_v < κ∫_C_x0^∞ψ (2s) ds,

then there exists C_x>0 such that 
    
    supp[f_t]⊂ B(x + tv, C_x) × B(v, C_v e^-κψ(2C_x) t).



    Under  assumption (<ref>) of ψ,  one has ψ(2C_x)≥ψ_0. Actually, the assumption of the lower bound ψ_0 is for the convenience of the proof and can be relaxed if (<ref>) is satisfied by both discrete and continuous systems, guaranteeing the flocking in the Cucker-Smale model.




 §.§ The Wasserstein distance


To introduce the topology of the space of probability measures, here we recall the definitions of the Wasserstein space <cit.>. For a metric space 
 (ℝ^d,ρ) and q≥1, define the Wasserstein space of order q by

    𝒫_q,ρ (ℝ^d):= {μ∈𝒫(ℝ^𝕕); ∫_ℝ^dρ(x,0)^q ν (dx)< ∞},

endowed with the Wasserstein metric 

    W_q,ρ(μ,ν) := ( inf_π∈Π (μ,ν){∫_ℝ^d ×ℝ^dρ(x,y)^q π (dx,dy)})^1/q,

where μ,ν∈𝒫_q (ℝ^d) and Π (μ,ν) is the set of probability measures on ℝ^d ×ℝ^d with marginals μ and ν respectively. The W_q metric, or distance, induces a kind of weak topology and measures the closeness between distributions. We use the notation W_q,p when ρ(x,y):= x-y _p and typically abbreviate W_q = W_q,2. 
For convenience in notation, we simplify · as |·| without ambiguity.


Recall the Jordan decomposition for a signed measure |μ| := μ^+ + μ^-. One can control the W_q distance by the weighted total variation by the following lemma (see e.g. Theorem 6.15 in <cit.>), which helps the proof of our main theorems, i.e., Theorem <ref> and Theorem <ref>, in Section <ref>.

    Let μ and ν be two probability measures on a Polish space (𝔛, d). Let q∈ [1,∞) and x_0 ∈𝔛. Then 
    
    W_q (μ,ν) ≤ 2^1-1/q( ∫ d(x_0,x)^q d |μ-ν|(x) )^1/q.





 §.§ The Random Batch method and its mean-field dynamics


In each sub-time interval [t_k-1,t_k), we set [i]_k as the batch containing the particle i. The RBM-(approximated) system of (<ref>) becomes:

    {∂_t X_i^R(t) =    V_i^R(t),   i=1,⋯, N,
    ∂_t V_i^R(t) =   κ/p-1∑_j∈[i]_kψ(|X_j^R(t)-X_i^R(t)|)(V_j^R(t)-V_i^R(t)),
            
        .

with initial data (X_i^R(0),V_i^R(0)) = (X_i^in,V_i^in), i = 1,⋯,N. See Algorithm <ref>.









In the N→∞ limit, the N-particle system is then reduced to a p-particle
subsystem described by the following system for t∈ [t_k,t_k+1):

    {∂_t X_i(t) =    V_i(t),   i=1,⋯, N,
    ∂_t V_i(t) =   κ/p-1∑_j=1^p 
            ψ(|X_j(t)-X_i(t)|)(V_j(t)-V_i(t)),
        .

with Z_i:=(X_i,V_i) and Z_i (t_k) being i.i.d drawn from f(·; t_k). Here f(·; t_k) is the law of Z_1(t_k^-) for k≥ 1 and f(·;0) equals the initial distribution f_0 . We impose Z_1 (t_k^-) = Z_1 (t_k^+).  Except the individual particle 1, the rest p-1 particles are sampled from an infinite pool of independent particles from particle 1 at each time step t_k. This process becomes the mean-field limit model of the RBM system (<ref>), and one may write the following mean-field limit for the RBM in terms of the probability distribution as shown in Algorithm <ref>, while (<ref>) becomes its microscopic description.










§ THE MAIN RESULT


In this paper, we show the conservation law and the mean-field limit of the Random-Batch Cucker-Smale model. First, we show the conservation of momentum and dissipation of kinetic energy of the  mean-field limit system of RBM in Section <ref>. Then, we analyze the procedure (b) and (c) in Section <ref>. 


Assume that the initial data of Algorithm <ref> and Algorithm <ref> obey the same law f_0(x,v)dxdv. For notational convenience, recall x:= ∫ x f_0(x,v) and v:=∫ vf_0 (x,v); and set z_i:=(x_i,v_i) for each i and 

    f(x,v;t):= ∫_ℝ^2d(p-1) f^(p)(⋯ ; t^-)dx_2 ⋯ dx_p dv_2⋯ dv_p = f_t(x,v).

Note that f is defined on ℝ^2d in Algorithm <ref> and f^(p) is defined on ℝ^2dp. Also, the definition of f is consistent with that in Algorithm <ref> at t=t_k.

In the following, we suppose that f_0 is compactly supported and then there exists a constant R>0 such that supp[f_0]⊂ B(0,R), where B(0,R) is a ball with radius R centered in the origin. Without loss of generality, we set v=0 as a standardization of the initial momentum.




 §.§ Basic characteristics of the mean-field limit system of RBM


Here, we show that the two basic characteristics: conservation of momentum and dissipation of kinetic energy, as observed in the Cucker-Smale model (<ref>) and the RBM system (<ref>), also holds for the RBM mean-field system described in Algorithm <ref>.

 
For t∈ (t_k,t_k+1), it holds
    
    ∂/∂ t∫_ℝ^2d v f(x,v;t) dx dv = 0.

In particular, since v=0, ∫ vf(x,v;t) = 0 for any t∈ [0,T].



By the definition of f, one has

    ∂/∂ t∫_ℝ^2d vf(x,v;t)dxdv =    ∫_ℝ^2d∫_ℝ^2d(p-1) v ∂/∂ t f^(p) (x,·, v, ·;t^-)dxdx_2⋯ dx_p dvdv_2⋯ dv_p
    
            =    ∫_ℝ^2dp v_1 [ -∑_i=1^p ∇_x_i· v_i f^(p)(⋯ ; t) - ∑_i=1^p ∇_v_i·ξ_i f^(p)(⋯ ;t)]dz_1⋯ dz_p
    
            =    κ/p-1∑_j=2^p∫_ℝ^2dpψ (|x_j - x_1|)(v_j-v_1) f^(p)_t(z_1,⋯ ,z_p)dz_1⋯ dz_p
    
            =     0.

The last equality is derived from the fact that ψ (|x_j - x_1|)(v_j-v_1)f^(p)_t is an odd function on (z_j,z_1).





Note that f(x,v;t_k^-) = f(x,v;t_k) = f(x,v;t_k^+). Then by induction one has ∫ vf(x,v;t) = v.


 
For any t∈ (t_k,t_k+1),
    
    ∂/∂ t∫_ℝ^2d |v|^2 f(x,v;t)dxdv ≤ 0.

In particular, M_2^v f_t ≤ M_2^vf_0, where M_2^v f_t:=∫_ℝ^2d |v|^2 f(x,v;t)dxdv. Moreover, since 0<ψ_0≤ψ, it holds
    
    M_2^v f_k+1≤ M_2^v f_0(e^-2κψ_0 τ + 8κ^2 ψ_0/p-1τ^2 )^k .



    
    For any t∈ (t_k,t_k+1),
        
    ∂/∂ t M_2^v f_t =    ∫_ℝ^2dp |v_1|^2 ( -∑_i=1^p ∇_x_i· (v_i f^(p)_t) - ∑_i=1^p ∇_v_i· (ξ_i f^(p)_t) )
    
                    =   ∫_ℝ^2dpκ/p-1∑_j=1^pψ(|x_1-x_j|)(v_j-v_1)· 2v_1 f^(p)_t
    
                    =    -∫_ℝ^2dpκ/p-1∑_j=1^pψ(|x_1-x_j|)|v_j-v_1|^2 f^(p)_t
    ≤    0.

        The last equality is derived by the symmetry of ψ and f_t^(p). By the continuity of f, one has M_2^v f_t ≤ M_2^vf_0. Since ψ has a positive lower bound ψ_0, furthermore, it holds
        
    ∂/∂ t M_2^v f_t ≤    -∫_ℝ^2dpκ/p-1∑ _j=1^p ψ_0 |v_j-v_1|^2 f^(p)_t
    
                    =    -2κψ_0 ∫_ℝ^2d |v|^2 f_t(x,v) dxdv + 2κψ_0 ∫_ℝ^2dp v_1· v_2 f^(p)_t,

        by the symmetry of f^(p)_t. For the second term in (<ref>), one has
        
    ∂/∂ t   ∫_ℝ^2dp v_1 · v_2 f^(p)_t
    
                    =    κ∫_ℝ^2dpv_2/p-1·∑_k 1^p ψ(|x_1-x_k|)(v_k-v_1)f^(p)_t + v_1/p-1·∑_k 2^p ψ(|x_2-x_k|)(v_k-v_2) f^(p)_t
    
                    =    2κ∫_ℝ^2dpv_2/p-1·∑_k 1^p ψ(|x_1-x_k|)(v_k-v_1)f^(p)_t.

        For the last integration in (<ref>), by the indistinguishability of the particles, one has  
        
    ∫_ℝ^2dp v_2·∑_k 1^p ψ(|x_1-x_k|)(v_k-v_1)f^(p)_t
       = ∫_ℝ^2dp v_2·∑_k 1,2^p ψ(|x_1-x_k|)(v_1-v_k)f^(p)_t + ∫_ℝ^2dp v_2·ψ(|x_1-x_2|)(v_2-v_1)f^(p)_t
       = ∫_ℝ^2dp v_2·∑_k 1^p ψ(|x_1-x_k|)(v_1-v_k)f^(p)_t 
                    +∫_ℝ^2dp v_2·ψ(|x_1-x_2|)(v_2-v_1)f^(p)_t
       + ∫_ℝ^2dp v_2·ψ(|x_1-x_2|)(v_2-v_1)f^(p)_t
       = -∫_ℝ^2dp v_2·∑_k 1^p ψ(|x_1-x_k|)(v_k-v_1)f^(p)_t +∫_ℝ^2dp |v_2-v_1|^2 ψ(|x_1-x_2|) f^(p)_t.

        Hence, recalling ψ≤ 1, it holds
        
    ∫_ℝ^2dp v_2·∑_k 1^p ψ(|x_1-x_k|)(v_k-v_1)f^(p)_t
            = 1/2∫_ℝ^2dp |v_2-v_1|^2 ψ(|x_1-x_2|)f^(p)_t≤ 2 M_2^v f_t.

        Then it yields from (<ref>) that 
        
    ∂/∂ t∫_ℝ^2dp v_1 · v_2 f^(p)_t≤4κ/p-1 M_2^vf_t_k-1.

        By combining (<ref>) and (<ref>), it holds
        
    ∂/∂ tM_2^vf_t    ≤ -2κψ_0 M_2^vf_t + 2κψ_04κτ/p-1M_2^vf_t_k-1 +  2κψ_0∫ v_1 · v_2 f_t_k-1^(p)
       =  -2κψ_0 M_2^vf_t + 2κψ_04κτ/p-1M_2^vf_t_k-1
       ≤  -2κψ_0M_2^v f_t + 8κ^2ψ_0/p-1τ M_2^vf_t_k-1.

        By Grönwall's inequality, one has
        
    M_2^v f_k≤ M_2^v f_t_k-1(e^-2κψ_0 τ + 8κ^2ψ_0/p-1τ^2 ) ≤ M_2^v f_0(e^-2κψ_0 τ + 8κ^2 ψ_0/p-1τ^2 )^k .

    
    


    
    
    
    
    
    
    
    
    
    
    
    
    





 §.§ The main theorems



First we discuss the Wasserstein distance between the distribution of the RBM of the N-particle system (<ref>) and the distribution of the N→∞ system (<ref>). 

Define the operator 𝒬_N^(k) on the probability measure space 𝒫(ℝ^2d) as follows. Let Z_i^R(0) be i.i.d drawn from f_0. Corresponding to Algorithm <ref>, we define 

    𝒬_N^(k)(f_0):= Law(Z_1^R(t_k)),

where Law(Z_1^R) means the law of Z_1^R.  Conditioning on a specific sequence of random batches, the particles are not exchangeable. However, when considering the mixture of all possible sequences of random batches, the laws of Z_i^R(t_k) are identical.

In addition, we define 𝒬_∞ as one time step evolution in Algorithm <ref>. Thus one arrives at 

    𝒬_∞^k (f_0) = 𝒬_∞∘⋯∘𝒬_∞(f_0),  (k copies),

which is expected to be the mean field limit of the RBM after k steps. Note that the 𝒬_∞ dynamics can fully determine the probability transition, knowing the marginal distribution of Z_1, while knowing only the marginal distribution is not enough for the dynamics of 𝒬_N^(k). The joint distribution must be known in the latter case. 


Theorem <ref> gives an estimate between 𝒬_∞^k (f_0) and 𝒬_N^(k)(f_0) on the W_q distance. In this theorem, we introduce a quantity ϵ_k∈ℝ, which was first introduced in <cit.>. Plainly speaking, the ϵ_k means the probability that the particle i is not “clean", where a particle i is clean at t_k^- means its batchmates at t≤ t_k were mutually independent and independent to the particle i when they interacted. For better organization, we give the definition of "clean"   and more details in Section <ref>.


    For f_0 with the compact support supp [f_0], it holds that 
    
    W_q(𝒬_∞^k (f_0),𝒬_N^(k) (f_0))≤ C (1+t_k) ϵ_k^1/q,

    with q ∈ [1,∞) and C=C(κ,ψ_0,R). For fixed k, it holds
    
    lim_N→∞ϵ_k =0,

    with ϵ_k≤𝒪(N^-1).







    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    



    Then, we consider the limit dynamics given by the operator 𝒬_∞ and how 𝒬_∞^n (f_0) approximates the dynamics of the Fokker-Planck equation (<ref>), i.e. the procedure (c) in (<ref>). We prove that the W_q distance can be controlled by τ with a smooth initial distribution f_0 with a compact support uniformly in time. 
    
     
        Let f be a solution to the Fokker-Planck  equation (<ref>) with initial condition f(0) = f_0. 
        It holds
        
        
        
        
        the uniform-in-time estimate
        
    sup_n W_q (𝒬_∞^n (f_0), f(nτ))≤ Cτ,

        with q∈[1,∞)and C=C(κ,ψ_0,R).
    

    







 §.§ Some helpful discussions



In addition, we give some  discussions on Lemma <ref> that will be used for the proof of Theorem <ref>, and on the commutation of (<ref>).





  §.§.§ Approximation using PDE analysis


Similar with [<cit.>, Lemma 4.4], one can obtain an 𝒪(τ^2/q) bound on the W_q distance between 𝒬_∞ (f_t_k) and S(f_t_k) in Proposition <ref>. Here S(f_t_k) denotes the law of (<ref>) in Section <ref>.


     Let f be a solution to the Fokker-Planck  equation (<ref>) with initial condition f(0) = f_0 ∈ (C^2∩ W^2,∞)(ℝ^2d). Suppose that the interaction kernel ψ satisfies 
    sup_r≥ 0 (|ψ^'(r)| + |ψ^''(r)|) ≤ϵ_ψ
 with a constant ϵ_ψ. It holds
    
    W_q (𝒬_∞ (f_t_k),S(f_t_k)) ≤ C(κ,ϵ_ψ , R, d) τ^2/q,    q∈ [1,∞).


The proof relies on the PDE analysis on the evolution equations on 𝒬_∞ (f_t_k) and S(f_t_k), using the corresponding operator groups. There are two limitations on this method. One is that more regular assumptions on both the interaction kernel and initial data are needed to guarantee the existence of a unique classical solution f_t ∈ C^2([0,T)×ℝ^2d) for (<ref>). In addition, if one uses the estimate obtained by Lemma <ref> with the total variance bound, the order of τ is 2/q. 
A detailed proof of Proposition <ref> is given in Appendix <ref>. 





  §.§.§ The mean-field limit of the Cucker-Smale system
 

Up to now, we have given a brief analysis of the path ((b),(c)) in the lower right corner of (<ref>). Here, we discuss the upper left corner path in the sense of the Wasserstein distance as well. Denote by 

    μ_t^N:=∑_i=1^N 1/Nδ_z_i(t)
 the empirical measure and μ_t^R the empirical measure derived by the RBM with μ_0^R = μ_0^N.



 
    For the path from (a) to (d) in (<ref>), it holds 
    
    𝔼 W_2 (f_t,μ_t^R)     ≤ C(κ, R,ψ)√(τ/p-1) +
            C(R)·{   N^-1/4     if  d<4,
       N^-1/4√(log(1+N))     if  d=4,
       N^-1/d     if  d>4.
            .



    We first consider the procedure (a). For the first τ→ 0 limit, by the definition of the W_q distance, one has
    
    W_2^2 (μ_t^R, μ_t^N) 
            ≤1/N∑_i=1^N (|X_i^R - X_i|^2 + |V_i^R - V_i|^2).

    Based on [<cit.>, Theorem 3.2], one has
    
    𝔼(1/N∑_i-1^N |V_i^R (t) - V_i (t)|^2)
            ≤ C(1/p-1-1/N-1)τ + Cτ^2 + C(1+τ) exp(-κψ_0 t),

    for some constant C depending on κ, R and ψ.
    Note that
    
    d/dt∑_i=1^N |X_i^R - X_i|^2 ≤
        2√(∑_i=1^N |X_i^R - X_i|^2)√(∑_i=1^N |V_i^R - V_i|^2),

    by the Cauchy-Schwarz inequality and
    
    √(∑_i=1^N |X_i^R - X_i|^2)≤√(∑_i=1^N |V_i^R - V_i|^2) T.

    It thus holds
    
    𝔼 W_2 (μ_t^N,μ_t^R) ≤𝔼1/√(N)√(∑_i=1^N |X_i^R - X_i|^2 + ∑_i=1^N |V_i^R - V_i|^2)
    ≤√(C τ(1/p-1-1/N-1))(1+T).
.

    Next, for the procedure (d), by the estimate of [<cit.>, Corollary 1], one has W_q (μ_t^N,f_t) ≤ C(κ,R)W_q(μ_0^N,f_0). 
    In addition, according to the convergence rate of the empirical measure in [<cit.>, Theorem 1], with f_0 compactly supported, one has 
    
    𝔼 W_q^q (f_0,μ_0^N) ≤ C(R)·{   N^-1/2     if  q>d/2,
       N^-1/2log(1+N)      if  q=d/2,
       N^-q/d     if  q<d/2.
        .


    Therefore, under the same assumption of Theorem <ref> and Theorem <ref>, for the path from (a) to (d), one has 
    
    𝔼 W_2 (f_t,μ_t^R)    ≤𝔼[W_2 (μ_t^N,μ_t^R) + W_2 (f_t,μ_t^N)]    
             ≤ C(κ,R,ψ)√(τ/p-1) + C(κ,R) 𝔼 W_2 (f_0,μ_0^N)
       ≤ C(κ,R,ψ)√(τ/p-1) + C(κ,R)(𝔼 W_2^2 (f_0,μ_0^N))^1/2.











§ PROOF OF THEOREM <REF>


    In this section, we prove Theorem <ref> by an observation in combinatorics. 
    Recalling the critical quantity ϵ_k we mentioned above Theorem <ref>, we first give the Lemmas <ref>-<ref>, for the proof of Theorem <ref>, and show the mathematical definition of ϵ_k in Section <ref>.
    The proof of Lemmas <ref>-<ref> are shown in Sections <ref>-<ref> respectively.
    
    Finally, Theorem <ref> will be proved in Section <ref> by combining Lemma <ref>, Lemma <ref> and an estimate on the diameters of the compact support in space and velocity, shown in Lemma <ref>.


    
    
        Under the assumption and notation of Theorem <ref>, it holds
            
    |𝒬_∞^k (f_0) - 𝒬_N^(k) (f_0)|_TV≤ 2ϵ_k,

        where ϵ_k is a positive constant  with respect to N.
    
    
    
    
        For any fixed k, with the assumption and notation of Theorem <ref>, it holds that ϵ_k ≤𝒪(N^-1).
    


    




 §.§ Prerequisites

    To give a detailed explanation of ϵ_k in Theorem <ref>, we introduce the concept “clean” in Definition <ref>. First, for each particle i, we define a sequence of lists {L_i^(k)} associated with i, given as follows:
    
        
  * L_i^(0) = {i};
        
  * For k≥ 1, let 𝒞_q^(k-1) be the batch that particle i stays in for t∈[t_k-1,t_k), then
        
    L_i^(k) = ∪_j∈𝒞_q^(k-1) L_j^(k-1).

    
    Here, L_i^(k) can be viewed as the particles that have impacted i for t < t_k. Clearly, a particle i_1∈ L_i^(k) might not have been a batchmate of i. It could have been a batchmate of i_2, and then i_2 was a batchmate of i at some time. The important observation is that if L_i^(k) and L_j^(k) do not intersect for a given sequence of random batches, then particles i and j are independent at t_k^-. Note that we are not claiming all particles in L_j^(k) are independent of those in L_j^(k) at t_k^-. In fact, it is possible that some i_1 ∈ L_i^(k) and j_1 ∈ L_j^(k) are in the same batch on [t_k-1, t_k). However, i_1 and j_1 must be independent at the times when they were added to the batches that eventually impact i,j at t_k^-. Then we are motivated to define the following.
    
    
        We say the particle i is clean on [t_k,t_k+1) if the batch 𝒞_q^(k) that contains i at t_k^+ satisfies the following:
        
            
  * any j∈𝒞_q^(k) is clean at t_k^-;
            
  * any j,ℓ∈𝒞_q^(k) with jℓ, L_j^(k) and L_ℓ^(k) do not intersect.
        
    
    In other words, a particle i is “clean" at t_k^- if its batchmates at t<t_k were mutually independent and independent to i when they interacted.
    
   
    Let A_k denote the set of particles that are clean at t_k^-. Then
    
    A_0 = {1,⋯, N}.

    For k≥ 1, one has 
    
    A_k = { i ∈ A_k-1: i∈𝒞_q^(k-1), ∀ j,ℓ∈𝒞_q^(k-1),  jℓ ,
    
            j∈ A_k-1, ℓ∈ A_k-1,  L_j^(k-1)∩ L_ℓ^(k-1) = ∅}.

    Now we define
    
    ϵ_k := ℙ(1∉ A_k).

    Note that by symmetry, ϵ_k is also the probability that particle i is not clean.









 §.§ Proof of Lemma <ref>


    We first introduce Lemma <ref>.
     
        Consider a fixed sequence of divisions of random batches {𝒞^(ℓ)}_ℓ≤ k-1.
        
            
  * It holds that
                
    |L_i^(k)|≤ p^k,

                and the particle i is clean at t_k^- if and only if the equality holds.
            
  * The distribution of Z_i (Recall Z_i := (X_i,V_i).) for a clean particle i at t_k^- is 𝒬_∞^k (f_0).
        
        Here the symbol |A| means the cardinality of a set A.
    
    
        The proof is a straightforward induction. Here, let us just briefly mention the proof of the second claim. For k=0, the statement is trivial. Now, suppose the statement is true for all k ≤ m-1. We now consider k = m.
        
        For the given sequence of random batches {𝒞^(ℓ)}_ℓ≤ k-1, that a particle i is clean at t_m^- means that on [t_m-1, t_m), the particles in the batch for i are independent of i at t_m-1. By the induction assumption, the distribution of one particle at t_m-1 is given by 𝒬_∞^m-1(f_0). By the independence, the joint distribution of them at t_m-1 is therefore
        
    f^(p) (⋯, t_m-1) = 𝒬_∞^m-1(f_0)^⊗ p.

        From t_m-1 to t_m, the evolution of the joint distribution obeys  equation (<ref>). Hence, at t_m^-, the distribution of particle i is given by 𝒬_∞^m(f_0). 
    



    Now we begin to prove Lemma <ref>.

    
        By Lemma <ref>, one has
        
    𝒬_N^(k) (f_0) = ℙ(1∈ A_k)𝒬_∞^k (f_0)
                + ℙ(1∉ A_k)ν_k,

        for some probability measure ν_k. To see this, consider all possible sequences of random batches. Only the first k divisions of batches (i.e. ones at t_0,⋯, t_k-1) will affect the distribution at t_k. This subsequence (the first k divisions) can take only finitely many values, and let {c^ℓ}_ℓ≤ k-1 be such values. Then, for any E⊂ℝ^d that is Borel measurable,
        
    𝒬_N^(k) (f_0)[E] = ∑_{𝒞^ℓ = c^ℓ , ℓ≤ k-1}ℙ(𝒞^ℓ = c^ℓ ,  ℓ≤ k-1)
                ℙ(Z_1 ∈ E  | 𝒞^ℓ = c^ℓ ,  ℓ≤ k-1 ).

        Lemma <ref> tells us if {c^ℓ}_ℓ≤ k-1 is a value such that particle 1 is clean, then
        
    ℙ(Z_1 ∈ E  | 𝒞^ℓ = c^ℓ ,  ℓ≤ k-1 ) = 𝒬_∞^k (f_0)[E].

        Hence,
        
    𝒬_N^(k) (f_0)[E]    =  ℙ(1∈ A_k)𝒬_∞^k (f_0)[E] 
         + ∑_{𝒞^ℓ = c^ℓ , ℓ≤ k-1},  1∉ A_kℙ(𝒞^ℓ = c^ℓ ,  ℓ≤ k-1)
                ℙ(Z_1 ∈ E  | 𝒞^ℓ = c^ℓ ,  ℓ≤ k-1 )
       = ℙ(1∈ A_k)𝒬_∞^k (f_0)[E] + ℙ(1∉ A_k)ν_k (E),

        
        
        
        
        
        
        with
        
    ν_k (E) = ∑_{𝒞^ℓ = c^ℓ , ℓ≤ k-1},  1∉ A_kℙ(𝒞^ℓ = c^ℓ ,  ℓ≤ k-1)/ℙ(1∉ A_k)ℙ(Z_1 ∈ E  | 𝒞^ℓ = c^ℓ ,  ℓ≤ k-1 ).

        Clearly, ν_k is a convex combination of some conditional marginal distributions of Z_1, each being Law(Z_1) conditioning on a particular sequence of batches for {1∉ A_k}. Hence ν_k is a probability measure. 
        
        By (<ref>), it holds that
        
    |𝒬_∞^k(f_0) - 𝒬_N^(k) (f_0)| ≤ 
                (1-ℙ(1∈ A_k))𝒬_∞^k (f_0) + ℙ(1∉ A_k)ν_k
                = ϵ_k (𝒬_∞^k (f_0) + ν_k).

        Therefore, the total variation distance between the two measures is controlled by
        
    |𝒬_∞^k(f_0) - 𝒬_N^(k) (f_0)|_TV≤ 
                (1-ℙ(1∈ A_k)) + ℙ(1∉ A_k) = 2ϵ_k.

    




    
    
    




 §.§ Proof of Lemma <ref>

    Furthermore, we provide the proof of Lemma <ref> which is a simple variant of [<cit.>, Theorem 3.2].

    
        First of all, clearly, one has
        
    ϵ_0  = 1-1 = 0.

        Now, one can do induction on k. Assume
        
    lim_N→∞ϵ_k = 0,

        and then consider the batches for t_k → t_k+1^-. Assume the batch for particle 1 is 𝒞_q^(k). Denote
        
    B_k = {∀ j, ℓ∈𝒞_q^(k),  jℓ,   L_j^(k)∩ L_ℓ^(k) = ∅}.

        Let ℬ := 𝒞_q^(k)\{1} be the set of other particles that share the same batch with particle 1. Then, by definition of A_k+1, it holds
        
    ℙ (1∈ A_k+1) = ∑_j_1,⋯, j_p-1ℙ(ℬ={j_1,⋯, j_p-1})×
    ℙ(B_k ∩{1∈ A_k}∩_ℓ =1^p-1{j_ℓ∈ A_k} | ℬ= {j_1,⋯, j_p-1}).

         Denote E:= {ℬ={j_1,⋯, j_p-1}}, where we omit the dependence in j_ℓ, 1≤ℓ≤ p-1 for notational convenience. Conditioning on B_k ∩ E (i.e., provided that the event B_k ∩ E happens), whether the particles are clean or not is independent. Hence,
         
    ℙ(B_k ∩{1∈ A_k}∩_ℓ =1^p-1{j_ℓ∈ A_k} |  E) 
    
                 = ℙ(B_k | E))ℙ({1∈ A_k}∩_ℓ = 1^p-1{j_ℓ∈ A_k}|E,B_k)
    
                 = ℙ(B_k|E)Π_ℓ=1^pℙ(j_ℓ∈ A_k|E,B_k),

         where one has set j_p =1. Moreover,
         
    ℙ(j_ℓ∈ A_k|E,B_k) = ℙ({ j_ℓ∈ A_k }∩ E∩ B_k)/ℙ(E∩ B_k) = ℙ({1 ∈ A_k }∩ E∩ B_k)/ℙ(E∩ B_k) = ℙ({1 ∈ A_k }∩ B_k)/ℙ(B_k).

         The second and the last equalities are due to symmetry. For the last equality, ℙ({ℬ = {j_1⋯,j_p-1}}∩ B_k) should be equal for all possible j_1,⋯,j_p-1, and the same  is true for the numerator. This actually is a kind of independence. Hence, eventually due to the fact
         
    ∑_j_1,⋯, j_p-1ℙ (ℬ = {j_1,⋯, j_p-1}) ℙ(B_k | ℬ = {j_1,⋯, j_p-1}) = ℙ (B_k),

        one has
        
    1-ϵ_k+1 = ℙ (1∈ A_k+1) ≥ℙ (B_k) (1-ϵ_k 
                )^p.

        Hence, it suffices to show
        
    lim_N→∞ℙ (B_k) =1.

        To get an estimate for this, we consider the following equivalent way to construct L_i^(k): one starts with L_i ←{i} and repeat the following k times:
        
            
  * Set L_tmp← L_i and A=∅.
            
  * Loop the following until L_tmp is empty.
            
                
  (a) Pick a particle i_1 ∈ L_tmp, then choose p-1 particles from {1,⋯,N}\ A ∪{i_1} denoted by { i_2,⋯ , i_p}.
                
  (b) Set L_i← L_i ∪{ i_2, ⋯, i_p}.
                
  (c) Set A← A ∪{ i_1, i_2, ⋯, i_p}.
                
  (d) Set L_tmp← L_tmp\{ i_1, i_2, ⋯, i_p}.
            
        
        In the above, we are actually looking back from t_k-1. In the j-th iteration, we are constructing batches at t_k-j. Hence, this is an equivalent way to construct L_i^(k).

        Now, we estimate ℙ(B_k) by constructing the lists L_j_ℓ^(k): 1≤ℓ≤ p for j_ℓ∈𝒞_q^(k) using the above procedure. Consider that the lists for j_1,⋯,j_ℓ -1 have been constructed, which have included at most (ℓ -1)p^k particles. Now, for L_j_ℓ^(k) not to intersect with the previous lists, one has to choose particles from {1,⋯,N}\ [∪_z=1^ℓ -1 L_j_z^(k)∪ A ∪{i_1}] in 2.(a) step. Conditioning on the specific choices of L_j_ℓ^(k): 1≤ℓ≤ p and A with 
        
    N_1:= |L_j_ℓ^(k)∪ A∪{i_1}|,   N_2:= |A|,

        this probability is controlled from below by
        
    N- N_1p-1/N-1- N_2p-1≥N- 1-ℓ p^kp-1/N-1p-1.

        Hence, as N→∞,
        
    ℙ(B_k)≥Π_ℓ=1^p [N- 1-ℓ p^kp-1/N-1p-1]^k = 1- 𝒪(N^-1).

        Hence, lim_N→∞ϵ_k+1=0 and the claim follows.
    
    





 §.§ Proof of Theorem <ref>

    


    
    
    Now we can prove Theorem <ref>.
    
    Recall that the TV distance is defined by
    
    |μ - ν|_TV := sup_A∈ℱ |μ(A) - ν(A)|,

    for two probability measures μ, ν defined on certain sample space Ω with events ℱ. 


    We denote the diameters of the compact support in the spatial and velocity variables at time t by 𝒟_X(t) and 𝒟_V(t), respectively, i.e.
    
    𝒟_X(t) := max_i,j| X_i-X_j|,   𝒟_V(t) := max_i,j| V_i-V_j|.

    
    For the RBM system (<ref>), Ha et al. <cit.> gave an estimate on 𝒟_X^R and 𝒟_V^R shown in Lemma <ref>. This estimate works on any fixed sequence of batches and we provide the proof in Appendix <ref> for completeness.
    




        
        Let X^R,V^R be a solution of the RBM model (<ref>) with initial data f_0 satisfying 
        
    𝒟_X(0) + 𝒟_V(0)<∞.

        Then one has
        
    𝒟_V^R (t) ≤𝒟_V(0),   𝒟_X^R (t)≤𝒟_X(0) + 𝒟_V(0)t,   ∀ t ≥ 0.

    



    
    

    
        Denote z:= (x,v), Z^R_i:=(X_i^R,V_i^R), and |z|^q:=|x|^q + |v|^q.
        By Lemma <ref>, taking μ = 𝒬_∞^k (f_0) and ν = 𝒬_N^(k)(f_0), one has
        
    W_q (𝒬_∞^k (f_0),𝒬_N^(k)(f_0)) ≤ 2^1-1/q(∫_ℝ^2d |z|^q d | 𝒬_∞^k (f_0)-𝒬_N^(k)(f_0)|(z))^1/q.


       Note that Lemma <ref> degenerates into the original Cucker-Smale model when p=N. Hence for t∈[0,t_1), any {Z_i^(0)(t)}_i=1^p satisfying (<ref>), one obtains
       
    𝒟_V^(0)(t)≤𝒟_V(0),   𝒟_X^(0) (t) ≤𝒟_X (0) + 𝒟_V(0)t.

       Also, by the assumption of f_0, one has v=0, and therefore x=0. Since (<ref>) holds independent of the choice Z_i, defining 
       
    𝖷(t) :=   max{ |x|  |∃ v,  s.t.  f(x,v;t) >0},
    𝖵(t) :=   max{ |v|  |∃ x,  s.t.  f(x,v;t) >0},

       one has for t∈[0,t_1],
       
    𝖵(t)≤ C,   𝖷(t)≤ C(1+t),

       where C is a constant depending on R. 
       Then for t∈[t_1,t_2], for the new subsystem {Z_i^(1)}_i=1^p with initial distribution f_t_1, one can use Lemma <ref> again to get
       
    𝒟_V^(1)(t)   ≤𝒟_V^(1)(0)≤ 2𝖵(t_1) ≤ C, 
    𝒟_X^(1) (t)    ≤𝒟_X^(1) (0) + 𝒟_V^(1)(t-τ)
               ≤ 2𝖷(t_1) + 2𝖵(t_1)(t-τ)≤ C(1+t),

       and (<ref>) holds for t∈[t_1,t_2].
       By induction, for any t, (<ref>) holds. Therefore |𝖷(t_k)|^q + |𝖵(t_k)|^q ≤ C(1+t_k)^q.       
       
       Also, by Lemma <ref>, it holds |Z^R_i|^q≤ C(1+t_k)^q.
       Then for any z∈supp[| 𝒬_∞^k (f_0)-𝒬_N^(k)(f_0)|], |z|^q≤ C(1+t_k)^q. Therefore, 
       
    ∫_ℝ^2d |z|^q d | 𝒬_∞^k (f_0)-𝒬_N^(k)(f_0)|(z) 
               
               
               
               ≤  C(1+t_k)^q | 𝒬_∞^k (f_0)-𝒬_N^(k)(f_0)|_TV.

       
       By Lemma <ref>, one has 
       
    ∫_ℝ^2d |z|^q d | 𝒬_∞^k (f_0)-𝒬_N^(k)(f_0)|(z) ≤ C(1+t_k)^q ϵ_k.

       Hence W_q (𝒬_∞^k (f_0),𝒬_N^(k)(f_0)) ≤ C(1+t_k)ϵ_k^1/q. Then by Lemma <ref>, we finish the proof.
    


    
        The above analysis is not enough to get the mean-field limit independent of τ and for fixed N, ϵ_k→ 1 as k→∞. This is a limitation of the technique in the paper.
    














§ PROOF OF THEOREM <REF>


    Here gives an outline of the proof of Theorem <ref>. First, we show the stability of 𝒬_∞ in Section <ref>. Second, an auxiliary system is introduced in Section <ref> as a bridge to estimate the W_q distance between 𝒬_∞(f_t_k) and f_t_k+1. 
    
    Last,  the proof of Theorem <ref> is given in Section <ref>.




 §.§ Stability of 


In Lemma <ref> and Lemma <ref>, we give the stability analysis on the operator 𝒬_∞.
 
    Suppose two probability measures μ_1,μ_2 satisfying ∫ vdμ_1=∫ vdμ_2, with compact supports supp[μ_1] , supp[μ_2] respectively, where supp[μ_i]⊂ℝ^d× B(0,R_μ), i=1,2, and the constant R_μ >0. Then it holds
    
    W_q (𝒬_∞ (μ_1) ,𝒬_∞ (μ_2)) ≤ CW_q(μ_1,μ_2),    q=[1,∞),

    where C(κ,ψ,R_μ) is a positive constant.



    Set (X_i,V_i)(0)∼μ_1, and (X_i,V_i)(0)∼μ_2, for i=1, ⋯, p. The key to the proof is to make use of Lemma <ref> to get the contraction property of the velocity difference. 
    
    The two steps are as follows.
    

  
Step 1.
    First, we show an auxiliary estimate on 𝒟_V(t). By following the formulation of [<cit.>, Lemma 2.2], denote
    
    Ψ_ij:= ψ(|X_i-X_j|)/p + ( 1- ∑_j=1^p ψ(|X_i-X_j|)/p)δ_ij,

    where δ_ij is the Kronecker delta. Then Ψ_ij is nonnegative and satisfies
    
    Ψ_ij≥ψ(|X_i -X_j |)/p,   ∑_j=1^p Ψ_ij =1,  ∑_j=1^pΨ_ij(V_j-V_i) = ∑_j=1^pψ(|X_i -X_j |)/p(V_j-V_i).

    We denote V_i = (V_i^k) and obtain the following equality
    
    1/2d/dt   |V_i - V_j|^2
            = 1/2d/dt∑_k=1^d|V_i^k - V_j^k|^2
    
            =   ∑_k=1^d(V_i^k - V_j^k)(V_i^k - V_j^k)
    
            =   ∑_k=1^d κ p/p-1 (V_i^k - V_j^k)[
            ∑_m=1^p Ψ_im(V_m^k - V_i^k) - ∑_m=1^p Ψ_jm(V_m^k - V_j^k)]
    
            =    -∑_k=1^d κ p/p-1 (V_i^k - V_j^k)^2
            +  ∑_k=1^d κ (V_i^k - V_j^k)( ∑_m=1^p (Ψ_im-Ψ_jm)V_m^k)
    
            =    -∑_k=1^d κ p/p-1 (V_i^k - V_j^k)^2
       +  ∑_k=1^d κ p/p-1 (V_i^k - V_j^k)[ ∑_m=1^p (Ψ_im-min{Ψ_im,Ψ_jm} + min{Ψ_im,Ψ_jm} - Ψ_jm)V_m^k].

    For a given time t, one can choose indices i_t and j_t satisfying the relation 
    
    𝒟_V(t) = |V_i_t(t) -V_j_t(t)|,

    and thus for each m=1,2,⋯,p,
    
    ∑_k=1^d (V_i_t^k -V_j_t^k)V_j_t^k ≤∑_k=1^d (V_i_t^k -V_j_t^k)V_m^k ≤∑_k=1^d (V_i_t^k -V_j_t^k)V_i_t^k.

    Combining (<ref>) and (<ref>), one obtains
    
    1/2d/dt|V_i_t - V_j_t|^2       
            ≤     -∑_k=1^d κ p/p-1 (V_i_t^k - V_j_t^k)^2
       +  ∑_k=1^d κ p/p-1 (V_i_t^k - V_j_t^k)V_i_t^k ∑_m=1^p (Ψ_im-min{Ψ_im,Ψ_jm}) 
       -  ∑_k=1^d κ p/p-1 (V_i_t^k - V_j_t^k)V_j_t^k ∑_m=1^p (Ψ_jm -min{Ψ_im,Ψ_jm})
    
            =   -∑_m=1^pκ p/p-1min{Ψ_im,Ψ_jm}|V_i_t - V_j_t|^2
    ≤    - κ p/p-1ψ_0 |V_i_t - V_j_t|^2.

    Note that 𝒟_V is Lipschitz continuous with respect to t and thus it is differentiable almost everywhere. Consider the instant t at which 𝒟_V is differentiable. Then,
    
    d/dt𝒟_V^2 ≤d/dt|V_i_t - V_j_t|^2.

    Combining with (<ref>) and (<ref>), one obtains
    
    |d/dt𝒟_X| ≤𝒟_V,   d/dt𝒟_V ≤ -κ p/p-1ψ_0 𝒟_V,

    which is an SDDI. By Lemma <ref>, there exists a positive constant R_μ such that 
    
    𝒟_V(t)≤ 2e^-κψ_0 tR_μ.




    

  
Step 2.
    
    It can be obtained that 
    
    d/dt𝔼 | V_1 - V_1 |^q =  
               1/pd/dt𝔼∑_i=1^p (∑_k=1^d | V_i^k - V_i^k |^2)^q/2,

    with
    
    d/dt𝔼   (∑_k=1^d | V_i^k - V_i^k |^2)^q/2 
            =  𝔼[ q/2(∑_k=1^d | V_i^k - V_i^k |^2)^q/2-1d/dt∑_k=1^d 
    |V_i^k - V_i^k|^2 ]
    
            =    qκ/p-1𝔼[ |V_i - V_i|^q-2∑_j=1^p ψ(|X_j - X_i|)∑_k=1^d(V_j^k - V_i^k - V_j^k + V_i^k)( V_i^k - V_i^k). 
       .  + ∑_j=1^p 
            |V_i - V_i|^q-2(ψ(|X_j - X_i|) - ψ(|X_j - X_i|))
           ∑_k=1^d(V_j^k - V_i^k)( V_i^k - V_i^k) 
           ].

    Rewrite the right-hand side of (<ref>) as
   
    d/dt𝔼 | V_1 - V_1 |^q
           := I_21 + I_22,

   where
   
    I_21:= qκ/p-1𝔼∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) |V_i - V_i|^q-2 (V_j^k - V_i^k - V_j^k + V_i^k)(V_i^k - V_i^k),

   and 
   
    I_22:= qκ/p-1𝔼∑_i,j=1^p ∑_k=1^d
            (ψ(|X_j - X_i|) - ψ(|X_j - X_i|))
    ·|V_i - V_i|^q-2
           ( V_i^k - V_i^k )(V_j^k - V_i^k).

   First estimate I_21 using the index changing trick:
   
    I_21 =    qκ/p-1𝔼∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) |V_i - V_i|^q-2 [(V_j^k -V_j^k)-( V_i^k - V_i^k )](V_i^k - V_i^k)
    
               =    qκ/p-1𝔼[ ∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) |V_i - V_i|^q-2 (V_j^k -V_j^k)(V_i^k - V_i^k).
       .- ∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) |V_i - V_i|^q-2(V_i^k - V_i^k)^2]
    
               =    qκ/p-1𝔼[ ∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) |V_j - V_j|^q-2 (V_j^k -V_j^k)(V_i^k - V_i^k).
       .- ∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) |V_i - V_i|^q-2(V_i^k - V_i^k)^2]
    
               =     -qκ/p-1𝔼[ ∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|). 
       ·[|V_i - V_i|^q-2 (V_i^k -V_i^k) - |V_j - V_j|^q-2 (V_j^k -V_j^k)]
               .(V_i^k - V_i^k)]
    
               =    qκ/2(p-1)𝔼[ ∑_i,j=1^p ∑_k=1^d  ψ(|X_j - X_i|) .
       .·[|V_i - V_i|^q-2 (V_i^k -V_i^k) - |V_j - V_j|^q-2 (V_j^k -V_j^k)][(V_j^k - V_j^k)-(V_i^k - V_i^k)]].

    One can use the relation 
    
    ∑_k=1^d[|V_i - V_i|^q-2 (V_i^k -V_i^k) - |V_j - V_j|^q-2 (V_j^k -V_j^k)]
            ·.[(V_j^k - V_j^k)-(V_i^k - V_i^k)]] ≤ 0,

    and the condition (<ref>) of ψ to obtain
    
    I_21≤   qκ/2(p-1)ψ_0 𝔼{∑_i,j=1^p ∑_k=1^d 
            [|V_i - V_i|^q-2 (V_i^k -V_i^k) - |V_j - V_j|^q-2 (V_j^k -V_j^k)]
       · [(V_j^k - V_j^k)-(V_i^k - V_i^k)]}
    
            =    qκ/2(p-1)ψ_0 𝔼[∑_i=1^p ∑_k=1^d 
            |V_i - V_i|^q-2 (V_i^k -V_i^k)∑_j=1^p (V_j^k -V_j^k).
       -  ∑_i=1^p ∑_k=1^d 
             p|V_i - V_i|^q-2 (V_i^k -V_i^k)^2
            -  ∑_j=1^p ∑_k=1^d 
             p|V_j - V_j|^q-2 (V_j^k -V_j^k)^2
       +  .∑_j=1^p ∑_k=1^d 
             |V_j - V_j|^q-2 (V_j^k -V_j^k)∑_i=1^p (V_i^k -V_i^k)].

    By the momentum condition 𝔼∑_i=1^p (V_i^k - V_i^k) = 0, it holds
    
    I_21≤  -p^2qκ/p-1ψ_0 𝔼|V_1 - V_1|^q.


   For I_22, one has
    
    I_22≤   qκ/p-1ψ_Lip𝔼[
                ∑_i,j=1^p ∑_k=1^d | |X_j - X_i| - 
                |X_j - X_i | | V_i - V_i|^q-2|V_i^k - V_i^k| |V_j^k - V_i^k|
                ]
    ≤   qκ/p-1ψ_Lip𝔼[
                ∑_i,j=1^p| |X_j - X_i| - 
                |X_j - X_i | | |V_i - V_i|^q-1 |V_j - V_i|
                ],

    by the Cauchy inequality. Then by the inequality | |a-b| - |c-d| | ≤ |a-c| + |b-d|, one has
   
    I_22≤   qκ/p-1ψ_Lip𝔼[
                ∑_i,j=1^p| |X_i - X_i| + 
                |X_j - X_j | | |V_i - V_i|^q-1 |V_j - V_i|
                ]
    ≤   qκ/p-1ψ_Lip𝒟_V(t) 𝔼[p
                ∑_i=1^p |X_i - X_i| | V_i - V_i |^q-1 + 
                ∑_i,j=1^p |X_j - X_j| | V_i - V_i |^q-1]
    ≤   2q p^2κ/p-1ψ_Lip𝒟_V(t) 𝔼[
                 |X_1 - X_1| | V_1 - V_1 |^q-1],

    by Hölder's inequality. Now by combining (<ref>) and (<ref>) and (<ref>), one has
    
    d/dt𝔼| V_1 - V_1 |^q ≤    -p/p-1qκψ_0 
                𝔼|V_1 - V_1|^q 
       + 2p/p-1qκψ_Lip𝒟_V(t)
                𝔼[|X_1-X_1| |V_1 - V_1|^q-1]
    ≤    -p/p-1qκψ_0 
                𝔼|V_1 - V_1|^q 
       + 2p/p-1qκψ_Lip𝒟_V(t)
                (𝔼[|X_1-X_1|^q])^1/q (𝔼|V_1 - V_1|^q)^1-1/q,

    similarly by Hölder's inequality. Since
    
    d/dt (𝔼   |V_1 - V_1|^q)^1/q = 
                1/q(𝔼|V_1 - V_1|^q)^1/q-1d/dt𝔼|V_1 - V_1|^q
    ≤    -pκ/p-1ψ_0 
                (𝔼[|V_1 - V_1|^q_q])^1/q 
                + 2pκ/p-1ψ_Lip𝒟_V(t)
                (𝔼[|X_1-X_1|^q])^1/q.

    and 
    
    d/dt𝔼|X_1 - X_1|^q =    𝔼q/2(∑_k=1^d |X_1^k - X_1^k|^2)^q/2-1∑_k=1^d(X_1^k - X_1^k)· 2(V_1^k - V_1^k)
    ≤   𝔼 q |X_1 - X_1|^q-1 |V_1 - V_1 |
    ≤    q (𝔼|X_1 - X_1|^q)^1-1/q (𝔼|V_1 - V_1|^q)^1/q,

    thus it holds 
    
    d/dt(𝔼|X_1 - X_1|^q)^1/q = 
            1/q(𝔼|X_1 - X_1|^q)^1/q-1d/dt𝔼|X_1 - X_1|^q
            ≤ (𝔼|V_1 - V_1|^q)^1/q.

    Note that 𝒟_V≤ 2 e^-κψ_0 t  R_μ by Step 1. 
    Now one knows that both (𝔼|X_1 - X_1|^q)^1/q and (𝔼|V_1 - V_1|^q)^1/q satisfy Lemma <ref> and then there exists a constant C such that
    
    (𝔼|X_1 - X_1|^q)^1/q(t)
            + (𝔼|V_1 - V_1 ^q)^1/q(t)
            ≤ C [(𝔼|X_1 - X_1 |^q)^1/q(0)
            + (𝔼|V_1 - V_1 |^q)^1/q(0)].

    Then, 
    
    W_q (𝒬_∞ (μ_1) ,𝒬_∞ (μ_2)) ≤   (𝔼 |X_1 -X_1|^q (t) + 𝔼 |V_1 -V_1|^q(t))^1/q
    ≤   (𝔼|X_1 - X_1|^q (t))^1/q
             + (𝔼|V_1 - V_1|^q (t))^1/q
    ≤    C [(𝔼|X_1 - X_1|^q)^1/q(0)
            + (𝔼|V_1 - V_1|^q)^1/q(0)]
    ≤   2 C [𝔼 |X_1 -X_1|^q (0) + 𝔼 |V_1 -V_1|^q(0)]^1/q,

    since a^1/q+ b^1/q≤ 2^1-1/q(a+b)^1/q for a≥0, b≥ 0 with q≥ 1. For any ϵ>0, the initial data can be chosen by a coupling such that 
    
    [𝔼 |X_1 -X_1|^q (0) + 𝔼 |V_1 -V_1|^q(0)]^1/q≤ϵ + W_1(μ_1,μ_2).

    Hence by the arbitrariness of ϵ, it holds W_q (𝒬_∞ (μ_1) ,𝒬_∞ (μ_2)) ≤ C W_q(μ_1,μ_2),







    
    For f defined in (<ref>) with R<∞, one has
    
    W_q(𝒬_∞^n-m+1(f_m-1),𝒬_∞^n-m(f_m))
            ≤ C W_q(𝒬_∞(f_m-1), f_m),    q=[1,∞),

    where C is relevant to ψ,κ,R and d.
    
    
        
        By the flocking property shown in Step 1 in Lemma <ref>, one has supp^v[𝒬_∞(f_m-1)]⊂supp^v[f_m-1]. Also, supp^v[f_m-1]⊂supp^v[f_0] by Lemma <ref>. Furthermore, for n-m-1≥ k≥ 0, 𝒬^k+1_∞ (f_m-1) and 𝒬^k_∞ (f_m) have compact supports, where 𝒬^0_∞ (f_m):=f_m.
        
        Similarly with the proof of Proposition <ref>, note that (X_1(kτ),V_1(kτ) and (X_1(kτ), V_1(kτ)) obey 𝒬^k+1_∞ (f_m-1) and 𝒬^k_∞ (f_m) respectively, which satisfy the momentum equality assumption of Lemma <ref>. 
    
        
        Denote 
        
    |Δ X_1|:=(𝔼|X_1 - X_1|^q)^1/q,   |Δ V_1|:=(𝔼|V_1 - V_1|^q)^1/q,   
            |Δ Z_1|:=|Δ X_1|+|Δ V_1|,

        for q≥ 1.
        Hence, by the analysis in Lemma <ref>, (<ref>) and (<ref>) still hold for t∈ [0,(n-m)τ].  
        By Lemma <ref>, one obtains
        
    |Δ Z_1 ((n-m)τ)| ≤ C |Δ Z_1 (0)|.

        By choosing a suitable coupling of the initial data, the proof is finished.
    


    

 §.§ An auxiliary system

    
    We introduce an auxiliary system (<ref>) with law S(f_t_k) to get the estimate on 
    W_q (𝒬_∞ (f_t_k), f_t_k+1) ≤ W_q(S(f_t_k),f_t_k+1) + W_q(S(f_t_k),𝒬_∞ (f_t_k) ).
 

    Rewrite (<ref>) into the following particle system
    
    {d/dtX_i(t) =   V_i(t),  i=1,⋯, N,
    d/dtV_i(t) =   ∫κψ(|x-X_i|)(v-V_i) f_t(x,v)dxdv.
            .


    In order to show the law of (X,V) at time nτ is close to 𝒬_∞^n, we consider the transient system
    
    {d/dtX̂_i(t) =   V̂_i(t),  i=1,⋯, N,
    d/dtV̂_i(t) =   ∫κψ(|x-X̂_i|)(v-V̂_i) f_t_k(x,v)dxdv,
            .

    with (X̂(0),V̂(0))∼f_t_k.
    Denote S(f_t_k):=Law(X̂(τ),V̂(τ)).

    
    Under the same assumption of Lemma <ref>, it holds
        
    supp[S(f_t_k)]⊂
                B(0,C)× B(0,C e^-κψ_0 t_k+1).

    
    
           For S(f_t_k), define
        
    𝖷_k (t) :=   max{ |x(t,x^k,v^k)|  | (x^k,v^k)∈supp[f_t_k]},
    𝖵_k (t) :=   max{ |v(t,x^k,v^k)|  | (x^k,v^k)∈supp[f_t_k]},

        where the pair (x(t,x^k,v^k), v(t,x^k,v^k)) denotes (x(t),v(t)) being the solution of (<ref>) with (x(t_k),v(t_k)) = (x^k,v^k). Note that the maximum is reached because it is assumed that supp[f_t_k] is compact. Denote K_t^x ⊂supp^x [f_t_k] (resp. K_t^v ⊂supp^v [f_t_k]) the set of points (x^k,v^k) such that the maximum is reached in 𝖷_k(t) (resp. in 𝖵_k(t)). By definition, one has |𝖷_k(t)|^2 = |x(t,x^k,v^k)|^2 for every (x^k,v^k)∈ K_t^x. It follows from Danskin's theorem <cit.> and from the fact that ∂_t x(t,x^k,v^k) = v(t,x^k,v^k) that
        
    𝖷_k(t)·𝖷_k(t) = max{ x(t,x^k,v^k) · v(t,x^k,v^k) | (x^k,v^k)∈ K_t^x }.

        Using the Cauchy-Schwarz inequality, one obtains that 
        
    𝖷_k(t) ≤𝖵_k(t).

        Similarly, by  Danskin's theorem again, one has
        
    𝖵_k(t)·𝖵_k(t) = max{ v(t,x^k,v^k) ·∫κψ(|x-x(t,x^k,v^k)|)(v-v(t,x^k,v^k)) f_t_kdxdv
    | (x^k,v^k)∈ K_t^v }.

        Note that by the definition of K_t^v, v(t,x^k,v^k) · (v-v(t,x^k,v^k))≤ 0. It holds that 
        
    𝖵_k(t)·𝖵_k(t)  ≤
                -κψ_0 ∫ v(t,x^k,v^k) · v(t,x^k,v^k) f_t_k dxdv,   (x^k,v^k)∈ K_t^v ,

        since ∫ v f_t_k dxdv = 0 and ∫f_t_kdxdv =1. Then one has 
        
    𝖵_k(t) ≤ -κψ_0 𝖵_k(t),

        and thus (<ref>) holds.
    


    
    Under the same assumption of Lemma <ref>, there exists a positive constant α such that
        
    W_q(S(f_t_k),f_t_k+1)≤ C e^-κψ_0 t_kτ^2,    q∈[1,∞),

        with C=C(κ,ψ_0,ψ_Lip, R).
    
    
    
    
    
    
        Let ℛ(x,v,t) = ∫ψ(|y-x|)(w-v)(f_t(y,w)-f_t_k(y,w))dydw. For t∈ [t_k,t_k+1), by straightforward calculus, it holds
        
        
    | ∂ℛ/∂ t (x,v,t)| =    | ∫ψ(|y-x|)(w-v) ∂f_t/∂ t(y,w)dydw|
    
                    =    | ∫ψ(|y-x|)(w-v) [ - w·∇
                    _y f_t(y,w) ..
       + ..∇_w ·(f_t(y,w)κ∫ψ (|y-y|)(w-w)f_t(y,w)dydw)]dydw |
    
                    =    | ∫[w·∇_y ψ(|y-x|)(w-v) f_t(y,w)..
        - . .dκψ(|y-x|)f_t(y,w)∫ψ (|y-y|)(w-w)f_t(y,w)dydw]dydw |
    ≤   ψ_Lip∫ |w| |v-w| f_t (y,w) dydw 
                    + dκ∫ |w-w| f_t(y,w) f_t (y,w) dydwdydw
    ≤   ψ_Lip∫ (|w| |v| + |w|^2) f_t (y,w) dydw 
                    + dκ∫ 2|w| f_t(y,w) dydw.

        
         By Lemma <ref>, 
         
    |∂ℛ/∂ t(X_1,V_1,t)| ≤ C e^-κψ_0 t.

        
        
        
        
        We first consider q to be an even integer.
        For (X_1,V_1) obeying (<ref>) and (X̂_1,V̂_1) obeying (<ref>), with (X_1,V_1) (t_k) = (X̂_1,V̂_1) (t_k) ∼f_t_k, it holds
        
    d/dt   |V_1 -V̂_1|^2 =  2∑_k=1^d(V^k_1 -V̂^k_1)d/dt(V^k_1 -V̂^k_1)
    
                    =    2κ∑_k=1^d(V^k_1 -V̂^k_1)
                     [∫ψ (|x-X_1|) (v - V_1) f_t_k(x,v) dxdv .
       .- ∫ψ (|x-X̂_1|) (v - V̂_1) f_t_k(x,v) dxdv + ℛ(X_1,V_1)]^(k)
    
                    =     2κ∑_k=1^d(V^k_1 -V̂^k_1)( ∫ψ (|x-X_1|) [(v - V_1)-(v-V̂_1)] f_t_k(x,v) dxdv)^(k)
        + 2κ∑_k=1^d(V^k_1 -V̂^k_1) (∫ [ψ (|x-X̂_1|)- ψ (|x-X_1|)](v - V̂_1) f_t_k(x,v) dxdv ) ^(k)
        + 2κ∑_k=1^d(V^k_1 -V̂^k_1) ℛ(X_1,V_1)^(k)
    ≤     -qψ_0 | V_1 - V̂_1|^2 
                    +2κψ_Lip   |X_1-X̂_1 | ∑_k=1^d
                    (V^k_1 -V̂^k_1)(∫ (v-V̂_1)f_t_k)^(k)
       + 2κ |V_1 - V̂_1 | |ℛ(X_1,V_1)|,
    ≤     -2κψ_0 | V_1 - V̂_1|^2 
                    +2κψ_Lip   |X_1-X̂_1|  |V_1 - V̂_1 |
                    |V̂_1|
       + 2κ |V_1 - V̂_1 | |ℛ(X_1,V_1)|,

        where the superscript (k) means the k-th dimension of vectors. Since ∫ vf_t dxdv = v =0 by (<ref>), and |V̂_1|≤ C e^-κψ_0 t_k by Lemma <ref> and Lemma <ref>, it holds
        
    d/dt |V_1 -V̂_1|^2   
                    ≤   -2κψ_0 | V_1 - V̂_1|^2 
                    +Cκψ_Lip   e^-κψ_0 t_k |X_1-X̂_1|  |V_1 - V̂_1 |
    
                    + 2κ |V_1 - V̂_1 | |ℛ(X_1,V_1)|,

        and
        
    d/dt|V_1 - V̂_1| ≤ -ψ_0 κ |V_1 - V̂_1|
                + Cκψ_Lip   e^-κψ_0 t_k |X_1-X̂_1|
                + κ  |ℛ(X_1,V_1)|.

        Also, with
        
    d/dt | X_1 - X̂_1| ≤ |V_1-V̂_1|,

        one has
        
    d/dt(|X_1-X̂_1| + |V_1-V̂_1| ) ≤ C(|X_1-X̂_1| + |V_1-V̂_1| ) + C|ℛ|.

        By Grönwall's inequality, it holds
        
    |X̂_1-X_1| + |V̂_1-V_1| ≤ C e^-κψ_0 t_kτ^2.

        Then
        
    [𝔼[|X̂_1-X_1|^q + |V̂_1-V_1|^q]]^1/q≤[𝔼[|X̂_1-X_1| + |V̂_1-V_1|]^q]^1/q≤ C e^-κψ_0 t_kτ^2,

        which implies
        
    W_q(S(f_t_k),f_t_k+1)≤ C(κ,ψ_0,ψ_Lip, R ) e^-κψ_0 t_kτ^2.

    



    
    
    
    The estimate on W_q(S(f_t_k),𝒬_∞ (f_t_k) ) is given by Lemma <ref>.
    
    
    
    
        Under the same assumption of Theorem <ref>, it holds
        
    W_q (𝒬_∞ (f_t_k),S(f_t_k)) ≤ C(κ, R) τ^2,    q∈ [1,∞).

    
    
    
    Set (X̂_1,V̂_1) and {(X_i,V_i)} to be the solutions of (<ref>) and (<ref>) respectively. The initial data (X̂_1(0),V̂_1(0)) and {(X_i(0),V_i(0))} follow f_t_k. Take the coupling (X̂_1(0),V̂_1(0)) = (X_1(0),V_1(0)). We introduce {(X_i^E, V_i^E)} as the approximation of {(X_i,V_i)} by the forward Euler method with time step τ. Then, taking the expectation of the selection of {(X_j(0),V_j(0)}_j 1, one has 
    
    d/dt1/2𝔼 |V_1^E - V̂_1|^2 =   ∫κψ (| x- X_1(0)|) (v- V_1(0))·𝔼(V_1^E(t)-V̂_1(t))f_t_k
        - ∫κψ (| x- X̂_1(t)|) (v- V̂_1(t))·𝔼(V_1^E(t)-V̂_1(t))f_t_k
    ≤   κ∫ψ (|x-X_1(0)|) (V̂_1(t)-V_1(0))·𝔼(V_1^E(t)-V̂_1(t))f_t_k
       + κ∫ψ_Lip |X_1(0)-X̂_1(t)| (v-V̂_1(t))·𝔼(V_1^E(t)-V̂_1(t))f_t_k.

    By Lemma <ref> and Lemma <ref>, one has 
    
    | v-V̂_1(t)|≤ Ce^-κψ_0 t_k.

    Note that 
    
    |X_1(0)-X̂_1(t)|≤ Ce^-κψ_0 t_kτ

    and 
    
    |V̂_1(t)-V_1(0)|≤ Ce^-κψ_0 t_kτ.

    Take the expectation of (X_1(0),V_1(0)), one has
    
    d/dt𝔼 |V_1^E - V̂_1|^2
             ≤ Ce^-κψ_0 t_kτ√(𝔼|V_1^E - V̂_1|^2).

    Hence
    
    𝔼 |V_1^E - V̂_1|^2 ≤ Ce^-2κψ_0 t_kτ^4.

    Similar with (<ref>), one has
    
    𝔼 |X_1^E - X̂_1|^2 ≤ Ce^-2κψ_0 t_kτ^6.

    Now consider
    
    d/dt1/2𝔼|V^E_1(t) -V_1(t)|^2 =   𝔼(V^E_1(t) -V_1(t))κ/p-1∑_j=1^p[ψ(X_j(0)-X_1(0))(V_j(0)-V_1(0)) 
       - ψ(X_j(t) - X_1(t))(V_j(t) - V_1(t))]
    ≤   𝔼κ(V^E_1 -V_1)[ψ_Lip|X_2(0)-X_1(0)- X_2(t) + X_1(t))(V_2(0)-V_1(0)) 
       + ψ(X_2(t) - X_1(t))(V_2(0) - V_1(0) - V_2(t) + V_1(t))].

    By Lemma <ref> and Lemma <ref>, one knows that |V_j(t)|≤ e^-κψ_0 t_k, for any j and t∈ [0,τ]. Hence it's easy to obtain  
    
    d/dt𝔼 |V_1^E - V_1|^2
             ≤ Ce^-κψ_0 t_kτ√(𝔼|V_1^E - V_1|^2),

    and 
    
    𝔼 |V_1^E - V_1|^2 ≤ Ce^-2κψ_0 t_kτ^4,   𝔼 |X_1^E - X̂_1|^2 ≤ Ce^-2κψ_0 t_kτ^6.

    Then, by the triangular inequality of Wasserstein distance, one has
    
    W_2 (𝒬_∞ (f_t_k),S(f_t_k)) ≤ C(κ, R)e^-κψ_0 t_kτ^2.

    The estimate of W_q is similar by calculating 
    
    d/dt(𝔼|V_1^E - V̂_1|^q)^1/q=1/q(𝔼|V_1^E - V̂_1|^q)^1/q-1d/dt𝔼|V_1^E - V̂_1|^q ≤  C(κ, R)e^-κψ_0 t_kτ.

        
    
    


    

 §.§ Proof of Theorem <ref>

    Now we prove Theorem <ref> by combining the above lemmas.
    
    
    By the triangular inequality, it holds
    
    W_q(𝒬_∞^n (f_0), f(nτ))  ≤∑_m=1^n W_q (𝒬_∞^n-m+1 (f_m-1),𝒬_∞^n-m(f_m)).

    And then, by the stability result in Lemma <ref>, it holds
    
    ∑_m=1^n W_q (𝒬_∞^n-m+1 (f_m-1),𝒬_∞^n-m(f_m))
            ≤∑_m=1^n C W_q(𝒬_∞ (f_m-1), f_m).

    Again by the triangular inequality, one has
    
    W_q(𝒬_∞ (f_m-1), f_m) ≤ W_q(𝒬_∞ (f_m-1), S(f_m-1))
            + W_q(S(f_m-1), f_m).

    Then, by Lemma <ref> and Lemma <ref>, we have
    
    ∑_m=1^n W_q (𝒬_∞^n-m+1 (f_m-1),𝒬_∞^n-m(f_m))
            ≤∑_m=1^n  C e^-κψ_0 (m-1)ττ^2 ≤  C τ,

    where the constant C is independent of n.
        One completes the proof.
    
    
    
    
        Although our analysis is based on W_q,2 distance, the main results in Section <ref> also work on W_q,p, p∈ [1,∞), by noting that
    
    x _p≤ d^1-1/q+1/px _q,    x∈ℝ^d,    1<p,q<∞.

    







§ APPLICATION: THE RBM-GPC


In this section, we show a simple application for the mean-field limit analysis for the Cucker-Smale model combined with the RBM. 

Based on the above analysis, and inspired by Carrillo's work <cit.>, we can construct the numerical scheme for the approximation of stochastic mean-field models of swarming which preserve the nonnegativity of the solution. 

Let us consider the stochastic mean-field equation for the distribution function f(θ, x, v; t):

    ∂_t f + v·∇_x f = ∇_v · [ℋ[f]f],

where

    ℋ[f](θ,x,v;t) = κ∫∫ψ(|x-y|,θ)(v-w)f(θ,y,w;t)dwdy,

with ψ satisfying  positivity, boundedness, Lipschitz continuity and mononticity conditions: there exists a positive constant ψ_M  > 0  such that 

    0≤ψ(r,θ) ≤ψ_M, ∀ r≥0,   ψ_Lip<∞,


    (ψ(r_1,θ)-ψ(r_2,θ))(r_1-r_2)≤ 0,   r_1,r_2 ∈ℝ_+.

Here, θ is a random variable from a measurable space (Ω,ℱ) to (I_Θ,ℬ_ℝ), with a probability distribution π (θ), with I_Θ⊂ℝ and ℬ_ℝ the Borel set.

It is easy to see that the total mass and momentum of (<ref>) is conserved in time.
Note that if we directly apply the classical stochastic Galerkin method to (<ref>), the solution might lose its positivity. However, we can make use of the microscopic dynamics to approximate f. Our method is to consider the corresponding Cucker-Smale model with random input θ on the interaction kernel ψ:

    {∂_t x_i(t,θ) =    v_i(t,θ),  i=1,⋯, N,
    ∂_t v_i(t,θ) =   κ/N-1∑_j=1^N 
            ψ(|x_j(t,θ)-x_i(t,θ)|,θ)(v_j(t,θ)-v_i(t,θ)).
        .


Denote {Φ_k(θ)}_j=0^∞ to be the gPC basis with respect to π, which means that Φ_k is a polynomial of degree k, and
these polynomials form an orthonormal basis of L^2(Ω,ℱ,π). 
We approximate the position and the velocity of i-th agent as follows:

    x_i(θ,t)≈ x^K_i = ∑_k=0^K^kx̂_i(t)Φ_k(θ),
           v_i(θ,t)≈ v^K_i = ∑_k=0^K^kv̂_i(t)Φ_k(θ),

where

    ^kx̂_i=∫ x_i(θ,t)Φ_k(θ)dπ,   ^kv̂_i=∫ v_i(θ,t)Φ_k(θ)dπ.

We obtain the following system

    {d/dt^ℓx̂_i (t) =   ^ℓv̂_i (t),
    d/dt^ℓv̂_i (t) =   κ/N-1∑_j=1^N∑_k=0^K e_ℓ k^ij(^kv̂_j(t)-^kv̂_i(t)),
           l = 0,1,…,K..

where

    e_ℓ k^ij=1/|Φ_ℓ|^2∫ψ(|x_j-x_i|,θ) Φ_k(θ)Φ_ℓ(θ)dπ.

Then, we design Algorithm <ref>, on the generalized Polynomial Chaos expansion in the random space based on the RBM:




    Although being derived from different numerical methods (the RBM and the Monte Carlo method), Algorithm <ref> is similar with the Monte Carlo gPC (MCgPC) proposed in <cit.> at first glance. Both can reduce the total cost of (<ref>) to 𝒪(KN). The main difference between them is that the RBM-gPC preserves the momentum during evolution. Actually, since the RBM updates information in the unit of batches, (instead, the MCgPC updates information of each particle independently at per time step.) for any t∈[t_m,t_m+1], the total sum of velocities is conserved in each batch by calculus:
    
    d/dt∑_i∈𝒞_q^ℓv̂_i (t) = κ/p-1∑_i,j∈𝒞_q∑_k=0^K e_ℓ k^ij(^kv̂_j(t)-^kv̂_i(t))=0,

    for l=0,⋯,K. Furthermore, it holds 
    
    ∑_i=1^N ^ℓv̂_i (t) = ∑_q=1^N/p∑_i∈𝒞_q^ℓv̂_i (t) = ∑_q=1^N/p∑_i∈𝒞_q^ℓv̂_i (t_m) = ∑_q=1^N/p∑_i∈𝒞_q^ℓv̂_i (t_0) = ∑_i=1^N ^ℓv̂_i (0).

    










    We use 
    x_i^K,R(t,θ):= ∑_k=0^K^kx̂_i(t)Φ_k(θ),   v_i^K,R(t,θ):=∑_k=0^K^kv̂_i(t)Φ_k(θ)
 to denote the approximation of position and velocity obtained by Algorithm <ref>. The kinetic distribution f^R_N,K(θ,x,v,t) is recovered from the empirical density function 
    f^R_N,K(θ,x,v,t) = 1/N∑_i=1^N δ(x-x_i^K,R(t,θ))⊗δ(v-v_i^K,R(t,θ)).
 Common approximations methods for Dirac delta distributions can be used when needed. For example, we fix an upper and lower bound for the (x,v)-space, to restrict our computational domain in a cube space and discretize it in uniform cells {C_l}_l=1^N_l of size |C_l|=h^2d, where h is the edge length of each cell. One can reconstruct the distribution f^R_N,K by histograms counting the number of particles belonging to each cell. Then the discrete distribution function f^R_N,K reads
    
    f^R_N,K(θ,x,v,t) = 1/|C_l|(1/N∑_i=1^N 𝕀(x_i^K,R(t,θ),v_i^K,R(t,θ)∈ C_l))·(∑_l=1^N_l𝕀((x,v)∈ C_l)),

    where 𝕀(·) is the indicator function.
    Clearly, the positivity of the distribution function is preserved. One can also refer to a brief review, by Lee et al. <cit.>, on the numerical methods for regularized Dirac delta functions.








    With respect to the stochastic variable θ, the dynamics of the N-particle system achieves spectral accuracy as we adopt the generalized polynomial chaos (gPC) basis <cit.>, provided that we have a smooth dependence of the particle solution from the random field.

    For any given θ and K, denote the distribution function of the N-particle system (<ref>) by f_N,K. 
    
    If one uses the reconstruction (<ref>), it holds that
        
    𝔼 W_2 (f^R_N,K,f^R_N,K) ≤(𝔼1/N∑_i=1^N |(x_i^R,v_i^R) - c_l(i)|^2 )^1/2,

        where c_l(i) is the center of the cell C_l containing (x_i^R,v_i^R). Since h is the uniform edge length, |(x_i^R,v_i^R) - c_l(i)|^2 ≤∑^d (h/2)^2 for any i. Combining with the estimate of W_2 (f_N,K,f^R_N,K) in Section <ref>, it holds that
    
    
    
    
        
    𝔼 W_2 (f_N,K,f^R_N,K)     ≤𝒪(√(τ/p-1)) +
           {   𝒪(N^-1/4)     if  d<2,
       𝒪(N^-1/4√(log(1+N)))     if  d=2,
       𝒪(N^-1/d)     if  d>2,
            .
            + 𝒪(√(d)h).







 §.§ Numerical experiments

Here we present some numerical tests for the RBM-gPC. In the following, we take the Legendre gPC base.



  
The space homogeneous case
We consider the space-independent case in the one-dimensional setting:

    ∂_t f(θ,v;t) = ∂_v [ K(θ)(v-u)f(θ,v;t)],

with u = ∫ vf(v)dv and K(θ)=0.5 + 0.01θ, θ∼𝒰(0,1). The long time solution is a Dirac delta δ(v-u) provided K(θ)>0, see <cit.>. We take the Legendre polynomial basis. The gPC approximation is given for h=0,⋯,M by 

    ∂_t f̂_h (v;t) = 1/|Φ_h|^2∂_v [ ∑_k=0^M (v-u) ℋ_hkf̂_t_k(v;t)],

where

    ℋ_hk = ∫ K(θ)Φ_h (θ) Φ_k (θ) d g (θ),   f̂_h (v;t) = ∫ f(θ,v;t) Φ_h (θ) d g (θ).

We consider an initial density function f_0(v) by

    f_0(v) = β[exp(-(v-μ)^2/2σ^2) + exp(-(v+μ)^2/2σ^2)],   σ^2=0.1,  μ = 0.5,

with β>0 a normalization constant. Discrete samples of the initial data of the ODE system are obtained from f_0(v). Denote the expected temperature of the system 𝒯 by 

    𝒯(f) = ∫ (v-u)^2 f(θ,v;t)dvdg(θ).

In Figure <ref>, we show the evolution of the expected density toward the Dirac delta function δ(v-u), u=0 for the RBM-gPC scheme at different times with p=2 and N=10^4.

In Figure <ref>, we take T=0.5, K=3, dt=10^-2 and dv=10^-2. Time integration is performed through a 4th order Runge-Kutta method. In Figure <ref>, we evaluate the mean square error (MSE) for the expected temperature 

    MSE_T := 1/n_m∑_i=1^n_m|𝒯(f_ref) - 𝒯(f_rbm^(i)) |^2,

as N grows with p=2, where 𝒯(f_ref) denotes the reference expected temperature obtained by (<ref>) through a central difference scheme, and 𝒯(f_rbm^(i)) denotes the expected temperature obtained by the RBM-gPC in the i-th turn. We have applied n_m=100 for random initialization and batch splits. 
Figure <ref> shows the difference of TV distance between the expected reference distribution and the expected distribution by the RBM-gPC:

    Err_TV := | 𝔼_θ f_ref - 𝔼_θ1/n_m∑_i=1^n_m f_rbm^(i)|_L^1,

with n_m =100 as well.












Also, Figure <ref> and Figure <ref> show the decay of MSE_T as the batch size p and the time step dt, respectively. The reference distribution of Figure <ref> is obtained by the RBM-gPC with N=2^8, p = 2^8, and dt = 10^-2, and of Figure <ref> is obtained with N=2^8, p = 2, and dt = 10^-2. We still take n_m=100.






  
Stochastic 1D Cucker-Smale dynamics
In this test, we consider the 1D Cucker-Smale dynamics. Let us consider the initial distribution as the following bivariate and bimodal distribution

    f_0(x,v) = 1/2πσ_xσ_vexp(-x^2/2σ_v ^2)[exp(-(v+1)^2/2σ_x ^2)+ exp(-(v-1)^2/2σ_x^2)],

with σ_x^2 =0.5, σ_v^2=0.2. Our initial data for particle positions and velocities are
sampled from f_0. The stochastic interactions are given by

    H(θ; |x_i - x_j|) = 1/(1+|x_i - x_j|^2)^γ(θ),

with γ (θ) = 0.1+ 0.05 θ, θ∼𝒰[-1,1]. The results are obtained through the RBM-gPC scheme with dt = 10^-2, N=10^4, batch size p =2, and K=3. The expected distribution is reconstructed by  in Matlab in the domain [-3,3]×[-3,3] with 100 gridpoints in both space and velocity. Figure <ref> presents the evolution over the time interval t∈[0,4] of the expected distribution. 






  
Stochastic 2D Cucker-Smale dynamics
Figure <ref> shows the evolution over the time interval t∈ [0,4] of the 2D Cucker-Smale nonhomogeneous mean-field model. For the initial distribution, we consider uniformly distributed N=10^4 particles on a 2D annulus with a circular counterclockwise motion

    f_0 (x,v) = 1/|C|χ (x∈ C)δ( v - k∧ x/|x|),

being C: = {x∈ℝ^2: 0.5≤ |x_1-x_2|≤ 1 } and k the fundamental unit vector of the z-axis, where |C| means the cardinality of set C.
The interaction kernel shares the same form with (<ref>).
The evolution shows the flocking phenomenon in the top row and that the mean velocity is conserved in the bottom row of Figure <ref>. The results are obtained through the RBM-gPC scheme with dt = 10^-2, N=10^4, batch size p =2, and K=3. The reconstruction follows the same method as in the 1D case.








§ CONCLUSION

In conclusion, we give an analysis on the mean-field limit of the RBM for the Cucker-Smale model and obtain a long-time approximation. In addition, as a simple application, we propose a generalized Polynomial Chaos based Random Batch Method (RBM-gPC) as a numerical method to simulate the mean-field limit dynamics of the Cucker-Smale model with random inputs.

This paper focuses on the separate limits of the number of particles N and the time step τ but not a uniform limit of (N,τ). For future work, it will be interesting to investigate the mixture of particles as time evolves, and to study on the uniform limit. Also, numerically, an error estimate on this model applied with the particle method (corresponding to a new model) can be an interesting topic to study.







§ ACKNOWLEDGEMENT

We thank Prof. Shi Jin and Prof. Lei Li for detailed and valuable comments and advice on the manuscript. This work is supported by the NSFC grant (No. 12031013, 12201404), the Shanghai Municipal Science and Technology Key Project (No. 22JC1402300), and Project supported by the National Science Foundation for International Senior Scientists (No. 12350710181).



    
    

§ PROOF OF LEMMA <REF>

    The proof is given by Lemma 2.4 in <cit.>.
    
        For the first assertion, we claim that the relative velocities are non-increasing in time. Let t∈ [t_m-1, t_m) be given. Then one can choose time-dependent indices k and ℓ such that 
        
    |V_k^R (t) - V_ℓ^R (t)| = 𝒟_V^R(t).

        Then for such k and ℓ, one has 
        
    d/dt | V_k^R(t) -V_ℓ^R(t)|^2 =    
                2(V_k^R(t) - V_ℓ^R (t)) ·d/dt(V_k^R(t) - V_ℓ^R (t))
    
                =    2κ/p-1∑_j∈ [k]_mψ (|X_j^R - X_k^R|)(V_j^R - V_k^R)·(V_k^R - V_ℓ^R)
       - 2κ/p-1∑_j∈[i]_mψ (|X_j^R - X_ℓ^R|)(V_j^R - V_ℓ^R)·(V_k^R - V_ℓ^R).

        In order to show that the right-hand side of (<ref>) is not positive, we use the maximality of | V_k^R(t) -V_ℓ^R(t)| at time t. Since
        
    | V_k^R(t) -V_ℓ^R(t)| ≥ | V_j^R(t) -V_ℓ^R(t)|,    j = 1,⋯,N,

        one has
        
    (V_j^R(t)-V_k^R (t))·(V_k^R(t)-V_ℓ^R (t)) =
                - ((V_k^R-V_ℓ^R) - (V_j^R-V_ℓ^R))· (V_k^R - V_ℓ^R)
    ≤ - | V_k^R(t) -V_ℓ^R(t)|^2 + | V_j^R(t) -V_ℓ^R(t)|| V_k^R(t) -V_ℓ^R(t)|≤ 0,    j= 1,⋯,N.

        Similarly, one has 
        
    (V_j^R(t)-V_ℓ^R (t))· (V_k^R(t)-V_ℓ^R (t)) ≥ 0,    j=1,⋯,N.

        Hence, | V_k^R(t) -V_ℓ^R(t)|^2 is non-increasing in time.

        Then by the definition od 𝒟_X^R and 𝒟_V^R, one has 
        
    d/dt𝒟_X^R(t)≤𝒟_V^R(t) ≤𝒟_V^R(0),    a.e.  t>0.

        This yields the second assertion on 𝒟_X^R.
    
    
    
    

    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
        
    
    
    
    
    
    

    
    
    
    
    
    
    
    
    
    
    
    
    
    

    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
        
    
    
    
    
    
    
    
    
    
    
    
    


    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    

    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    


    




    
    
    
    
    

§ DETAILS ON PROPOSITION <REF>


    

  
Fixed-time regularity of f. First we check the regularity of f by Lemma <ref>, which is needed in the proof of Proposition <ref>.
    
    
    For the flocking kinetic model (<ref>), suppose that the initial probability measure f_0 ∈ (C^2∩ W^2,∞)(ℝ^2d) is compactly supported, and the interaction kernel ψ satisfies 
    sup_r≥ 0 (|ψ^'(r)| + |ψ^''(r)|) ≤ϵ_ψ
 with a constant ϵ_ψ. Then, for any T∈ (0,∞), there exists a unique classical solution f_t ∈ C^2([0,T)×ℝ^2d) for (<ref>). 

    The proof stem of Lemma <ref> is generally a smoothness check for f inspired by in [<cit.>, Theorem 3.1]. The main difference with <cit.> is that here we need better regularity for the operator (ℒ^*)^2 and (ℒ_k^*)^2 in Lemma <ref>.
    
    Local existence follows from [<cit.> Theorem 3.1] by the standard fixed point argument. One only needs to obtain a C^2 bound of f to conclude the global existence of classical solutions. One can express the non-conservative kinetic model (<ref>) in terms of the non-linear transport operator 𝒯:=∂_t + v·∇_x + ξ [f] ·∇_v,
    
    𝒯f = -f∇_v ·ξ[f].

    We claim that there exists a positive constant C(d,κ,ϵ_ψ, R) such that
    
    | 𝒯f| ≤ C |f|,

    
    | 𝒯(∂_x_if) | ≤ C (|f| + |∇_v f| + |∂_x_if)|,

    
    | 𝒯(∂_v_i f) | ≤ C (|∇_v f| + |∂_x_if|),

    
    | 𝒯(∂_x_i x_jf) | ≤ C (  |∂_x_if| + |∂_x_jf| + |∂_x_i x_jf| +|∇_v f| + |∇_v∂_x_if| + |∇_v∂_x_jf|),

    
    | 𝒯(∂_x_i v_jf) | ≤ C ( |∂_x_i x_jf| +|∇_v f| + |∂_v_j∂_x_if| + |∇_v∂_v_jf| + |∇_v∂_x_if|),


    
    | 𝒯(∂_v_i v_jf) | ≤ C ( |∂_x_i v_jf| + |∂_x_j∂_v_if| +  |∂_v_j∂_v_if|).


    
    To verify these inequalities, note that -∇_v·ξ [f] = κ d ∫ψ (|y-x|)f(y,w,t)dydw and thus |∇_v·ξ [f]| ≤κ d. By direct calculus, one obtains the following estimates:
 

    For (<ref>),
    
    | 𝒯f| ≤| -κf∇_v ·∫ψ(|y-x|) (w-v) f(y,w,t) dydw| ≤κ d |f|;

    for (<ref>),
    
    | 𝒯(∂_x_if) |    ≤| - (∂ _x_iξ [f]) ·∇_v f -  (∂_x_i∇_v ·ξ [f])f  -  (∇_v ·ξ [f])∂ _x_if|
       ≤κϵ_ψ (√(M_2^v f_0) + 2R) |∇_v f| + κ d ϵ_ψ |f| + κ d |∂_x_if|)
       ≤ C (|f| + |∇_v f| + |∂_x_if)|;

    For (<ref>),
    
    |𝒯∂_v_if | ≤ |-∂_x_if| + κ |∇_v f| + κ d |∂_v_if|
            ≤ C (|∇_v f| + |∂_x_if|).

    The estimate on (<ref>)-(<ref>) is similar and we omit it for brevity.
    
    Now, let F(t) measure the W^2,∞-norm of f,
    
    F := f(·,t) _W^2,∞ .

    The above inequalities imply
    
    d/dtF(t) ≤ CF(t).

    Then we end up with the energy bound by Grönwall's inequality
    F(t)≤ F(0) e^Ct.
    Equipped with this a priori W^2,∞ estimate, standard continuation principle yields a global extension of local classical solutions. 




The following is a simple prerequisite for Proposition <ref>.

    
For any g_1∈ L^1(ℝ^2d),g_2∈ L^1(ℝ^2dp), the operators ℒ_k^* and ℒ^* defined by (<ref>) and (<ref>) respectively, it holds
    
    ∫ | e^τℒ_k^*g_1| dz ≤∫ | g_1| dz,

    
    ∫|  e^τℒ^*  g_2  | dz_1 ⋯ dz_p ≤∫|  g_2  | dz_1 ⋯ dz_p.


     
    
    One only needs to consider g_1≥ 0 since there is composition g_1 = g_1^+ -g_1^-. Also, since e^τℒ_k^*cg_1 = ce^τℒ_k^*g_1 and e^τℒ^*cg_2 = ce^τℒ^*g_2 for any constant c, one can set g_1 be a probability density function. Then,
    
    ∫ |e^τℒ_k^*g_1| dz = ∫ e^τℒ_k^*g_1 dz,

    since e^τℒ_k^*g_1 is another probability density function derived from g_1 by the dynamics (<ref>). It's clear that
    
    ∫ e^τℒ_k^*g_1 dz = 
        ∫∫_0^τ∂/∂ tg_1(t)dtdz + ∫ g_1 dz = ∫ g_1dz,

    where g_1(t) denotes the solution of (<ref>) with initial datum g_1. For the general g_1, 
    
    ∫ | e^τℒ_k^*g_1| dz ≤∫ | e^τℒ_k^*g_1^+| dz
             + ∫ | e^τℒ_k^*g_1^-| dz
             =  ∫ | g_1^+| dz + ∫ | g_1^-| dz =  ∫ | g_1| dz.

    The proof of (<ref>) is similar.





    
    

  
Proof of Proposition <ref>.
    It's clear that S = e^τℒ_k^*f_t_k, where 
    
    ℒ_k^* := -  (v ·∇_x) · -
        ∇_v ·( ∫κψ(|y-x|)(w-v)f_t_k(y,w)dydw ·).

    Define the operator of (<ref>) as
    
    ℒ^* := -∑_i=1^p (v_i∇_x_i)·  - ∑_i=1^p∇_v_i· (ξ_i ·).

    Then one has
    
    𝒬_∞ (f_t_k) =   ∫ e^τℒ^*Π_i=1^p f_t_k(x_i,v_i)dx_2⋯ dx_pdv_2⋯ dv_p 
    
            =   f_t_k (x_1,v_1) + τ∫ℒ^* Π_i=1^p f_t_k (x_i,v_i)dx_2⋯ dx_pdv_2⋯ dv_p 
       + ∫_0^τ (τ -s) ∫ (ℒ^*)^2 e^sℒ^*Π_i=1^p f_t_k (x_i,v_i)dx_2⋯ dx_pdv_2⋯ dv_p ds,

    where
    
    
    ∫ℒ^* Π_i=1^p f_t_k (z_i) dz_2⋯ dz_p 
                =     -∑_i=1^p ∫ (v_i ∇_x_i)·Π_j=1^p f_t_k (z_j) - ∇ _v_i· (ξ_i Π_j=1^p)f_t_k (z_j)dz_2⋯ dz_p)
    
                =     - (v_1 ∇_x_1) ·f_t_k (z_1) - ∇_v_1·∫κψ (|y-x_1|)(w-v_1) f_t_k (y,w) dydw.

    
    Note that S(f_t_k) = f_t_k + τℒ_k^*f_t_k + ∫_0^τ (τ -s)(ℒ_k^*)^2 e^sℒ_k^*f_t_k ds, then
    
        
    | S(f_t_k) - 𝒬_∞ (f_t_k) |(z_1)≤∫ _0^τ (τ-s) [ |(ℒ_k^*)^2 e^sℒ_k^*f_t_k(z_1)|  +  | ∫ (ℒ^*)^2 e^sℒ^*Π_i=1^p f_t_k (z_i) dz_2⋯ dz_p | ]ds.

    
    
    As shown in Lemma <ref>, f_t_k∈ C^2(z), hence it holds 
    
    ∫ |(ℒ_k^*)^2 f_t_k| dz≤ C ,  ∫| (ℒ^*)^2 Π _i=1^p f_t_k (z_i)| dz_2 ⋯ dz_p≤ C,
 
    where C=C(κ,ϵ_ψ ,R,d). By Lemma <ref>, taking g_1 = (ℒ_k^*)^2  f_t_k and g_2 = (ℒ^*)^2 Π _i=1^p f_t_k (z_i), one has
    
    ∫ |(ℒ_k^*)^2 e^τℒ_k^*f_t_k| dz ≤∫ |(ℒ_k^*)^2 f_t_k| dz,

    and
    
    ∫| (ℒ^*)^2 e^τℒ^*Π _i=1^p f_t_k (z_i)| dz_1 ⋯ dz_p ≤∫| (ℒ^*)^2 Π _i=1^p f_t_k (z_i)| dz_1 ⋯ dz_p,

    Using the exchangability of ℒ^* and e^τℒ^*, with ℒ^*=ℒ_k^* or ℒ^*. 
    
    Then one has |S(f_t_k) - 𝒬_∞ (f_t_k)|_TV≤ C τ^2. 
    
    For S(f_t_k), by Lemma <ref>, it holds
    
    supp[S_∞(f_t_k)]⊂
            B(0,C)× B(0,C e^-κψ_0 t).


    For the support of 𝒬_∞(f_t_k), by Lemma <ref>, one knows that 
    
    supp[𝒬_∞(f_t_k)]⊂
            B(0,C)× B(0,C e^-κψ_0 t_k).

    
    Hence for any z ∈supp|S(f_t_k) - f_t_k|, it holds |z|≤ C.
    Then, by Lemma <ref> and applying similar analysis on Lemma <ref> Step 1, one has 
    
    W_q (𝒬_∞ (f_t_k),S(f_t_k)) ≤ C(κ,ϵ_ψ , f_0,d) τ^2/q.















plain





