On the mean field limit of consensus based methods
Marvin Koß Simon Weissmann Jakob Zech

Speaker and Style Disentanglement of Speech Based on Contrastive Predictive Coding Supported Factorized Variational Autoencoder

    Yuying Xie1, Michael Kuhlmann2, Frederik Rautenberg2 , Zheng-Hua Tan1, Reinhold Haeb-Umbach2

    1. Department of Electronic Systems, Aalborg University, Denmark 

    2. Department of Communications Engineering, Paderborn University, Germany 



    
===========================================================================================================================================================================================================================================================



§ ABSTRACT 
Consensus based optimization (CBO) employs a swarm of particles evolving as a system of stochastic differential equations (SDEs). Recently, it has been adapted to yield a derivative free sampling method referred to as consensus based sampling (CBS). In this paper, we investigate the “mean field limit” of a class of consensus methods, including CBO and CBS. This limit allows to characterize the system's behavior as the number of particles approaches infinity. Building upon prior work such as <cit.>, we establish the existence of a unique, strong solution for these finite-particle SDEs. We further provide uniform moment estimates, which allow to show a Fokker-Planck equation in the mean-field limit. Finally, we prove that the limiting McKean-Vlasov type SDE related to the Fokker-Planck equation admits a unique solution.




§ INTRODUCTION

Consensus based optimization (CBO) methods comprise a class of stochastic interacting particle systems utilized as metaheuristic optimization techniques for non-convex and high-dimensional optimization problems. Metaheuristic methods, which aim to design efficient algorithms for solving challenging optimization problems effectively, have garnered significant attention in recent years <cit.>. These methods can be tailored to a wide array of optimization challenges, rendering them suitable for diverse applications, including machine learning, data science and engineering, among others.

Interacting particle optimization methods, e.g. <cit.>,
blend the exploration of complex landscapes inherent to the underlying cost function with the exploitation of experiential knowledge stored within the particle system. Through interactions among the particles, these methods have demonstrated promising success in avoiding local minima and discovering global solutions.

At 
its core, CBO, as introduced in <cit.>, can be motivated as a system of multiple agents that collaboratively work toward finding the optimal solution, with the ultimate goal of reaching a consensus. More precisely, it considers the minimization problem

    min_x∈^d f(x),

where f:^d→ is a given cost function. The stochastic dynamical system employed by CBO relies solely on the evaluation of this cost function and remains derivative-free. In other words, the scheme does not involve the computation of (potentially costly or unobtainable) gradients.
In comparison to other particle swarm optimization methods, CBO also avoids the need for computing min_j=1,…,J f(x^j) across the particle system (x^j)_j=1,…,J. Instead, it introduces a smooth approximation through a weighted average over the particle system. The weight function depends on f and a tuning parameter β>0, and is defined by

    :=ℝ^d →ℝ
    x↦exp(-β f(x)).

By construction, the weight function gives more weight to x∈^d with low value of the cost
function f(x). We define
ℳ_β: 𝒫 (^d) →^d with

    η:= ∫_^dx e^-β f(x)dη(x)/∫_^de^-β f(x)dη(x)

as the operator mapping a probability measure η∈𝒫(^d) to its weighted average. The weighted average of the interacting particle system (x^j)_j=1,…,J can then be computed as

    η^J= 1/∑_j=1^J e^-β f(x^j)∑_j=1^J x^j e^-β f(x^j)  ,

where η^J=1/J∑_j=1^J δ_x^j∈𝒫(^d) denotes the empirical measure over the particle system. In the limit for β→∞ one may exploit the fact that
<cit.>

    β→∞lim-1/βlog∫_^d(x)dη(x)   = x∈supp(η)inf f(x)

to justify the smooth approximation

    η^J≈x^j,J,j=1,…,Jmin f(x^j,J).

CBO uses this idea to evolve a swarm of J particles (θ^j,J_t)_j=1,…,J according to competing terms for both local exploration around the current optimum of the particles and global exploration away from it by obtaining the optimum through the application ρ^J_t, where ρ^J_t:=1/J∑_j=1^J δ_θ^j,J_t. 
Local exploration is thus governed by moving the particles to the weighted average defined in (<ref>), while global exploration is incorporated through random pertubation depending on the distance of the particles from it. As introduced in <cit.>, the interacting particle system of CBO is described as a coupled system of stochastic differential equations (SDEs) of the form

    dθ_t^j,J = -(θ_t^j,J-) dt + λ^-1_d(|θ_t^j,J-|)  dW_t^j

with j=1,…,J and ensemble size J≥2, where (W_t^j)_t∈[0,T] are independent Brownian motions in ^d. Here, 
for x=(x_1,…,x_d)^⊤∈^d we write

    _d(x) = (x_1,…,x_d)∈^d× d ,

to denote a diagonal matrix. We assume that the initial ensemble is drawn as iid samples of some initial distribution ρ_0∈𝒫(^d). 
The heuristic behind the considered system of SDEs is that the particles will meet a consensus in ρ^J_t which can potentially lead to attaining the global optimum of f.

The existing theoretical convergence analysis of the scheme is mainly characterized by the system's behavior in the large ensemble size limit (J→∞), known as the mean field limit. In the mean field limit, CBO can be described by a McKean-Vlasov type SDE 
of the form

    dθ̅_t = -(θ̅_t-ρ_t)  dt + λ^-1_d(|θ̅_t-ρ_t|)  dW_t

where λ>0 is fixed, 
(W_t)_t∈[0,T] is a Brownian motion in ^d, and for each t≥0, ρ_t denotes the law of the solution θ̅_t itself. Through Itô's formula, the evolution of the law (ρ_t)_t∈[0,T] can be characterized as a weak solution of the Fokker-Planck equation

    ∂ρ/∂ t = ∇· ((θ-ρ)ρ + 1/2λ_d(|θ - ρ|^2)∇ρ ),

with initial condition ρ_0. Both of these equations have been analyzed in terms of well-posedness in <cit.>. While the convergence behavior of CBO is characterized through the mean field limit (<ref>) and (<ref>), the practical implementation is based on the particle approximation (<ref>). 
A convergence analysis of (<ref>) towards the mean field limit (<ref>) for J→∞ has been provided in <cit.>. 



 §.§ General class of consensus based methods


Recently, in <cit.>, the authors introduced a more general class of consensus based methods that allow the scheme to be applied as both an optimization 
and a sampling method. 
For optimization, the particle system may reach a consensus at the global minimum of the considered cost function f, i.e. all particles coalesce in the minimum. 
For sampling, 
the consensus is characterized as Gaussian approximation of the stationary distribution ρ_∗∝exp(-f) of the considered mean field equation. As such the method lends itself for use in Bayesian inference, to generate approximate samples from a posterior distribution.

Beside the weighted mean ℳ_β, the proposed formulation relies on the weighted covariance matrix 𝒞_β: 𝒫(^d)→^d× d defined as

    𝒞_β(η) :=∫_^d(x-η)⊗ (x-η) e^-β f(x)dη(x)/∫_^de^-β f(x)dη(x) .

In the finite particle regime, the proposed algorithm can be written in the form

    dθ_t^j,J = -(θ_t^j,J-)dt + √(2)d_t,

for j=1,…,J, where (_t)_t∈[0,T] again denote independent Brownian motions. The corresponding mean field limit as J→∞ can again be written as SDE of McKean-Vlasov type

    d_t = -(_t-ρ_t)dt + √(2ρ_t)dW_t,

where for each t≥0, ρ_t = ℙ ^_t denotes the law
of _t.  One fundamental advantage of the considered formulation
is that the dynamical system remains invariant under affine
transformations. This property is particularly beneficial for sampling
methods based on Markov chain Monte Carlo methods, see, e.g.,
<cit.>.

Depending on the choice of λ, the authors identify the scheme as an optimization method (λ=1) or a sampling method (λ = (1+β)^-1). This observation has been supported by a detailed theoretical analysis of the mean field limiting system. 
However, the proposed reformulation of consensus based methods in the form of (<ref>) leaves open questions regarding the well-posedness of the finite particle system as well as the well-defined limit J→∞. The goal of the present paper is to address this issue. 



 §.§ Related Work

Various particle based optimizers have been proposed, including Particle Swarm Optimization <cit.> and Consensus Based Optimization <cit.>, for which analyses of the mean field regime have been done, respectively, by <cit.> and <cit.>.
For CBO in particular, an analysis in the finite particle regime has been done in <cit.>. The authors in <cit.> modify CBO for high-dimensional problems, <cit.> adapt it and other bayesian inference algorithms for gradient inference, and <cit.> apply CBO to multi-objective optimization. An overview over the research on CBO can be found for example in <cit.>.

A recently proposed variant is Consensus Based Sampling (CBS) <cit.>, which is amenable both to sampling and optimization and has successfully been applied to rare event estimation in <cit.>. 
Moreover, a kernelized variant has been proposed by <cit.>, which is able to deal with multimodal distributions and objective functions with multiple global minima, respectively. When it comes to the implementation of CBS, the bias due to discretization and Gaussian approximation can be eliminated using a Metropolization step as proposed by <cit.>.

Upon completion of this manuscript we were made aware of the recent study <cit.>, which also analyzes convergence to the mean-field limit for both CBO and CBS. Specifically, in Thm. 2.6 & 2.12 of <cit.> using a
synchronous coupling approach they show pathwise Wasserstein p-chaos <cit.>, which actually implies our pointwise empirical propagation of chaos <cit.>. 
  Our independent analysis, conceived concurrently with theirs,
differs in that we employ a compactness argument for the proof, considering weak convergence of the empirical measures rather than 
a coupling approach.



 §.§ Our contribution

We make the following contributions: In the finite particle regime, we make use of a stochastic Lyapunov function in order to 
show well-posedness of the system of SDEs (<ref>); i.e. 
  we prove existence of unique and strong solutions. For the mean field limit, we verify tightness of the empirical measures along the lines of <cit.>, and provide a compactness argument which closely follows the arguments in <cit.>. These arguments imply weak convergence of the empirical measures  and, using Skorokhod's Representation, ℙ-a.s. convergence on some appropriate probability space.
Lastly, we adopt the use of the Leray-Schauder fixed point theorem from <cit.> to verify well-posedness of the McKean-Vlasov type SDE (<ref>) in the form of existence of a unique solution.


  
Structure The paper is structured as follows. In Section 2 we investigate the well-posedness of the finite particle SDEs. In Section 3 we investigate the mean field limit of the system of SDEs, by first verifying a weak solution of the Fokker Planck Equation exists, and then continuing to show a unique solution to the associated McKean-Vlasov limiting SDE exists. We only sketch proofs in the main part and refer to the supplementary material for some more results and proofs of the Lemmas and Theorems given in the main part.




  
Notation
We write ℙ^X for the law of a random variable X.
For a vector x∈^d, we write |x| for its Euclidean norm,
while
C_F=√(CC^T)=√(∑_i=1^d∑_j=1^d (C_ij)^2) denotes
the 
Frobenius norm of a matrix C∈^d× d and |[C]|_p=√(C^*C)^p^1/p its Schatten p-Norm. Note that the Frobenius norm is compatible with the Euclidean norm. 
We call the space of positive semi-definite d× d-Matrices 𝕊^d_≥.
Throughout the article, we write particles as θ_t^j,J to make explicit the dependence on the number of particles.
We furthermore write (^d) for the space of probability measures on ^d, and for p∈ℕ, _p(^d) 
is the space of probability measures on ^d with finite p-th moment.
We write C(^d) for the space of continuous functions from ^d→,
C^p(^d) for the space of
p∈ℕ times continuously differentiable functions from ^d→,
C_c^p(^d) for
compactly supported functions in C^p(^d), and
C_b(^d) for bounded functions in C(^d).
Additionally, we'll require the space C([0,T];^d)
of continuous functions from [0,T]→^d.
For p≥ 1 we use Łp(^d,ρ) (or Łp(ρ))
to denote the usual Lebesgue spaces of functions from ^d→.




§ WELL-POSEDNESS OF THE PARTICLE APPROXIMATION

Before considering the mean-field limit analysis, we 
firstly ensure that the finite particle SDE (<ref>) admits a unique and strong solution. This is
the primary concern of this section.
Following
<cit.>,
throughout this paper 
we make 
the following assumptions on the
cost function:

  
  The cost function f:^d→ satisfies:

        
  * There exists a constant Lip(f)>0 such that for all x,y ∈ℝ^d it holds
        
    |f(x)-f(y)| ≤Lip(f)·(|x|+|y|)·|x-y|.

        
  * The cost function f is bounded from below with -∞<f_*:=x∈^dinf f(x) and there exists a constant c_u>0 such that for all x∈ℝ^d it holds
        
    f(x)-f_* ≤ c_u(1+|x|^2) .

        
  * There exist constants c_l, M >0 such that for all x ∈ℝ^d with |x|>M it holds
        
    f(x)-f_* ≥ c_l|x|^2 .



We make a further assumption on the finiteness of the sixth moment of the initial measure.
 
    The initial measure ρ_0 satisfies 
    ∫_ℝ^d|x|^6dρ_0(x) < ∞, 
    that is ρ_0∈𝒫_6(ℝ^d).



For a fixed weight parameter β>0, in the following we
  use the operator (cp. (<ref>))

    L_β :=(^d) →(^d) 
    η↦η/_Ł1(η)

which maps a probability measure η to the reweighted probability
measure denoted by L_βη. Note that
ℳ_β(η)=𝔼_x∼
  L_βη[x].  We next state two lemmas that are used
throughout all parts of the mean field limit proof, and in the proof
of well-posedness, too.  The first Lemma has been proven in
<cit.> and states that second moments of
reweighted probability measures L_βμ can always be bounded by
the second moments of μ∈𝒫_2(^d).

 Let f satisfy Assumption <ref>  and
  let μ∈_2(^d).
Then 
    
    |ℳ_β (μ)|^2≤∫_^d |x|^2 dL_βμ(x) ≤  b_1 + b_2∫_^d|x|^2dμ(x)  ,

    where
      b_1 := M^2+b_2,
      b_2 := 2c_u/c_l (1+1/β c_l1/M^2)
    are positive reals
    depending on f and β, but not on μ.

The following result extends the estimate provided by Lemma <ref> to the weighted covariance matrix. This property is crucial in our considered analysis, since the diffusion is given by 𝒞_β(ρ_t^J) and 𝒞_β(ρ_t) respectively. 
  Let f satisfy Assumption <ref> 
and μ∈_2(^d).
Then 

    max(μ_F,√(μ)^2_F ) ≤b_1 + b_2 ∫_ℝ^d |x|^2 dμ(x) ,

where b_1, b_2>0 are the constants from Lemma <ref>.


We now provide global existence of a unique strong solution for the coupled system of SDEs (<ref>) via stochastic Lyapunov theory. We refer the reader to <cit.>.  Let us rewrite the system of d· J coupled SDEs (<ref>) as a single SDE in the joint state space ×_j=1^J ^d of the form 

    dθ_t = F(θ_t) dt + G(θ_t) dW_t,

where θ_t = (θ_t^j)_j=1,…,J∈^J· d × 1, W_t = (W_t^j)_j=1,…,J∈^J· d× 1 and for x=(x^1,…,x^J)^⊤∈^J· d we defined


    F(x)    = (ρ^J-x^j)_j=1,…,J∈^J· d× 1,   G(x) = (√(2λ^-1η^J))_j=1,…,J∈^J· d× J· d,


with empirical measure η^J := 1/J∑_j=1^J δ_x^j and (B_j)_j=1,…,J denoting a block diagonal matrix with blocks B_j, j=1,…,J.
Moreover, let (ℱ_t^J)_t≥0 = (σ(θ_0,W_s,s≤ t))_t≥0 be the canonical filtration generated by the Brownian motion (W_t)_t≥0 and the initial state θ_0.

In order to ensure unique existence of strong solutions for the SDE (<ref>), following <cit.>, we need to construct a function V∈ C^2(^J· d) satisfying


    (x) := ∇ V(x)· F(x)+1/2G^⊤(x)∇^2V(x)G(x)≤ cV(x),

for some c>0 and 

    inf_x>R V(x) →∞  for R→∞.


Here, ℒ is the generator of the Markov process
(θ_t)_t∈[0,T].  This leads us to our first main result, in
particular the one of well-posedness of the finite particle SDEs:
  
Let f satisfy Assumption <ref> and let θ_0=(θ_0^j)_j=1,…,J be ℱ_0^J-measurable maps θ_0^j:Ω→^d. For all T≥0 there exists a unique strong solution (θ_t)_t∈[0,T] (up to ℙ-indistinguishability) of
the system of coupled SDEs (<ref>).

For the proof of Theorem <ref> we construct the Lyapunov
function 
    V(x):=1/2Jj=1J|x^j,J|^2

  and verify the two
conditions 
in (<ref>) using the previous two lemmas. Note that
the limiting condition (<ref>) is obviously
satisfied.  Beside the existence of unique strong solutions, we also
derive estimates that are uniform in J for the second,
fourth and sixth moments of the particle system in
Section <ref>. These estimates are crucial to
derive the mean field limit.



§ MEAN-FIELD LIMIT


Taking the limit of the number of particles J→∞ in (<ref>), we pass to the mean field limit of the dynamical system. This limit is given by the following SDE of McKean-Vlasov type: 

    d_t = -(_t-ρ_t)dt + √(2ρ_t)dW_t,

where for each t≥0, ρ_t = ℙ ^_t denotes the law of _t. Deriving the mean field limit in this setting is challenging, as we have nonlinear and nonlocal terms ρ_t and √(ρ_t) arising in the drift and diffusion, which depend on the law of the solution itself. By applying Itô's formula, one however expects that the law of the solution of (<ref>) satisfies the Fokker-Planck Equation 

    ∂ρ/∂ t   = ∇· ((θ-ρ)ρ + ρ∇ρ ).

We follow the arguments in
<cit.>, where the mean field limit is obtained for the system of SDEs (<ref>). For each φ∈ C_c^2(^d) and t∈[0,T],
we define the functional 𝔽_φ,t on 𝒞([0,T];(^d)) as

    ρ :   = ∫_^dφ(x) dρ_t(x) - ∫_^dφ(x) dρ_0(x) + ∫_0^t ∫_^dℒφ(x) dρ_s(x) ds 
       = ∫_^dφ(x) dρ_t(x) - ∫_^dφ(x) dρ_0(x) + ∫_0^t ∫_^d(x - ρ_s)^⊤∇φ(x) dρ_s(x) ds 
        - ∫_0^t i=1dk=1d∫_^dρ_s_ik∂^2/∂ x_i∂ x_kφ(x) dρ_s(x) ds .


We then provide the definition of a weak solution to the FPE (<ref>) as defined in <cit.> by ρ∈ C([0,T];_2(^d)) satisfying the two conditions
    
        
  * For all ϕ∈C_b(^d) and t_n→ t it holds
        
    ∫_^dϕ(x)dρ_t_n(x) →∫_^dϕ(x)dρ_t(x)

        
  * For all φ∈C_c^2(^d) and t∈[0,T] holds 
        
    ρ = 0,

    

By this definition of weak solution we restrict to elements in C([0,T];𝒫(^d)) limiting the type of convergence result to be expected. We make the following distinction between pointwise and pathwise convergence:

  <cit.> 
Let ρ^J:=1/J∑_j=1^Jδ_θ^j,J∈𝒫(C([0,T], ^d)) be the empirical measure for a particle stochastic process {(θ^j,J)_t∈[0,T]}_j=1^J of ensemble size J 
and let e_t:C([0,T],^d)→^d,f↦ f(t) be the evaluation map. We define the time marginals ρ_t^J:=(e_t)_#ρ^J=ρ^J(e_t^-1(·)). Then ℙ^ρ^J∈𝒫(𝒫(C([0,T],^d))) and (ℙ^ρ^J_t)_t∈[0,T]∈ C([0,T],𝒫(𝒫(^d))) and we define the following types of convergence:

    
  * We say {(ℙ^ρ^J_t)_t∈[0,T]}_J∈ℕ⊂ C([0,T],𝒫(𝒫(^d))) converges pointwise, if there exists (ρ_t)_t∈[0,T]∈ C([0,T],𝒫(^d)) such that ℙ^ρ^J_tJ→∞→δ_ρ_t in 𝒫(𝒫(^d)), with δ_ρ_t being a Dirac measure, for every t∈[0,T],
    
  * We say {ℙ^ρ^J}_J∈ℕ⊂𝒫(𝒫(C([0,T],^d))) converges pathwise, if there exists ρ∈𝒫(C([0,T],^d)) such that ℙ^ρ^JJ→∞→δ_ρ, and δ_ρ is a Dirac measure in 𝒫(𝒫(C([0,T],^d))).
 
The convergence is weak convergence. We call elements of C([0,T],𝒫(^d)) measure flows. 
 

Pathwise convergence is stronger than pointwise convergence in the sense that pathwise implies pointwise, but not vice versa. We begin the proof of our following main result, Theorem <ref>, by working in 𝒫(𝒫(C([0,T];^d))) in steps 1 and 2, which take place in Section <ref>, and then project to C([0,T];𝒫(𝒫(^d))) for steps 3 and 4 (transferring the previously obtained weak convergence to the time marginals using <cit.>), identifying the limit of the sequences in 𝒫(𝒫(^d))
for all t∈[0,T]. Therefore, our final result yields pointwise convergence. 

In order to guarantee uniqueness of the FPE (<ref>), we make the following assumption on the solution of the corresponding McKean SDE (<ref>):
  Let (ρ_t)_t∈[0,T] be the 
the solution to the McKean SDE (<ref>). We assume that there exists σ̅>0 such that 

    ρ_t - σ̅ Id≻ 0

for all t∈[0,T].

We note that for Gaussian initial distributions ρ_0 = 𝒩(m_0,C_0), Assumption <ref> is satisfied for both CBO (λ=1) and CBS (λ = (1+β)^-1) <cit.>. Moreover, it is worth mentioning that in Theorem <ref>, we will verify the existence of a strong solution to the McKean SDE (<ref>). 



We are now ready to formulate our main result, which is analogous to <cit.>. In particular, we are able to extend the mean-field limit analysis for the system (<ref>) presented in <cit.> to the stochastic particle system driven by (<ref>). 

  
Let Assumptions <ref>, <ref> and <ref> hold and for T>0 and J≥2 let (θ^j,J_t)_t∈[0,T] be the unique solution of (<ref>) with empirical measures ρ_t^J=1/Jj=1Jδ_θ_t^j,J, t∈[0,T], and ρ_0^⊗ J-distributed initial data (θ^j,J_0)_j=1,…,J. 
Then {(ℙ^ρ^J_t)_t∈[0,T]}_J∈ℕ converges pointwise to (δ_ρ_t)_t∈[0,T], where (ρ_t)_t∈[0,T]∈ C([0,T];𝒫(^D)) is the (deterministic) unique weak solution of the FPE (<ref>).

    The proof of Theorem <ref> follows a compactness argument for verifying existence of the weak solution to the FPE and 
    can be sketched as follows:

    
  * We show in Lemma <ref> that the sequence of measures 𝔉 = {ℙ^ρ^J}_J∈ℕ is tight in 𝒫(𝒫(C([0,T];^D))).
    
  * Prohorov's Theorem implies that 𝔉 is weakly relatively sequentially compact.
    
  * If a subsequence {ρ^J_k}_k∈ℕ is convergent in distribution[Equivalent to weak convergence of the ℙ^ρ^J_k.], the limit of {(ρ^J_k_t)_t∈[0,T]}_k∈ℕ solves the FPE weakly, which we show in Theorem <ref>. In this step we employ Skorokhod Representations of the ℙ^ρ^J_k_t and work with ℙ_t-a.s. convergence on some appropriate space.
    
  * Corollary <ref> implies that the limits of all {(ℙ^ρ^J_k_t)_t∈[0,T]}_k∈ℕ for weakly convergent subsequences {ℙ^ρ^J_k}_k∈ℕ of 𝔉 are equal.
    
  * Because every subsequence has a weakly convergent subsequence, and all convergent subsequences attain the same limit, so does the entire sequence.

This line of arguments is in principle analogous to
  <cit.>.
  Nevertheless, our analysis significantly deviates from this work as we
  consider a complete covariance matrix rather than a diagonal diffusion term. This results in more intricate computations and estimations.
We proceed with the first step of the above procedure, which is proving tightness of the sequence of laws, for which the following moment estimates and propagation of chaos <cit.> will be instrumental.




 §.§ Moment Estimates

We now provide some moment estimates, which are fundamental to multiple steps of the compactness argument we present for Theorem <ref>. These estimates include the second, fourth and sixth moments along the lines of <cit.>.

  Let θ_0 = (θ_0^j)_j=1,…,J be ℱ_0^J-measurable such that ρ_0 satisfies Assumption <ref>. Then, for p=1,2,3 there exists a constant K_p(T)>0, depending on T and p but independent of J, such that 
    
    t∈[0,T]∫_^dxd(x)≤ K_p(T).


    


 §.§ Tightness of the empirical measures

The following theorem provides the tightness of the empirical measures using the criteria of Aldous' <cit.>, which we recall in the supplementary material in Theorem <ref>. We adapt the strategy of the proof of <cit.>. Tightness ensures the particles do not tend to infinity with notable probability and is fundamental to the compactness argument.

  
    Let f satisfy Assumption <ref> and let ρ_0 satisfy Assumption <ref>. Furthermore, let {(θ^j,J)_t∈[0,T]}_j=1^J be the unique solution to (<ref>) with ρ_0^⊗ J-distributed {θ^j,J_0}_j=1^J. Then 𝔉 ={ℙ^ρ^J}_J∈ℕ is tight in 𝒫(𝒫(C([0,T];^D))).




 §.§ Convergence of the empirical measures



Due to Prokhorov’s theorem <cit.> it is a direct consequence of Lemma <ref> that there exists a subsequence 
{ℙ^ρ^J_k}_k∈ℕ which converges weakly (i.e. {ρ^J_k}_k∈ℕ converges in distribution) to some random measure ρ:Ω→𝒫(C([0,T];^D)). By Skorokhod’s lemma <cit.> we then find a common probability space (Ω̃,ℱ̃,ℙ̃) on which {ρ^J_k}_k∈ℕ converges ℙ̃-almost surely as random variable to a random variable ρ:Ω̃→𝒫(C([0,T];^D)) with values in 𝒫(C([0,T];^D)). 
It is important to note that at this point in our overall argument we 
project from 𝒫(C([0,T],^d)) to C([0,T],𝒫(^d)) by use of the map

    Π : 𝒫(C([0,T], ^d))    → C([0,T],𝒫(^d)) 
    ρ   ↦ (ρ_t)_t∈[0,T]

defined in <cit.> and work with the measure flows 

    ρ^J:=Π(ρ^J)=(ρ^J_t)_t∈[0,T] ,

where the time marginals ρ_t^J are as in Definition <ref>.
This 
enables us to verify the 
two pointwise convergences described below. The weak convergence of the time marginals is then obtained via <cit.>.

In continuing with our compactness argument for the unique solvability of the FPE, we now verify condition (<ref>) for the limit of a convergent subsequence {ρ^J_k}_k∈ℕ (which we simply denote by the full sequence {ρ^J}_J∈ℕ) 
using Ł1- and Ł2-convergence with respect to the probability measure ℙ̃ on the common probability space (Ω̃,ℱ̃,ℙ̃), i.e. ∀ t∈[0,T] and arbitrary φ∈ C_c^2(^d):


    
  * ρ^J2 0 as J→∞,
    
  * ρ^J1ρ as J→∞.

Since the limits are ℙ̃-almost surely unique, we obtain that the limiting random measure ρ_t satisfies ρ=0, ℙ̃-almost surely. 
These two points are shown in the following two Lemmas. 


The first point is shown using 
Ł2-convergence.
By the reasoning in <cit.>, we have the following result:

 
    Let f satisfy Assumption <ref> and let ρ_0 satisfy Assumption <ref>. For J∈ℕ, assume that {(_t)_t∈[0,T]}_j=1,…,J is the unique strong solution to the particle system (<ref>) with ρ_0^⊗ J-distributed initial data {_0}_j=1,…,J. For the projections {ρ^J}_J∈ℕ of the empirical measures it then holds for all t∈[0,T] and φ∈ C_c^2(^d) that
    
    ρ^J2 0,

    where ρ^J∈ C([0,T],𝒫(^d)).

One obtains this result by applying Itô's Lemma to the functional and using Lemma <ref>. The second part of the limiting procedure is shown in the following Lemma, which is analogous to <cit.>:
   
    Let f satisfy Assumption <ref> and let ρ_0 satisfy Assumption <ref>. For J∈ℕ, assume that {(_t)_t∈[0,T]}_j=1,…,J is the unique strong solution to the particle system (<ref>) with ρ_0^⊗ J-distributed initial data {_0}_j=1,…,J. 
    Let {ρ^J}_J∈ℕ⊂𝒫(C([0,T],^d)) be a weakly convergent subsequence of the empirical measures, denoted by the full sequence, and let ρ be their limit. It then holds for the projections for all t∈[0,T] and φ∈ C_c^2(^d) that
    
    ρ^J1ρ.


In order to obtain this result, one inspects the summands of the functional one by one for convergence.
The previous two lemmas permit us to obtain the following result:
 
    Let f satisfy Assumption <ref> and let ρ_0 satisfy Assumption <ref>. For J∈ℕ, assume that {(_t)_t∈[0,T]}_j=1,…,J is the unique strong solution to the particle system (<ref>) with ρ_0^⊗ J-distributed initial data {_0}_j=1,…,J. Let {ρ^J}_J∈ℕ be a weakly convergent subsequence of the  empirical measures, denoted by the full sequence, with  limit ρ∈𝒫(C([0,T],^d)). Then the projection of the limit ρ∈ C([0,T],𝒫(^d)) is a weak solution of the FPE (<ref>). 

The above result provides that the limiting measure of convergent subsequences indeed (weakly) solves the FPE.



 §.§ Properties of the McKean-Vlasov SDE


For our proof of the unique solvability of the FPE (<ref>) we require existence and uniqueness of the solution to the following associated SDE of McKean-Vlasov type:


    dθ_t = -(θ_t-ρ_t)dt + √(2ρ_t)dW_t ,

where ρ_t = ℙ^θ_t denotes the law of θ_t.
This SDE describes the behavior of a representative particle in the mean field limit, and if the solution exists, its law solves the FPE (<ref>). The existence and solution are nontrivial to obtain since the drift and diffusion term depend in nonlinear ways on the law of the solution itself. Hence, we dedicate the following Theorem to the existence and the one after it to the uniqueness of the solution to this SDE. We follow the strategy of <cit.>, wherein the authors obtain unique solvability for the McKean SDE for CBO using Schaefer's Fixed point Theorem (SFPT)[cf. Thm <ref> in the Appendix]. The difference in our analysis lies in the fact that we have the covariance matrix of the reweighted measure instead of a diagonal diffusion matrix. We emphasize that this extension is non-trivial.

Next, we obtain estimates that will be useful in proving both existence and uniqueness of the solution to the McKean-Vlasov type SDE and is made in analogy to <cit.>. 
 
    Let μ, ν∈𝒫_6(^d)  with 
    (∫_^d |x|^6 dμ(x))^1/6,  (∫_^d |x|^6 dν(x))^1/6≤ K  .
 
    Then there exist c_0,c_1>0 such that
        
        
  (i) |μ-ν| ≤ c_0 W_2(μ,ν),
        
  (ii) μ-ν_F  ≤ c_1 W_2(μ,ν),
        
    where c_0 and c_1 depend on μ, ν solely through K.
  
  
We now provide the existence of the solution to the McKean-Vlasov type SDE using a fixed point argument which is made closely following the strategy of the proof in <cit.>.

    Let θ_0∼ρ_0 with ρ_0 satisfying Assumption <ref> and let f satisfy Assumption <ref>. For all T≥0 there exists a strong solution (θ_t)_t∈[0,T] of the McKean SDE (<ref>).


Using Assumption <ref>, we verify that solutions to the McKean-Vlasov type SDE are unique in the following sense.

     Let Assumption <ref> hold, let θ_0∼ρ_0 with ρ_0 according to Assumption <ref> and let f satisfy Assumption <ref>. Then the solution of the McKean SDE (<ref>) is unique up to ℙ-indistinguishability. 





 §.§ Uniqueness of the FPE

Until now we have been concerned with weakly convergent subsequences of the empirical measures, and shown that their limit is indeed a solution to the FPE. For the convergence of the whole sequence, it suffices to show that the solution of the FPE is unique. This is shown in Corollary <ref>, which is analogous to <cit.> and uses the results of the previous section, most crucially the previously obtained unique solvability of the associated McKean-Vlasov type SDE.

In order to prove the Corollary, we also need the following auxiliary result, which is analogous to <cit.>:
 
    For T>0, let (u,C)∈ C([0,T],^d×^d× d) and ρ_0 according to Assumption <ref>. Then the following linear PDE 
    
    ∂ρ/∂ t   = i=1dk=1d∂^2/∂ x_i∂ x_k[(C_t)_ikρ_t] - ∇·[(x-u_t)ρ_t]

    has a unique weak solution ρ∈ C([0,T];_2(^d)). 


Using the above result for the unique solvability of a linearized version of the FPE, we are able to prove the pointwise uniqueness, which is stated in the following Corollary:
  
    Let ρ_0 satisfy Assumption <ref> and let it be the initial data of two weak solutions ρ^1, ρ^2 to the FPE (<ref>). 
    Under Assumption <ref> we have
    
    t∈[0,T]supW_2(ρ_t^1, ρ_t^2)    = 0.





 §.§ Proof of Theorem <ref>

By connecting the results of the previous subsections, we are ready to prove our main result, Theorem <ref>:


     Due to the assumptions on our initialization, we 
     are able to apply propagation of chaos and the initially computed moment estimates, and 
     obtain with Lemma <ref>  that the sequence of measures 𝔉 = {ℙ^ρ^J}_J∈ℕ is tight in 𝒫(𝒫(C([0,T];^D))). Subsequently applying Prohorov's Theorem implies weak relative sequential compactness of 𝔉. 
     Theorem <ref> shows that if we take a subsequence of the {ρ^J}_J∈ℕ to be convergent in distribution, the limit of the projections {ρ^J}_J∈ℕ solves the FPE weakly.
     Furthermore, Corollary <ref> implies that the limit of the projections of  all weakly convergent subsequences of 𝔉 is equal, implying the weak convergence of the entire sequence {ρ^J}_J∈ℕ. 




§ CONCLUSION

After providing a stability result for the finite particle SDE, we investigated the mean field limit of the system of CBO and CBS, successfully showing that it holds under moderate assumptions that are common in the literature. Specifically, we employed propagation of chaos via a compactness argument by analyzing the weak convergence of the empirical measures, which additionally yielded that the limiting measure is not only a solution to the FPE, but in fact the only one.






plain








§ PROOFS OF SECTION 2



 §.§ Proofs of preliminary results


By linearity of the trace operator we have

    √(μ)^2_F 
               = ∫_^d(x-μ)⊗(x-μ)dL_βμ(x)
       = ∫_^d|x-μ|^2dL_βμ(x)
       = ∫_^d|x|^2dL_βμ(x)-|μ|^2
       ≤∫_^d|x|^2dL_βμ(x) ,

where we used |x-y|^2 = |x|^2 - 2⟨ x,y⟩ +|y|^2, the definition of μ=∫_^dx  dL_βμ(x) and the fact that μ is a constant independent of x. We apply Lemma <ref> to deduce that 

    √(μ)^2_F ≤∫|x|^2dL_βμ(x) ≤ b_1 + b_2 ∫ |x|^2 dμ(x) .

Similarly, applying Cauchy-Schwarz inequality we obtain

    μ_F^2    = ∫_^d×^d⟨ x - μ,x̂ - μ⟩ (x-μ)⊗ (x̂-μ) d L_βμ( x) d L_βμ(x̂)
       = ∫_^d×^d⟨  x - μ,x̂ - μ⟩^2 d L_βμ( x) d L_βμ(x̂)
       ≤∫_^d×^d | x - μ|^2 |x̂ - μ|^2  d L_βμ( x) d L_βμ(x̂)
        = (∫_^d | x - μ|^2 d L_βμ( x) )^2
       ≤(∫_^d | x|^2 d L_βμ( x) )^2 ,

which yields again with Lemma <ref> that

    μ_F ≤ b_1 + b_2 ∫ | x|^2 dμ( x) .






 §.§ Proofs of main results


  The proof strategy is to apply <cit.>.
    To this end we verify the assumptions of this theorem. First we
    note that the drift F and diffusion G defined in (<ref>)
    are both locally Lipschitz and satisfy a linear growth condition,
    which follows straightforwardly from the Lipschitz continuity of
    f. For more details, we refer for example to <cit.> for a similar calculation which can be adapted up to minor modifications to the present setting.
    In the rest of the proof we construct a
    Lyapunov function V and show that (<ref>) holds.
  
  Define
  
    V( x) = 1/2J∑_j=1^J 
        | x^j|^2    for
         x=( x^j)_j=1,…,J∈^J· d.

  Condition (<ref>) is clearly satisfied for
    this V,
    so that it only remains to show (<ref>).
  We compute the generator and derive the following upper bound using
  Cauchy-Schwarz inequality

    ( x)    = 1/J∑_j=1^J ⟨  x^j, ρ^J- x^j⟩ + 1/2J∑_j=1^J 2λ^-1ρ^J
       ≤ -1/J∑_j=1^J | x^j|^2 + 1/J∑_j=1^J| ρ^J|| x^j|+ 1/Jλ∑_j=1^Jρ^J.

By Lemma <ref>, we have

    1/J∑_j=1^Jρ^J≤b_1+b_21/J∑_i=1^J| x^j|^2

and using Lemma <ref>, we similarly obtain

    |ℳ_β(ρ^J)|^2≤ b_1 + b_2 1/J∑_i=1^J | x^i|^2 .

By Young's inequality we deduce that

    1/J∑_j=1^J|ℳ_β(ρ^J)|| x^j|    ≤1/J∑_j=1^J ( |ℳ_β(ρ^J)|^2/2 +| x^j|^2/2)≤b_1/2 + 1+b_2/2J∑_j=1^J| x^j|^2.

Hence, there exist constants c_1, c_2>0
such that ( x) ≤ c_1 + c_2 V( x)
and thus V_c_1( x) = c_1+V( x) gives the desired Lyapunov
function. In all this shows that the assumptions of <cit.> are satisfied. An application of this theorem
then implies the claim.



  Let p, q≥ 0, let μ be a probability measure on ^d, and
  let f∈Łp+q(^d,μ). Then
  
    ∫_^d|f(x)|^p dμ(x)
        ∫_^d|f(y)|^q dμ(y)
        ≤∫_^d|f(x)|^p+q dμ(x).



  Without loss of generality p≤ q. For p=0 the statement is
  trivial, so let p>0. Then q/p≥ 1 and by Hölder's inequality
  
    ∫_^d|f(x)|^p dμ(x)
        ≤(∫_^d|f(x)|^pq/p dμ(x))^p/q
        =(∫_^d|f(x)|^q dμ(x))^p/q.

  This estimate together with Jensen's inequality yields
  
    ∫_^d|f(x)|^p dμ(x)
        ∫_^d|f(y)|^q dμ(y)    
        ≤(∫_^d|f(x)|^q dμ(x) )^q+p/q≤∫_^d|f(x)|^q+p dμ(x),

  which gives the claim.



  Set
    
    g:=×^d →
    
              (t, x) ↦ 
              | x|^2p.

    For x=( x_k)_k=1^d∈^d we have
    
    ∂ g/∂ t(t,  x)    = 0, 
    ∂ g/∂ x_k(t,  x)    = 2p x_k|x|^2(p-1),
    ∂^2 g/∂ x_k^2(t,  x)    = 2p| x|^2(p-1) + 4p(p-1)| x|^2 max(0,p-2) x_k^2.


    Now let θ_t^j∈^d, j=1,…,J, be the solution of
      (<ref>). Applying the Itô formula to g yields
    
    d|θ^j_t|^2p   = 2p|θ^j_t|^2(p-1)⟨θ_t^j,dθ^j_t ⟩  + p|θ^j_t|^2(p-1)⟨dθ_t^j,dθ^j_t ⟩ + 2p(p-1)|θ^j_t|^2(p-2)⟨θ_t^j,dθ^j_t⟩
       = 2p|θ^j_t|^2(p-1)ıθ^j_t-(θ^j_t-ρ^J_t)dt+√(2ρ^J_t)dW_t
          + p|θ^j_t|^2(p-1)ı-(θ^j_t-ρ^J_t)dt+√(2ρ^J_t)dW_t-(θ^j_t-ρ^J_t)dt+√(2ρ^J_t)dW_t
          + 4p(p-1)|θ^j_t|^2max(0,p-2)ıθ^j_t-(θ^j_t-ρ^J_t)dt+√(2ρ^J_t)dW_t .

    Integrating over time and applying the expectation gives
    
    |θ^j_t|^2p   = [|θ_0^j|^2p -∫_0^t2p|θ^j_s|^2(p-1)⟨θ^j_s,θ^j_s-ρ^J_s⟩ds
         + ∫_0^t2p|θ^j_s|^2(p-1)⟨θ^j_s,√(2ρ_s^J)⟩dW_s + ∫_0^t 2 p|θ^j_s|^2(p-1)√(ρ^J_s)_F^2 ds 
         + 4p(p-1)(∫_0^t|θ_s^j|^2max(0,p-2)⟨θ^j_s,θ^j_s-ρ^J_s⟩ds+∫_0^t|θ_s^j|^2max(0,p-2)⟨θ_s^j,√(2ρ^J_t)⟩dW_s)].

    Using the linearity of the expectation and the integral and the fact that the third and last integrands are martingales, we have
    
    |θ^j_t|^2p   = |θ_0^j|^2p - 2p∫_0^t|θ^j_s|^2(p-1)⟨θ^j_s,θ^j_s-ρ^J_s⟩ds
            +2p∫_0^t|θ^j_s|^2(p-1)√(ρ^J_s)_F^2ds 
         + 2p(p-1)∫_0^t|θ_s^j|^2max(0,p-2)⟨θ^j_s,θ^j_s-ρ^J_s⟩ds  
       ≤|θ_0^j|^2p  +  4p^2∫_0^t(|θ^j_s|^2(p-1)+|θ^j_s|^2max(0,p-2))(|θ^j_s|^2+|θ^j_s-ρ^J_s|^2)ds
         +2p∫_0^t|θ^j_s|^2(p-1)√(ρ^J_s)_F^2ds .

    Next, we sum over the particles and divide by J, after which we apply the inequality |a+b|^2≤ 2|a|^2+2|b|^2 to obtain 
    
    ∫_^d| x|^2pdρ^J_t( x)   ≤∫_^d| x|^2pdρ_0^⊗ J( x)
          + 4p^2∫_0^t∫_^d(| x|^2(p-1)+ | x|^2max(0,p-2))(3| x|^2+2|ρ^J_s|^2)dρ^J_s( x)ds
            
          +2p∫_0^t∫_^d| x|^2(p-1)√(ρ^J_s)^2_Fdρ^J_s( x)ds

    We now apply
    Lemma <ref> and
    Lemma <ref>, which yields
    
    ∫_^d| x|^2pdρ^J_t( x)   ≤∫_^d| x|^2pdρ_0^⊗ J( x) + 4p^2∫_0^t3∫_^d| x|^2pdρ_s^J( x)
         + [∫_^d| x|^2(p-1)(2b_1+3 +2b_2∫_^d|η|^2 dρ^J_s(η))dρ^J_s( x)]
         +∫_^d| x|^2max(0,p-2)(2b_1+2b_2∫_^d|η|^2dρ_s^J(η))dρ^J_s( x)ds 
         + 2^2p+1p^2∫_0^t∫_^d| x|^2(p-1)(b_1+b_2∫_^d|η|^2 dρ^J_s(η) )dρ^J_s( x)ds 
       ≤∫_^d| x|^2pdρ_0^⊗ J( x) + (4p^2(3+2b_2)+2^2p+2p^2 b_2)
            ∫_0^t∫_^d| x|^2pdρ_s^J( x)ds 
          + (4p^2(2b_1+3+2b_2)+2^2p+2p^2 b_1)
            ∫_0^t∫_^d| x|^2(p-1)dρ^J_s( x)ds 
          + (4p^2(2b_1))∫_0^t∫_^d| x|^2max(0,p-2)dρ^J_s( x)ds 
       =: ∫_^d| x|^2pdρ_0^⊗ J( x) + C∫_0^t∫_^d| x|^2pdρ^J_sds + C̃∫_0^t∫_^d| x|^2(p-1)dρ^J_s( x)ds 
          + C' ∫_0^t∫_^d| x|^2max(0,p-2)dρ^J_s( x)ds.

    Here we used 
      ∫ | x|^2(p-1)dρ( x)∫ |η|^2dρ(η)≤∫ | x|^2pdρ( x) which follows by Lemma <ref> .
      Continuing with our estimates, for p=1, we arrive at
    
    ∫_^d| x|^2dρ^J_t( x)   ≤∫_^d| x|^2dρ^J_0( x) + C ∫_0^t∫_^d| x|^2dρ^J_s( x) ds + (C̃+C') t.

    Applying Grönwall's inequality gives the upper bound
    
    ∫_^d| x|^2dρ^J_t( x)≤(∫_^d| x|^2dρ_0^J( x) + (C̃ +C')T)exp(CT)  < ∞.

    Similarly, for p=2, using the above estimate we obtain
    
    ∫_^d| x|^4dρ^J_t( x)   ≤∫_^d| x|^4dρ^J_0( x) + C ∫_0^t∫_^d| x|^4dρ^J_s( x) ds + C̃ t sup_t∈[0,T]∫_^d| x|^2dρ^J_t( x) + C't
       ≤∫_^d| x|^4dρ^J_0( x) + C ∫_0^t∫_^d| x|^4dρ^J_s( x) ds 
         + C̃ T (∫_^d| x|^2dρ_0^J( x) + (C̃ +C') T)exp(CT) + C'T,

    such that we can derive an upper bound by applying Grönwall's inequality once more
    
    ∫_^d| x|^4dρ^J_t( x)   ≤( ∫_^d| x|^4dρ^J_0( x) + C̃ T (∫_^d| x|^2dρ_0^J( x)
          + (C̃ +C') T)exp(CT)+C'T) exp(CT)  < ∞.

    Using the same procedure, one may obtain a bound for the sixth
    moment.




§ PROOFS OF SECTION <REF>


  The proof is a straightforward adaptation of the argument in <cit.>, and we refer to this paper for a more detailed exposition. In particular, according to Theorem <ref> it sufficies to prove the tightness of (ℙ^θ^1,J)_J∈ℕ due to the particles'
  exchangeability. 
  We prove tightness of the solution
    {θ^1,J}_J∈ℕ to (<ref>) on
    C([0,T];^d), using the two criteria by Aldous stated in
    Theorem <ref> in Appendix <ref>:

1. Let ϵ >0, then  by Markov's inequality
    
    θ_t^1,J>K_p(T)/ϵ≤ϵ/K_p(T)θ_t^1,J≤ϵ,   ∀ J∈ℕ,

    where K_p(T) is the uniform bound on the moments from Lemma <ref> and t∈[0,T].
    
2.  Let τ be a discrete σ(θ^1_s,s∈[0,T])-stopping time with τ+δ≤ T. By (<ref>) we have
    
    θ_τ+δ^1,J-θ_τ^1,J   = |-∫_τ^τ+δθ^1,J_s-ρ_s^Jds + ∫_τ^τ+δ√(2ρ_s^J)dW^1_s|^2
       ≤2|∫_τ^τ+δθ^1,J_s-ρ_s^Jds|^2 + 2|∫_τ^τ+δ√(2ρ_s^J)dW^1_s|^2

    We can bound the first term via our moment estimate. Using Jensen's
    inequality
    
    |∫_τ^τ+δρ_s^J-θ^1,J_s ds|^2   ≤δ∫_τ^τ+δρ_s^J-θ^1,J_sds 
       ≤ 2δ^2(t∈[0,T]+t∈[0,T]θ^1,J_t) 
       ≤ 2δ^2 K_1(T),

    where for the last inequality we used Lemma <ref>
      and Lemma <ref>.
    For the second moment of the covariance of the reweighted measure we similarly obtain
    
    |∫_τ^τ+δ√(ρ_s^J)dW_s|^2   =∫_τ^τ+δ√(ρ_s^J)^2_F ds
       ≤∫_τ^τ+δ  b_1+b_2∫_^d|x|^2dρ_s^J(x)ds 
       ≤δ( b_1+b_2t∈[0,T]∫_^d|x|^2dρ_t^J(x)) 
       ≤δ (b_1 + b_2K_1(T)),

    where in the first equality we used the Itô Isometry, the
      first inequality uses Fubini's theorem and Lemma
      <ref>, and the final inequality follows
      by Lemma <ref>.  
      In all, using δ≤ T and
    Jensen's
    inequality 
    once more,
    
    |θ_τ+δ^1,J-θ_τ^1,J|   ≤√(2δ·max(TK_1(T), (b_1+b_2K_1(T))))
       =: √(δ)C ,

    with C>0 depending on T but independent of δ.
    To 
    obtain
    the desired inequality (<ref>)
    for given ϵ, η>0,
    we define
    
    δ_0 := min(ϵη/C^2,T)

    and apply Markov's inequality to obtain
    
    δ∈[0,δ_0]supℙ(|θ^1,J_τ+δ-θ^1,J_τ|>η) ≤δ∈[0,δ_0]sup1/η|θ^1,J_τ+δ-θ^1,J_τ|≤ϵ,

    which concludes the proof.





  Plugging ρ^J into 𝔽_φ,t defined in
    (<ref>) we 
    get
    
    ρ^J   = φ(_t) - φ(_0) + ∫_0^t(_s-ρ^J_s)∇φ(_s) ds 
         - ∫_0^ti=1dk=1dρ_s^J_ik∂^2/∂ x_i∂ x_kφ(_s) ds 
       =  ( φ(_t) -  φ(_0)+ ∫_0^t(_s-ρ^J_s)∇φ(_s) ds 
         - ∫_0^t i=1dk=1dρ_s^J_ik∂^2/∂ x_i ∂ x_kφ(_s) ds ).

    Since φ∈C_c^2(^d), we may apply Itô's formula
    to the solution _t of (<ref>), which yields   
    
    φ(_t)    = φ(_0) - ∫_0^t(_s-ρ^J_s)∇φ(_s) ds + ∫_0^t√(2ρ_s^J)∇φ(_s) dW^j_s 
         + ∫_0^t i=1dk=1dρ_s^J_ij∂^2/∂ x_i∂ x_kφ(_s) ds.

    Inserting this into (<ref>) leads to 
    
    ρ^J = ∫_0^t√(2ρ_s^J)∇φ(_s) dW^j_s .

    For the second moment, we obtain
    
    |ρ^J|^2   = 2/λ J^2|j=1J∫_0^t√(ρ_s^J)∇φ(_s) dW_s^j|^2
       = 2/λ J^2i=1d(j=1J∫_0^t√(ρ^J_s)∇φ(_s) dW^j_s)_i^2
       =2/λ J^2i=1d(j=1Jk=1J(∫_0^t√(ρ^J_s)∇φ(_s) dW^j_s)(∫_0^t√(ρ^J_s)∇φ(θ^k,J_s) dW^k_s))_i
       =2/λ J^2i=1dj=1J∫_0^t∇φ(_s)^⊤ρ^J_s∇φ(_s) ds
       ≤ C(λ, K_p(T), T, [L^∞]∇φ1/JJ→∞→ 0

    where for the final inequality we used
    Lemma <ref> and Lemma <ref>,
    and the 
    equality in the second to last line
    holds due to Itô's Isometry and the fact that ∫_0^tg(t, θ_t)d(B_s^j⊗ B_s^k)=0 for j≠ k and g∈Ł2;
    this holds 
    due to φ∈ C_c^2(^d) and Lemma <ref>.
    We were able to apply Lemma <ref> due to our assumption about the initialisation.
    


  Firstly, we observe that for every t∈[0,T], {ρ^J_t}_J∈ℕ converges[remember that we identify the subsequence {ρ^J_k}_k∈ℕ as {ρ^J}_J∈ℕ.] ℙ̃_t-almost surely (on (Ω̃_t,ℱ̃_t,ℙ̃_t)) to ρ_t∈𝒫(^D).
  
  It suffices to check the Ł1-convergence of the
  terms of ρ^J in (<ref>) one by one.
    The difference to <cit.> lies in the diffusion term, which we split as
    
    |∫_0^ti=1dk=1d∫_^dρ_s^J_ik∂^2/∂ x_i∂ x_kφ( x) dρ_s^J( x)ds-∫_0^ti=1dk=1d∫_^dρ_s_ik∂^2/∂ x_i∂ x_kφ( x) dρ_s( x) ds| 
           ≤(∫_0^t|i=1dk=1d∫_^dρ_s^J_ik∂^2/∂ x_i∂ x_kφ( x) d(ρ_s^J( x)-ρ_s( x))|ds 
             + ∫_0^t|i=1dk=1d∫_^d(ρ^J_s_ik-ρ_s_ik)∂^2/∂ x_i∂ x_kφ( x) dρ_s( x)|ds) 
           =: (∫_0^t|I_1^J(s)|ds + ∫_0^t|I_2^J(s)|ds),

    where we used the triangle inequality and Jensen's inequality.
    The expectation of the first integrand vanishes, since φ
    vanishes at the boundaries: using Jensen's inequality, Lemma
    <ref> and K_1(T) from Lemma <ref>
    
    |I^J_1(s)|   ≤i=1dk=1d|∫_^dρ_s^J_ik∂^2/∂ x_i∂ x_kφ( x) d(ρ_s^J( x)-ρ_s( x))|
       ≤(K_1(T)^1/2i=1dk=1d|∫_^d∂^2/∂ x_i∂ x_kφ( x)d(ρ_s^J( x)-ρ_s( x))|)→ 0    as J→∞.

    Moreover, for the second integrand, we claim that for all s∈[0,T]
    
    |I^J_2(s)|   = |i=1dk=1d∫_^d(ρ^J_s_ik-ρ_s_ik)
            ∂^2/∂ x_i∂ x_kφ( x)dρ_s( x) |
       ≤sup_ x∈^d[F]∇^2 φ( x)·ρ^J_s-ρ_s_F→ 0    as J→∞.

    To show this
      claim, we first note that there holds the
      ℙ̃_s-almost sure convergence
    
    J→∞limρ^J_s-ρ_s_F = 0,

      for each fixed s because xexp(-β
        f( x)) and exp(-β f( x)) belong to
      C_b(^d). 
        More precisely, together with  
        ℙ̃_t-almost sure convergence
      of the ρ^J_t towards
        ρ_t, 
        this implies
    
    J→∞limρ_t^J   = J→∞lim∫_^d( x-ℳ_β(L_βρ^J_t))⊗( x-ℳ_β(L_βρ_t^J) dL_βρ_t^J( x) 
       = J→∞lim∫_^d( x-ı xexp(-β f( x))dρ_t^J( x)/ıexp(-β f( x))dρ_t^J( x))⊗( x-ı xexp(-β f( x))dρ_t^J( x)/ıexp(-β f( x))dρ_t^J( x))
         ·exp(-β f( x))/ıexp(-β f( x))dρ_t^J( x) dρ_t^J( x) 
       = ∫_^d( x-ı xexp(-β f( x))dρ_t( x)/ıexp(-β f( x))dρ_t( x))⊗( x-ı xexp(-β f( x))dρ_t( x)/ıexp(-β f( x))dρ_t( x))
         ·exp(-β f( x))/ıexp(-β f( x))dρ_t( x) dρ_t( x) 
       = ∫_^d( x-ℳ(L_βρ_t))⊗( x-ℳ(L_βρ_t) dL_βρ_t( x) 
       = ρ_t ,

    ℙ̃_t-almost surely. Here we wrote integrals over ^d with respect to a measure μ as ı·dμ( x) for better legibility. Importantly, we were able to use weak convergence of the time marginals ρ^J_t →ρ_t according to <cit.>.
    We apply Lemma <ref> to verify that 
    lim_J→∞ρ^J_s-ρ_s_F = 0, which can be seen as follows. Let A>0 be arbitrary, and consider
    
    ρ^J_s-ρ_s_F 
       = ρ^J_s-ρ_s_F 1_ρ^J_s-ρ_s_F ≤ A + ρ^J_s-ρ_s_F 1_ρ^J_s-ρ_s_F>A
       ≤ρ^J_s-ρ_s_F 1_ρ^J_s-ρ_s_F ≤ A + ρ^J_s-ρ_s_F^2 ^1/2ℙ̃(ρ^J_s-ρ_s_F>A)^1/2
       ≤ρ^J_s-ρ_s_F 1_ρ^J_s-ρ_s_F ≤ A + ρ^J_s-ρ_s_F^2 /A^2 ,

    where we have first applied Hölder's inequality followed by Markov's inequality. Note that by Lemma <ref> and Lemma <ref>,
    
    ρ^J_s-ρ_s_F^2≤ 2 ρ^J_s_F^2 + 2 ρ_s_F^2≤ K_2(T)<∞

    uniformly in J. Taking the limit J→∞ yields
    
    0≤lim sup_J→∞ρ^J_s-ρ_s_F ≤K_2(T)/A^2  ,

    and since A>0 is arbitrary, we also obtain lim_J→∞ρ^J_s-ρ_s_F = 0.
    
    In summary, we verified the convergence of the diffusion term
    
    J→∞lim   𝔼[|∫_0^ti=1dk=1d∫_^dρ_s^J_ik∂^2/∂ x_i∂ x_kφ( x)dρ_s^J( x)ds
            -∫_0^ti=1dk=1d∫_^dρ_s_ik∂^2/∂ x_i∂ x_kφ( x) dρ_s( x) ds|] = 0.

    The remaining two summands of the functional are identical to CBO, and their convergence has already been obtained in <cit.>, i.e. 
    it holds
    
    J→∞lim|(∫_^dφ( x)dρ^J_t( x)-∫_^dφ( x)dρ^J_0( x))-(∫_^dφ( x) dρ_t( x)-∫_^dφ( x)dρ_0( x))| = 0

      and
      
    J→∞lim|∫_0^t ∫_^d( x-ρ^J_s)·∇φ( x)dρ^J_s( x)ds-∫_0^t∫_^d( x-ρ_s)·∇φ( x)dρ_s( x)ds| = 0.

    Equations (<ref>), (<ref>), and 
    (<ref>) together
    with
    the triangle inequality
    give
    
    J→∞lim|ρ^J-ρ| = 0,

    which is the desired result.



 The proof is a direct adaptation of <cit.>.
  Considering the measure flow ρ=(ρ_t)_t∈[0,T] of time marginals of the limit ρ, we must check the two conditions in the definition of a weak solution. We begin with the first point, which requires checking that t↦ρ_t is continuous in time in the sense
  of (<ref>): For ϕ∈C_b(^d)
  and t_n→ t, dominated convergence yields
    
    ∫_C([0,T]; ^d)ϕ(f(t_n))dρ(f) n→∞→∫_C([0,T]; ^d)ϕ(f(t)) dρ(f).

    With the definition of the evaluation map e_t:C([0,T], ℝ^d)→ℝ^d, f↦ f(t) and the time marginals ρ_t:=(e_t)_#ρ=ρ(e_t^-1(·)) one can equivalently write the above as  
    
    ∫_^dϕ(x)dρ_t_n(x) n→∞→∫_^dϕ(x)dρ_t(x)

    using change of variables, which is applicable because ϕ is measurable. 
    
    The second part of the Definition now follows straightforwardly by combining the previously obtained convergence results from Lemma <ref> and Lemma <ref>:
    
    |ρ|    = |ρ|
       = |ρ-ρ^J+ρ^J|
       ≤|ρ-ρ^J| + |ρ^J|
       ≤|ρ-ρ^J| + C/√(J)→ 0    as J→∞.




  We next recall a standard argument to bound the difference
  of the square root of two matrices.
  

  Let A∈^d× d be symmetric positive semi-definite and let
  B∈^d× d be symmetric positive definite. Then
  
    √(A)-√(B)_2≤ 2√(B^-1_2)A-B_2.



  Let a≥ 0. Substituting t=√(a)tan(s) one can show
  2/π∫_0^∞a/a+t^2d
  t=2/π∫_0^π/2√(a)d s = √(a). Using the
  singular value decomposition (SVD) and applying the above equality to the
  singular values yields the well-known identity
  
    √(A) = 2/π∫_0^∞ (A+t^2I)^-1A dt,

  where I∈^d× d is the identity matrix. Since the same is true for B, we find
  
    √(A)-√(B)   = 2/π∫_0^∞ (A+t^2 I)^-1A - (B+t^2 I)^-1B dt
       =2/π∫_0^∞ ((A+t^2 I)^-1-(B+t^2 I)^-1)A dt +2/π∫_0^∞ (B+t^2 I)^-1(A-B) dt
       =:E+F.


  We first bound F_2. Denote by σ=B^-1_2^-1>0 the
  smallest eigenvalue of B. The smallest eigenvalue of B+t^2I is
  then σ+t^2. Thus
  
    (B+t^2I)^-1_2=1/σ+t^2

    and
  
    F_2≤A-B_22/π∫_0^∞1/σ+t^2d t = A-B_21/√(σ).

  
  To bound E_2, we use the matrix identity X^-1-Y^-1=Y^-1(Y-X)X^-1
  to obtain
  
    ((A+t^2 I)^-1-(B+t^2 I)^-1)A_2   ≤(B+t^2 I)^-1_2A-B_2(A+t^2 I)^-1A_2
       ≤1/σ+t^2A-B_2.

  Here we used (<ref>) and the fact that
  (A+t^2 I)^-1A_2≤ 1, which can be seen using the SVD of
  A. As before E_2≤A-B_2/√(σ), which concludes
  the proof.



The proof of the first claim is given in <cit.>. 

For the second claim, let μ, ν∈𝒫_6(^d) with 

    (∫_^d | x|^6 dμ( x))^1/6, (∫_^d |x|^6 dν(x))^1/6≤ K.

We define the normalizing constants with respect to the weight function w_β^f by

    Z_μ := ∫_^d w_β^f( x) dμ( x)  and   Z_ν := ∫_^d w_β^f(x) dν(dx).

According to
<cit.>, there exists a constant c_K>0 such that

    w_β^f( x)/Z_μ, w_β^f( x)/Z_ν≤ c_K

for all x∈^d. By definition of the weighted covariance matrix

    μ-ν   = ∫_^dw_β^f( x)/Z_μ x  x^⊤dμ( x) - ∫_^dw_β^f(x)/Z_νxx^⊤dν(x)+ M_β(μ) M_β(μ)^⊤ - M_β(ν)M_β(ν)^⊤ .

Next, we estimate

    μ-ν_F ≤[F]∫_^dw_β^f( x)/Z_μ x  x^⊤dμ( x) - ∫_^dw_β^f(x)/Z_νxx^⊤dν(x) + M_β(μ) M_β(μ)^⊤ - M_β(ν)M_β(ν)^⊤_F.

We will make use of the following estimates for arbitrary vectors x,y∈^d

    xx^⊤ -yy^⊤_F^2 =|y|^2 |x-y|^2+|x|^2 |x-y|^2 + 2⟨ y,x-y⟩⟨ x,y-x⟩≤ (|x|+|y|)^2|x-y|^2

where we have applied the Cauchy-Schwarz inequality. Hence, we first observe that

    M_β(μ) M_β(μ)^⊤ - M_β(ν)M_β(ν)^⊤_F ≤ (|M_β(μ)|+|M_β(ν)|)|M_β(μ)-M_β(ν)| ≤ 2(b_1 +b_2 c_K) c_0 W_2(μ,ν),

where we have used <cit.> and Lemma <ref>. Secondly, let π be an arbitrary coupling of μ and ν such that we can write

    [F]∫_^dw_β^f( x)/Z_μ x  x^⊤dμ( x) - ∫_^dw_β^f(x)/Z_νxx^⊤dν(x)   = [F]∫_^d∫_^d(w_β^f( x)/Z_μ x  x^⊤  -  w_β^f(x)/Z_νxx^⊤) dπ( x,x)
       ≤∫_^d∫_^d[F]w_β^f( x)/Z_μ x  x^⊤  -  w_β^f(x)/Z_νxx^⊤dπ( x,x) .

The outline of the remaining proof follows similar steps as the proof of <cit.>. In particular, we use triangle inequality to bound the integrand by

    [F]w_β^f( x)/Z_μ x  x^⊤  -  w_β^f(x)/Z_νxx^⊤   ≤ x  x^⊤ - xx^⊤_F |w_β^f(x)/Z_μ| 
         +  x x^⊤_F |w_β^f( x)-w_β^f(x)|/Z_μ
          + xx^⊤_F |w_β(x)|  |∫_^d∫_^d w_β^f(y)-w_β^f(y) dπ(y,y)|/Z_μ Z_ν  ,

where we used the equality

    w_β^f( x)/Z_μ x  x^⊤  -  w_β^f(x)/Z_νxx^⊤ = ( x x^⊤ - xx^⊤ ) w_β^f(x)/Z_μ +  x x^⊤w_β^f( x)-w_β^f(x)/Z_μ + xx^⊤ w_β^f(x) (1/Z_μ - 1/Z_ν)

and 

    1/Z_μ - 1/Z_ν = ∫_^d∫_^d w_β^f(y)-w_β^f(y) dπ(y,y)/Z_μ Z_ν .

We will integrate over and bound each tearm in (<ref>)–(<ref>) separately. We start with (<ref>) and apply (<ref>) which yields

    ∫_^d∫_^d x  x^⊤ - x x^⊤_F |w_β^f(x)/Z_μ| dπ( x,x)    ≤ c_K ∫_^d∫_^d (| x|+|x|) | x-x|dπ( x,x)
       ≤ c_K (∫_^d∫_^d (| x|+|x|)^2dπ( x,x))^1/2(∫_^d∫_^d (| x-x|)^2dπ( x,x))^1/2
       ≤ 4 c_K K (∫_^d∫_^d (| x-x|)^2dπ( x,x))^1/2 ,

where we have applied Hölder's inequality in the second line. For the second term (<ref>) we apply Assumption (<ref>) to derive

    ∫_^d∫_^d x x^⊤_F |w_β^f( x)-w_β^f(x)|/Z_μdπ( x,x)    ≤ 2c_K∫_^d∫_^d x x^⊤_F (| x|+|x|)| x-x| dπ( x,x)
       = 2c_K∫_^d∫_^d (| x|^3+| x|^2|x|) | x-x| dπ( x,x)
       ≤ 2c_K ( ∫_^d∫_^d (| x|^3+| x|^2|x|)^2 dπ( x, x) )^1/2( ∫_^d∫_^d | x-x|^2 dπ( x, x) )^1/2
       ≤ 2c_K K̃( ∫_^d∫_^d | x-x|^2 dπ( x, x) )^1/2,

where we once again used Hölder's inequality in the third line and afterwards 
the assumption on bounded second moments.
Finally, applying Hölder's inequality once more we estimate (<ref>) by

    ∫_^d∫_^d   xx^⊤_F |w_β(x)|  |∫_^d∫_^d w_β^f(y)-w_β^f(y) dπ(y,y)|/Z_μ Z_νdπ( x,x)
       ≤ c_K^2 βLip(f) ∫_^d∫_^dxx^⊤_F dπ( x,x) ∫_^d∫_^d (|y|+|y|) |y-y| dπ(y,y)
       ≤ c_K^2 βLip(f) 4K∫_^d∫_^d | x|^2 dπ( x,x) (∫_^d∫_^d |y-y|^2 dπ(y,y))^1/2
       ≤ c_K^2 βLip(f) 4 K^2 (∫_^d∫_^d |y-y|^2 dπ(y,y))^1/2 .

Taking the infimum over all couplings of μ and ν yields the claim. 



We begin by linearizing the process. Consider for fixed ρ_0∈𝒫_4(^d) and some given (u, D)∈ C([0,T],^d×^d× d) the following linear SDE

    dY_t    = -(Y_t-u_t) dt + √(2) D_t dW_t,  where  ℙ^Y_0=ρ_0.

By standard SDE theory, e.g. <cit.>, this SDE has a unique solution with laws ν_t=ℙ^Y_t, t≥0. More precisely, for every t, ν_t is the law of Y_t, leading to the function ν∈ C([0,T],𝒫(^d)).

We then define a mapping from (u, D) to the continuous function which consists of the regularized mean and regularized covariance of ν_t at every timestep:

    𝒯:=
      
      C([0,T],^d×^d× d)→ C([0,T],^d×^d× d)
    
        (u,D) ↦(ν_t,√(ν_t))_t∈[0,T],

for which we prove the conditions of Schaefer's Fixed Point Theorem (<ref>). The well-definedness of the mapping follows firstly from the ranges of ℳ_β and 𝒞_β, and secondly from their respective (Hölder) continuitiy, which we prove below.
Theorem <ref> is sufficient to go from the above linearized SDE (<ref>) to the nonlinear McKean process as a fixed point of the mapping 𝒯 ensures that (ℳ_β(ν_t),√(𝒞_β(ν_t))) can be plugged in for (u_t, D_t), which yields a solution of the nonlinear SDE in (<ref>). 

We now begin with checking the compactness requirement of Theorem (<ref>), which we obtain by showing the map t↦ (ν_t,√(ν_t)) is Hölder continuous and making use of the compact embedding from Hölder continuous functions to continuous functions.

By <cit.>, there exists c'>0 s.t. 

    |Y_t|^6   ≤ (1+|Y_0|^6)exp(c't),

for all t∈[0,T], i.e. sup_t∈[0,T]∫|x|^6dν_t≤ K for a K<∞, which means we will be able to apply Lemma <ref>. 
Furthermore, we have for t>s, t, s ∈ (0,T)

    |Y_t-Y_s|^2   = (-∫_s^tY_r-u_r dr)^2+(∫_s^t√(2) D_r dW_r)^2
         +2√(2)(∫_s^t(Y_r-u_r) dr)(∫_s^tD_r dW_r).

Bounding the three terms respectively using for example the Cauchy-Schwarz inequality, the Itô Isometry and the fact that the third term is a martingale, we have 

    |Y_t-Y_s|^2   ≤ |t-s|∫_s^t|Y_r-u_r|^2 dr+2∫_s^tD_r^2dr+ 0 
       ≤ 2(T+2)(K+u_∞^2+D^2^2_∞)|t-s|
       =: c|t-s|.

Applying <cit.> with μ=ν_t and μ̂=ν_s, we arrive at 

    |ν_t-ν_s| ≤ c_0 W_2(ν_t,ν_s) ≤ c_0c^1/2 |t-s|^1/2.

for c_0>0 from Lemma <ref>.
Similarly, using the well-known Powers-Størmer inequality <cit.>, estimating the resulting Schatten 1-norm |[·]|_1 using the Frobenius norm and applying Lemma <ref> we obtain

    √(ν_t)-√(ν_s)_F    ≤ (|[ν_t-ν_s]|_1)^1/2
       ≤ (d^1/2ν_t-ν_s_F)^1/2
       ≤  d^1/4√(c_1 W_2(ν_t,ν_s))
       ≤ c_1^1/2d^1/4  |t-s|^1/4.

This yields the Hölder continuity with exponent  
1/4 of 

    f:=
        [0,T] →^d×^d× d
    
        t ↦ (ν_t,√(ν_t)).

The compact embedding 

    C^0,1/4([0,T],^d×^d× d) ↪ C([0,T],^d×^d× d)

therefore provides the compactness of 𝒯 (See <cit.>.).

Computing the second moment using Itô's formula, we get

    d/dt∫_ℝ^d| x|^2 dρ_t = ∫_ℝ^d[2( x-u_t)·  x+D_t^2] dρ_t( x).

Lemma <ref> then yields the following for the Diffusion term:

    D_t^2 = τ^2ρ_t≤τ^2 (b_1 + b_2 ∫_^d| x|^2 dρ_t( x)).

Similarly, by Lemma <ref> it holds that 

    |u_t|^2 = τ^2ρ_t^2  ≤τ^2∫_^d| x|^2 dL_βρ_t( x) ≤τ^2(b_1 + b_2 ∫_^d| x|^2 dρ_t( x)).

Using Cauchy-Schwarz and the arithmetic-geometric mean inequality, we furthermore obtain

    -2∫_^d x· u_t dρ_t( x)    ≤ 2 ∫ | x· u_t|dρ_t( x) 
       ≤ 2∫ | x||u_t|dρ_t( x) 
       ≤∫ | x|^2 + |u_t|^2 dρ_t( x) 
       ≤∫ | x|^2 dρ_t( x) + |u_t|^2.

Therefore, we can bound (<ref>) as follows:

    d/dt∫_^d| x|^2 dρ_t( x)    ≤τ^2b_1 +(τ^2b_2+2) ∫_ℝ^d | x|^2 dρ_t( x) + |u_t|^2 
       ≤τ^2b_1(+1) +(τ^2 b_2(+1)+2)∫_^d| x|^2 dρ_t( x).

Applying Grönwall's inequality yields

    ∫_^d| x|^2 dρ_t( x)    ≤τ^2b_1(+1)exp(τ^2b_2(+1)+2)∫_^d| x|^2 dρ_0( x) < ∞.

We can transfer this estimate back to (u_t,D_t) using Jensen's inequality and Lemma <ref> respectively, resulting in

    |u_t|^2    = τ^2|ℳ_β(ρ_t)|^2 ≤τ^2 (b_1+b_2∫_^d| x|^2 dρ_t( x)) ≤ M < ∞
    and  D_t^2^2_F    = τ^2 𝒞_β(ρ_t)_F^2 ≤τ^2(b_1+b_2∫_^d| x|^2dρ_t( x))≤ M  < ∞,

and thus get upper bounds for u and D by considering the supremum norms u_∞ := sup_t∈[0,T]|u_t| and D_∞ := sup_t∈[0,T]D_t_F. Hence, we have verified all conditions to apply Theorem <ref> implying the existence of a fixed point of the mapping 𝒯. 



We showed above that a fixed point (u,D)∈ C([0,T],^d×^d) of 𝒯 satisfies u_∞,D_∞≤ M. Now let (u,D), (û,D̂) be two fixed points of 𝒯.
Recall, that we have

    u_∞, D_∞≤  M, t∈[0,T]sup∫_^d| x|^4 dρ_t( x)≤ K < ∞,
    û_∞, D̂_∞≤ M, t∈[0,T]sup∫_^d| x|^4 dρ̂_t( x) ≤ K < ∞

Taking the difference of the paths of their corresponding processes (Y_t)_t∈[0,T], (Ŷ_t)_t∈[0,T] yields 

    Y_t-Ŷ_t    =: z_t = z_0 - ∫_0^tz_s ds + ∫_0^t(u_s-û_s) ds + √(2)∫_0^t(D_s-D̂_s) dW_s.

We apply Itô's formula to obtain

    |z_t|^2 = |z_0|^2 - ∫_0^t 2|z_s|^2 ds + ∫_0^t 2⟨ z_s, u_s-û_s⟩ ds +2λ^-1∫_0^t (D_s-D̂_s)(D_s-D̂_s)^⊤ ds + ∫_0^t  dM_s,

where M_s is a Martingale such that ∫_0^t  dM_s = 0.
We apply Cauchy-Schwarz and Young's inequality to get

    ⟨ z_s, u_s-û_s⟩≤1/2|z_s|^2 + 1/2|u_s-û_s|^2 .

Taking the expectation it follows that

    |z_t|^2   ≤|z_0|^2- ∫_0^t |z_s|^2 ds + ∫_0^t |u_s-û_s|^2 ds + 2λ^-1∫_0^t (D_s-D̂_s)(D_s-D̂_s)^⊤ ds .

Note that we have interchanged the integral ∫_0^t and the expectation 𝔼 using Fubini's theorem. Moreover, (D_s-D̂_s)(D_s-D̂_s)^⊤ = D_s-D̂_s_F^2.)
Substituting ρ_t for u_t, ρ̂_t for û, √(ρ_t) for D_t, and √(ρ̂_t) for D̂_t, and using Lemma <ref> and <cit.>, we obtain

    |z_t|^2   ≤|z_0|^2- ∫_0^t |z_s|^2 ds + ∫_0^t |ρ_s-ρ̂_s|^2 ds 
          + 2λ^-1∫_0^t √(ρ_s)-√(ρ̂_s)_F^2 ds
       ≤|z_0|^2 +(c_0^2+2 c_14d/σ̅-1)∫_0^t|z_s|^2ds .

Here we used Assumption <ref> and Lemma <ref> to deduce that 

    √(ρ_s)-√(ρ̂_s)_F^2≤ d √(ρ_s)-√(ρ̂_s)_2^2    ≤ 4d ρ̂_s^-1_2 ρ_s-ρ̂_s_2^2 
       ≤4d/σ̅ρ_s-ρ̂_s_F^2 .

Applying Grönwall's inequality and using the fact that |z_0|^2=0 yields the desired |z_t|^2=0, implying uniqueness.




  The existence follows by applying the Itô's formula to the law of the solution of (<ref>).

  
    To obtain the uniqueness, consider for each t_0∈(0,T] and compactly supported smooth test function ψ∈ C_c^∞(^d) the Kolmogorov backward equation 
    
    ∂ h_t/∂ t   = -i=1dk=1d (C_t)_ik∂^2/∂ x_i∂ x_k(h_t)_k-(x-u_t)·∇ρ_t,   (t,x)∈[0,t_0]×^d;   h_t_0 = ψ

    By <cit.>, this linear PDE admits the solution 
    
    h_t(x)    = ψ(θ_t_0^t,x), t∈[0,t_0],

    where (θ_t_0^t,x)_0≤ t ≤ s ≤ t_0 is the strong solution to the linear SDE
    
    dθ^t,x_s    = -(θ^t,x_s-u_s) ds + √(2 C_s)dB_s,    θ_t^t,x = x

    The chain rule provides, for each (t,x)∈[0,t_0]×^d:
    
    ∇_kh_t(x)    = ∇_kψ(θ_t_0^t,x)∇_kθ_t_0^t,x.

     According to <cit.> we may apply the derivative w.r.t. parameter k to the drift and diffusion coefficients of dθ^t,x_t_0 separately, and simply obtain
    
    d∇_k(θ_s^t,x)_k    = ∇_k(θ^t,x_s)_k ds,  ∇_k(θ^t,x_t)_k = 1.

     We obtain a geometric brownian motion without diffusion and have
     
    ∇_k(θ^t,x_s)_k =  exp(s-t)

    Therefore, there exists a constant c>0 depending only on ψ s.t.
    
    (t,x)∈[0,t_0]×^dsup |∇_kh_t(x)|    ≤ cexp(T) < ∞, k=1,…,d

    Furthermore, for 0≤ t < t+δ < t_0, we have θ_s^t,x = θ_t_0^t+δ,θ_t_0^t+δ,x for t+δ < s ≤ t_0 and thus 
    
    h_t+δ-h_t(x)/δ   = 1/δψ(θ_t_0^t+δ,x)-ψ(θ_t_0^t,x)
       = 1/δψ(θ_t_0^t+δ,x)-ψ(θ_t_0^t+δ,θ^t,x_t+δ)
       =1/δh_t+δ(x)-h_t+δ(θ^t,x_t+δ)
       =1/δ-∫_t^t+δi=1dk=1d(C_s)_ik∂^2/∂ x_i∂ x_kh_t+δ(θ^t,x_s)-(x-u_t)·∇ h_t+δ(θ^t,x_s) ds

    Since we saw that the law ρ_t=ℙ^θ_t is a classical solution to the linearized FPE (<ref>), the limit δ→0limh_t+δ-h_t(x)/δ exists.

    
    Now suppose ρ^1 and ρ^2 are two weak solutions of (<ref>) with identical initial condition ρ_0^1=ρ_0^2. We set δρ_t=ρ^1_t-ρ^2_t. Now we integrate the solution h to the Kolmogorov backward PDE, which we have seen exists, against this (signed) measure: 
    
    ∫ h_t_0 dδρ_t_0(x)    = 
                ∫_0^t_0∫_^d∂_sh_s(x) dδρ_s(x)ds +∫_0^t_0∫_^di=1dk=1d (C_s)_ik∂^2/∂ x_i∂ x_kh_s dδρ_s(x)ds
           + ∫_0^t∫_^d(x-u_s)·∇ h_s dδρ_s(x)ds 
       = ∫_0^t_0∫_^d∂_sh_s(x) dδρ_s(x)ds + ∫_0^t_0∫_^d-∂_sh_s(x) dδρ_s(x)ds 
       = 0

    Since we chose h_t_0=ψ∈ C_c^∞(^d), we have ∫_^dψ(x)dδρ_t_0(x)=0, which means δρ_t_0=0. Since t_0 was arbitrary, we have ρ^1=ρ^2.



    We construct two linear processes (θ^l_t)_t∈[0,T] (l=1,2) satisfying
    
    dθ^l_t = -(θ^l_t-ρ_t^l)dt + √(2ρ_t^l)dW_t,

    with θ_0∼ρ_0. The laws of these linearized processes ρ^l_t:=ℙ^θ^l_t are thereby weak solutions to the linear PDE
    
    ∂ρ^l_t/∂ t   = i=1dk=1d∂^2/∂ x_i∂ x_k[ρ^l_t_ikρ^l_t] - ∇·((x-ρ^l_t)ρ^l_t).

    By assumption, ρ^l also solves the above PDE, since in this case it is identical to the nonlinear FPE. In the subsequent Proposition <ref>, we show that the weak solution to the above SDE is unique. From these two facts follows that ρ^l=ρ^l. But ρ^l=ℙ^θ^l is the law of the McKean Process
    
    dθ_t = -(θ_t-ρ^l_t) dt + √(2ρ^l_t)dW_t,

    which is solvable uniquely up to ℙ-indistinguishability according to Theorem <ref>.
    Thus, (θ^l_t)_t∈[0,T] = (θ^l_t)_t∈[0,T] is a solution to the McKean Process in (<ref>). Therefore
    
    0 = t∈[0,T]sup|θ^1_t-θ^2_t| 
              = t∈[0,T]sup|θ^1_t-θ^2_t|≥t∈[0,T]sup W_2(ρ^1_t,ρ^2_t) 
              ≥t∈[0,T]sup W_2(ρ^1_t,ρ^2_t),

    which is the desired result.





§ ADDITIONAL RESULTS USED IN THE PROOF


  Let E be a Polish Space, and ρ^J=1/Ji=1Jδ_θ^j,J be (E)-valued random variables, where (θ^1,J,…,θ^J,J) is distributed according to u_J. Assume that (u_J)_J∈ℕ is a sequence of symmetric probabilities on E^N. Then, (ρ^J)_J∈ℕ is tight if and only if (ℙ^θ^1,J)_J∈ℕ is tight.



   Let {X^n}_n∈ℕ be a sequence of stochastic processes on a common probability space (Ω,ℱ,ℙ) with X^n: Ω→ C([0,T];ℝ^d). The sequence is tight on C([0,T];ℝ^d) if the following two conditions hold:

 
     
  * {ℙ^X^n_t}_n∈ℕ is tight on ^d for every t∈[0,T].
     
  * ∀ϵ> 0,η> 0 and m there exists δ_0∈(0,T) and n_0∈ℕ s.t., when n≥ n_0, δ≤δ_0 and τ is a σ(X^n_s,s∈[0,T])-stopping time with finite range and τ≤ m, then:
     
    ℙ(|X^n_τ+δ-X_τ^n|≥ϵ) ≤η.

 


The solvability of the McKean-Vlasov type SDE is obtained by use of the following Theorem.
 
  Schaefer's Fixed Point Theorem: Let 𝒯:𝔅→𝔅 be a compact mapping of a Banach Space 𝔅 into itself and suppose there is a constant M such that if x∈𝔅 and τ∈[0,1] with x=τ𝒯x, then x_𝔅<M. 
Then 𝒯 has a fixed point. 





