We propose a novel non-parametric learning paradigm for the identification of drift and diffusion coefficients of multi-dimensional non-linear stochastic differential equations, which relies upon discrete-time observations of the state. The key idea essentially consists of fitting a RKHS-based approximation of the corresponding Fokker-Planck equation to such observations, yielding theoretical estimates of non-asymptotic learning rates which, unlike previous works, become increasingly tighter when the regularity of the unknown drift and diffusion coefficients becomes higher. Our method being kernel-based, offline pre-processing may be profitably leveraged to enable efficient numerical implementation, offering excellent balance between precision and computational complexity.

Non-parametric system identification with guarantees, non-linear stochastic differential equations, discrete-time observation, Reproducing Kernel Hilbert Space.

§ INTRODUCTION

Consider non-linear stochastic differential equations of type

\begin{equation} \label{eq:IntroSDE}
\mathrm{d}X(t) = b(t,X(t)) \, \mathrm{d}t + \sqrt{a(t,X(t))} \, \mathrm{d}W_t , \quad t \in [0,T] .
\end{equation}

Here, $X(t) \in \Rn$ denotes the (stochastic) state of dimension $n \in \mathbb{N}$ of the system at time $t \in [0,T]$, with $T > 0$ a fixed time horizon, $(W_t)_{t \in [0,T]}$ is a $n$–dimensional Wiener process, whereas $b : [0,T]\times\Rn\to\Rn$ and $a : [0,T]\times\Rn\to\SymPP$ are regular enough drift and diffusion coefficients, with $\SymPP \subseteq \Rnn$ the subset of symmetric positive-definite matrices.

Equations such as (<ref>) may be profitably leveraged to accurately model complex phenomena in a wide range of applications, such as aerospace, finance, and robotics to name a few [38, 50, 7, 6, 47].

In practice, both coefficients $a$ and $b$ might be completely unknown, e.g., $a$ might model external perturbations due to many different physical phenomena which affect the motion of the system, as it occurs in aerospace and robotics. Therefore, appropriate stochastic system identification procedures for both $a$ and $b$ must be devised.

§.§ Related Work

The problem of stochastic system identification has been investigated for several decades. The earliest methods mainly address either discrete-time models [23, 33] or continuous-time models that are however linear in the state variable [18], which thus do not fit the identification setting introduced by (<ref>).

More recently, identification of non-linear stochastic differential equations such as (<ref>) have seen an important surge of interest. More specifically, besides some likelihood-based [27] or Kalman filtering-based estimation methods [44, 5, 41], which assume continuous-time observation of the stochastic state, the vast majority of the existing methods leverage more realistic discrete-time observations of $X$. Efficient paradigms range from 1. non-parametric estimation [12, 19, 11, 21] and Bayesian estimation [46, 36], to 2. maximum likelihood and quasi-likelihood methods [15, 51, 30, 45], generalized methods of moments [17, 16, 37], and online gradient descent-based methods [20, 32], among others.
On the one hand, although Group 1. of these works provides learning rates which become increasingly tighter when the number of observations of the state grows, estimates of the error between the unknown drift and diffusion coefficients and the learned ones are obtained by assuming diffusion coefficients are either known or have specific parametric structures.

On the other hand, although Group 2. of the aforementioned works rely on generally milder assumptions to be implemented, they require working with given families of finite-dimensional parametric drift and diffusion coefficients,
which might hinder the identification process in the case the family of parametric coefficients is not rich enough.

Importantly, none of the above works has investigated how to leverage the regularity, i.e., high order continuous differentiability of drift and diffusion coefficients to improve learning rates. Although such analysis has been recently undertaken in, e.g., [1, 35, 2, 28], these studies often work with SDE that are either scalar or perturbed by constant diffusion coefficients, and they offer learning rates that are often only asymptotic. Therefore, designing methods for the identification of multi-dimensional non-linear SDE with non-constant diffusion coefficients under discrete-time observations of the state remains a challenging open question (note that multivariate settings are known to generally require very different techniques, see, e.g., [35, 2]). In particular, a key part of the challenge lies in developing non-asymptotic rates of convergence that become tighter not only when the number of observations of the state grows, but also when the regularity of drift and diffusion coefficients is higher.

Finally, from a numerical standpoint, learning-based methods, such as scalable gradient methods [24] and infinitely deep Bayesian neural networks [49], have shown remarkable performance on complex stochastic differential equations. Unfortunately, it seems extremely challenging to endow these latter paradigms with theoretical guarantees of convergence, motivating investigation of numerically efficient identification methods for (<ref>) which enjoy guarantees of accuracy under mild assumptions.

§.§ Outline and Contributions

We propose a non-parametric, Reproducing Kernel Hilbert Space (RKHS)-based learning paradigm for the identification of drift and diffusion coefficients of the multi-dimensional non-linear SDE (<ref>), which relies upon discrete-time observation of the state. In particular, motivated by classical likelihood-based methods [37] we propose a two-step, discrete-time observation-based scheme which entails fitting the Fokker-Planck equation related to (<ref>).

Under assumptions of smoothness for the unknown drift and diffusion coefficients, we provide theoretical estimates of non-asymptotic learning rates which become increasingly tighter when the number of observations of the state grows. In particular, given the nature of our data set, which essentially depends on observations of the state process, we consider the error between solutions to (<ref>) generated with the learned drift and diffusion coefficients and the unknown trajectories of (<ref>) is a “good metric” to test the accuracy of our identification method. Importantly, under this accuracy metric we can additionally prove our learning rates become tighter when the regularity (in a Sobolev sense) of the unknown drift and diffusion coefficients is higher. Finally, from a numerical standpoint, our method being kernel-based, offline pre-processing may be successfully leveraged to enable efficient implementations, offering excellent balance between precision and computational complexity (details are in Section <ref>).

Our method is composed of two steps which are informally summarized below.

A.  Learning the laws of the stochastic differential equation through independent discrete-time observation of the state.

We assume there exist regular enough, i.e. essentially $C^{2 m + 1}$, $m \in \mathbb{N}$, drift $b : [0,T]\times\Rn\to\Rn$ and diffusion $a : [0,T]\times\Rn\to\SymPP$, and a stochastic process $X : [0,T]\times\Omega\to\Rn$ in some filtered probability space $(\Omega,\G,\F,\Proba)$, such that $X$ solves (<ref>) with drift $b$ and diffusion $a$. In addition, we may assume we can sample $X$ at $M \in \mathbb{N}$ times $0 = t_1 < \dots < t_M = T$, and more specifically that at every time $t_{\ell}$, $\ell=1,\dots,M$, we have access to $N \in \mathbb{N}$ samples $X_{\ell,1} \triangleq X(t_\ell,\omega_1) , \dots , X_{\ell,N} \triangleq X(t_\ell,\omega_N)$ of the solution $X$ which have been independently drawn from

In such setting, as first step we propose to approximate the unknown densities $\unknownDensity : [0,T]\times\Rn\to\R$ of the laws of $X$ through the (random) RKHS-based model:

\begin{align*}
\approxDensity(t,x) \triangleq \sum_{\ell=1}^M c_{\ell}(t) \widehat{g}_{\ell}(x), \quad \textnormal{where} \quad \widehat{g}_{\ell}(x) \triangleq \frac{1}{N} \sum_{j=1}^N \rho_R(x - X_{\ell,j}) ,
\end{align*}

for appropriate coefficients $c_{\ell} : [0,T]\to\R$ and radial mappings $\rho_R$, $R > 0$. Let $\mu_X$ denote the probability measure which is generated by the process $X$. For every tuple of precision parameters $0 < \varepsilon , \delta < 1$, if the unknown drift $b$ and diffusion $a$ are regular enough, by leveraging RKHS approximation theory we can show appropriate values for $M$, $N$, and $R$ (which depend on $\varepsilon$ and $m$ uniquely) may be selected so that the following learning rate holds with probability $\mu_X$ at least $1 - \delta$:

\begin{equation} \label{eq:estimateDensityIntro}
\int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - \frac{\partial \unknownDensity}{\partial t}(t,\cdot) \right\|^2_{L^2} + \| \approxDensity(t,\cdot) - \unknownDensity(t,\cdot) \|^2_{H^2} \; \mathrm{d}t = %O\left( \left( \log\left( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 \right) ,
% \firstRev{O\left( \left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right)} ,
\firstRev{O\left( \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right)} ,
\end{equation}

where $\| \cdot \|_{L^2}$ and $\| \cdot \|_{H^2}$ denote the norms of the Hilbert spaces $L^2(\Rn,\R)$ and $H^2(\Rn,\R)$, respectively. Moreover, the higher the degree of smoothness of $b$ and $a$ is, the lower the values of $M$, $N$, and $R$ needed to achieve this precision become.

The main benefit which comes with the model $\approxDensity$ consists of computing accurate finite-dimensional approximations of the laws of solutions to (<ref>) without a priori resorting to conservative families of parametric densities. In particular, the model $\approxDensity$ being kernel-based, one may considerably reduce the computational effort by resorting to prior offline computations, which essentially boil down to simply inverting a $M \times M$ matrix. As a byproduct, (<ref>) provides a quantitative estimate of the approximation error which is key to derive theoretical guarantees for the accuracy of our identification method in the next step.

B.  Learning finite-dimensional models for the drift and diffusion coefficients by fitting approximated solutions to the Fokker-Planck equation.

If the unknown drift $b$ and diffusion $a$ are regular enough, the unknown densities $p$ satisfy the following Fokker-Planck equation:

\begin{equation} \label{eq:FPEIntro}
\frac{\partial p}{\partial t}(t,y) = (\mathcal{L}^{a,b}_t)^* p(t,y) , \quad (t,y) \in [0,T]\times\Rn ,
\end{equation}

where $(\mathcal{L}^{a,b}_t)^*$ is the dual operator of the Kolmogorov generator

\mathcal{L}^{a,b}_t \varphi(y) \triangleq \frac{1}{2} \sum^n_{i,j=1} a_{ij}(t,y) \frac{\partial^2 \varphi}{\partial y_i \partial y_j}(y) + \sum^n_{i=1} b_i(t,y) \frac{\partial \varphi}{\partial y_i}(y) , \quad \varphi \in C^2(\Rn,\R) .

Given the results at the previous step, it is then natural to learn models of the drift $\solAppDrift : [0,T]\times\Rn\to\Rn$ and the diffusion $\solAppDiffusion : [0,T]\times\Rn\to\SymPP$ which “best” match the Fokker-Planck equation when evaluated at $\approxDensity$, that is as solutions to the following finite-dimensional convex optimization problem:

\begin{equation} \label{eq:LPFinalIntro}
\underset{(\solAppDiffusion,\solAppDrift) \in \mathcal{H}_Q}{\min} \ \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\mathcal{L}^{\solAppDiffusion,\solAppDrift}_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (\solAppDiffusion,\solAppDrift) \|^2_{\mathcal{H}} ,
\end{equation}

where $\mathcal{H}_Q$ is an appropriate finite-dimensional subspace of a RKHS $\mathcal{H}$ with norm $\| \cdot \|_{\mathcal{H}}$, in which the unknown drift and diffusion coefficients lie, whereas $\lambda > 0$ is a regularization weight to be appropriately selected. As detailed in Section <ref>, problem (<ref>) admits a finite dimensional characterization that can be solved exactly, i.e., no approximation or integral discretization is needed, with reduced computational cost.

Thanks to estimate (<ref>), one shows that, for appropriate choices of the subspace $\mathcal{H}_Q$ and the regularization weight $\lambda$, the solution $(\solAppDiffusion,\solAppDrift) \in \mathcal{H}_Q$ to problem (<ref>) satisfies the following, with probability $\mu_X$ at least $1 - \delta$:

\begin{align} \label{eq:estimateCoeffIntro}
&\int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\mathcal{L}^{\solAppDiffusion,\solAppDrift}_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (\solAppDiffusion,\solAppDrift) \|^2_{\mathcal{H}} = \nonumber \\
%&\hspace{50ex} = O\left( \left( \log\left( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 \right) .
% &\hspace{50ex}\firstRev{= O\left( \left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right)} .
&\hspace{50ex}\firstRev{= O\left( \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right)} .
\end{align}

By combining estimate (<ref>) with energy-type estimates for parabolic partial differential equations, we may infer theoretical estimates of learning rates for the identification of drift and diffusion coefficients of non-linear stochastic differential equations, which we informally summarize as follows.

As we mentioned previously, since our data consists of observation of the state process, the error between the unknown densities and the densities stemming from the learned coefficients is a “good metric” with which the convergence of an identification algorithm for stochastic differential equations which leverage observations of the state process may be tested. To better formalize this metric, let $X^{\solAppDiffusion,\solAppDrift}$ and $p_{\solAppDiffusion,\solAppDrift}$ respectively denote the solutions to (<ref>) and (<ref>) with coefficients $(\solAppDiffusion,\solAppDrift) \in \mathcal{H}_H$. We thus define the following metric to test the accuracy:

\begin{equation*} %\label{eq:MetricIntro}
\Metric(\solAppDiffusion,\solAppDrift) \triangleq \| p_{\solAppDiffusion,\solAppDrift} - \unknownDensity \|^2_{L^2} = \int^T_0 \| p_{\solAppDiffusion,\solAppDrift}(t,\cdot) - \unknownDensity(t,\cdot) \|^2_{L^2} \; \mathrm{d}t .
\end{equation*}

By adopting this metric, our main result on the accuracy of our learning method may be summarized in the following meta-theorem:

Assume the unknown drift $b$ and diffusion $a$ coefficients are regular enough. The following estimate holds true with probability $\mu_X$ at least $1 - \delta$:

\Metric(\solAppDiffusion,\solAppDrift) = %O\left( \left( \log\left( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 \right) .
% \firstRev{O\left( \left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right)}
\firstRev{O\left( \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right)}

In particular, for every regular enough function $f : [0,T]\times\Rn\to\R$:

\begin{equation} \label{eq:ultimateEstimateIntro}
\mathbb{E}\left[ \int^T_0 f(t,X(t)) \; \mathrm{d}t \right] = \mathbb{E}\left[ \int^T_0 f(t,X^{\solAppDiffusion,\solAppDrift}(t)) \; \mathrm{d}t \right] + %O\left( \log\left( \frac{1}{\delta \varepsilon} \right) \varepsilon \right) .
% \firstRev{O\left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)} .
\firstRev{O\left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)} .
\end{equation}

Although the estimates provided in this meta-theorem are informal and need some clarification (see Section <ref> for more rigorous statements), they show the aforementioned two-step identification method enjoys practical theoretical guarantees of accuracy, i.e., estimate (<ref>): an observation/regulation metric $f$ computed at the unknown state solution to (<ref>) with unknown drift $b$ and diffusion $a$ may be rather observed through the process solution to (<ref>) with model drift $\solAppDrift$ and diffusion $\solAppDiffusion$ up to an error $O\left( \log\left( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)$. Such result has important implications in observation and regulation of stochastic differential equations [21], and it may represent a good starting result to develop paradigms for the identification of controlled stochastic differential equations, which are crucial for the control of complex autonomous systems.

§.§ Paper Organization

The paper is organized as follows. After gathering basic notation and preliminary results in Section <ref>, in Section <ref> we summarize both classical and less classical results about stochastic differential equations and corresponding relationships with the Fokker-Planck equation. To ease the reading of this section, we moved a more detailed description of the aforementioned results to Section <ref> and their technical proofs to Appendix <ref>. Similarly, in Section <ref> we expose the results in RKHS theory which we leverage in this work. Our main contributions are contained in Section <ref>, which in particular details the methodologies we exposed at the previous steps 1) and 2) and corresponding learning rates. Precision and computational complexity of our approach are discussed in Section <ref>. Finally, in Section <ref> we provide concluding remarks and some perspectives.

§ NOTATION AND PRELIMINARY RESULTS

We fix the dimension $n \in \mathbb{N}$ of the state space and a time horizon $T > 0$ for the identification process. We denote by $\SymP$ and $\SymPP$ the subsets of $\Rnn$ of symmetric semi-positive-definite and symmetric positive-definite matrices, respectively.

We assume stochastic differential equations are defined on a given filtered probability space $(\Omega,\G\triangleq\F_T,\F\triangleq(\Ft)_{t \in [0,T]},\Proba)$, which is complete, and the noise is generated by a $\F$–adapted Wiener process $W : [0,T] \times \Omega \to \Rn$ (e.g., we may consider the canonical process in the space $\Omega = C([0,T],\Rn)$, equipped with the Wiener measure). Moreover, we introduce the complete metric space $(\DistSpace,\Dist)$, where

\DistSpace \triangleq C([0,T],\Rn) , \quad \Dist(w_1,w_2) \triangleq \underset{t \in [0,T]}{\sup} \ \| w_1(t) - w_2(t) \| ,

and equip it with the Borel sigma-algebra $\BorelDist$ induced by the metric $\Dist$.

For any $\ell \in \mathbb{N}$, $r , s > 0$, and any $A \subseteq \R^{\ell}$, we denote by $H^{r,s}([0,T]\times\Rn,A)$ the Sobolev (Hilbert) space of functions whose image is in $A$, and whose weak time derivatives are defined up to order $r$ and whose weak space derivatives are defined up to order $s$; in particular, we denote $H^r([0,T]\times\Rn,A) \triangleq H^{r,r}([0,T]\times\Rn,A)$. Finally, we introduce the following Hilbert space and corresponding positive convex cone, in which we will assume the unknown drift and diffusion coefficients lie (this latter requirement will be made more explicit shortly):

\begin{align*}
\Cbasis \triangleq H^{d(m)}([0,T]\times\Rn,\Rnn) \times H^{d(m)}([0,T]\times\Rn,\Rn) , \\
\C \triangleq \Big\{ (a,b) :[0,T]\times\Rn \to \SymP\times\Rn : \ (a,b) \in \Cbasis \Big\} ,
\end{align*}

for every $m \in \mathbb{N}$, where $d(m) \triangleq 2 (m + 1) + \left\lfloor \frac{n}{2} \right\rfloor \in \mathbb{N}$ is the unique integer greater than $2 m + 1$ such that $\left\lfloor d(m) - \frac{n}{2} \right\rfloor = 2 m + 1$. The choice of taking the same exponent $d(m)$ for both the drift and the diffusion coefficients has been made without loss of generality for the sake of conciseness. The following regularity result will be crucial:

There exists a constant $C > 0$, which depends on $\expo$ uniquely, such that every $(a,b) \in \C$ satisfies the following properties:

* Differentiability:

a \in C^{2 \expo + 1}([0,T]\times\Rn,\Rnn) , \quad b \in C^{2 \expo + 1}([0,T]\times\Rn,\Rn) .

* Boundness of functions and their derivatives:

\sum^{2 m + 1}_{i=0} \Big( \| D^i_{(t,y)} a \|_{L^{\infty}} + \| D^i_{(t,y)} b \|_{L^{\infty}} \Big) \le C \| (a,b) \|_{\Cbasis} ,

where $D^i_{(t,y)}$ denotes the differential of order $i$ with respect to $(t,y)$. In particular, the following refined bound holds:

\sum^2_{i=0} \Big( \| D^i_{(t,y)} a \|_{L^{\infty}} + \| D^i_{(t,y)} b \|_{L^{\infty}} \Big) \le C \| (a,b) \|_{H^{d(m) - 2 m + 1}} .

The proof of this result makes use of classical embedding arguments and it is reported in Appendix <ref> for the sake of completeness.

§ STOCHASTIC DIFFERENTIAL AND FOKKER-PLANCK EQUATIONS

In this section, we summarize both classical and less classical results about stochastic differential equations, and corresponding relationships with the Fokker-Planck equation. In particular, we propose a minimally detailed discussion for the sake of conciseness, reporting a more structured exposition in Section <ref> for the sake of completeness.

§.§ The Fokker-Planck equation

From now on, we fix $m \in \mathbb{N}$ (to be specified later) and a constant $\alpha > 0$. Since we will need to work with diffusion coefficients which are never trivial, for every $(a,b) \in \C$ we will rather replace $a$ with the mapping

%a + \alpha I : [0,T]\times\Rn \to \SymPP \quad \textnormal{with} \quad a + \alpha I = \sqrt{a + \alpha I} \sqrt{a + \alpha I}^{\top} .
a + \alpha I : [0,T]\times\Rn \to \SymPP \quad \textnormal{with} \quad a + \alpha I = \sqrt{a + \alpha I} \sqrt{a + \alpha I} .

At this step, we fix a non-negative density $p_0 \in L^2(\Rn,\R)$ which will serve as appropriate initial condition, and we denote by $\mu_0 \in \ProbaSpace(\Rn)$ the associate probability measure. Motivated by [43, 14], we recall the following notions of stochastic differential equation and its solutions (though they differ from the ones in [43, 14]):

There exists a measurable mapping $X : \Rn\times\Omega\to\DistSpace$ such that each process $X_x(t,\omega) \triangleq X(x,\omega)(t)$ is $\F$–progressively measurable for every $x \in \Rn$, and such that the following Stochastic Differential Equation (SDE)

\textnormal{SDE}_x \quad \begin{cases}
\mathrm{d}X_x(t) = b(t,X_x(t)) \; \mathrm{d}t + \sqrt{(a + \alpha I)(t,X_x(t))} \; \mathrm{d}W_t , \\[5pt]
\Proba\big( X_x(0) = x \big) = 1 ,
\end{cases}

holds in $(\Omega,\G,\F,\Proba)$ for $\mu_0$-almost every $x \in \Rn$. We then say that $X$ solves or is solution to SDE (with coefficients $(a,b) \in \C$). The solution $X$ to SDE is unique in the following sense: if a measurable mapping $Y : \Rn\times\Omega\to\DistSpace$ solves SDE, then it holds that $X(x,\cdot) = Y(x,\cdot)$ a.s., for $\mu_0$-almost every $x \in \Rn$.

For the well-posedness of Theorem-Definition <ref>, see Section <ref>. Solutions to SDE share a close relationship with the solutions to the Fokker-Planck equation, which we introduce next. For every $(a,b) \in \C$, we denote the Kolmogorov generator by

\Kolmo_t \varphi(y) \triangleq \frac{1}{2} \sum^n_{i,j=1} (a + \alpha I)_{ij}(t,y) \frac{\partial^2 \varphi}{\partial y_i \partial y_j}(y) + \sum^n_{i=1} b_i(t,y) \frac{\partial \varphi}{\partial y_i}(y) , \quad \varphi \in C^2(\Rn,\R) .

Let $X : \Rn\times\Omega\to\DistSpace$ be solution of SDE with coefficients $(a,b) \in \C$. Thanks to specific regularity properties of $X$ (we refer to Section <ref> for these latter), we may define the curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ of probability measures

\begin{equation} \label{eq:ShortmesureFokker}
\mu_t(A) \triangleq \int_{\Rn} \int_{\Omega} \mathds{1}_{\{ X_x(t) \in A \}} \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) , \quad A \in \Borel ,
\end{equation}

and note that $\mu$ is narrowly continuous, i.e., for every $\varphi \in C_b(\Rn,\R)$, the mapping

t \in [0,T] \mapsto \int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \int_{\Omega} \varphi(X_x(t)) \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) \in \R

is continuous. By combining this latter property with the results in [14], we introduce the following notions of Fokker-Planck equation, its solutions, and additional relationship between these solutions and the solutions to SDE (see also Section <ref>):

The curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ defined through (<ref>) is the unique narrowly continuous curve satisfying the Fokker-Planck Equation

\textnormal{FPE} \quad \begin{cases}
\displaystyle \frac{\mathrm{d}}{\mathrm{d}t} \int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \Kolmo_t \varphi(y) \mu_t(\mathrm{d}y) , \quad \varphi \in C^{\infty}_c(\Rn,\R) , \\[5pt]
\mu_{t=0} = \mu_0 .
\end{cases}

We then say that $\mu$ solves or is solution to FPE (with coefficients $(a,b) \in \C$). If $X$ denotes the solution to SDE, the following representation formula holds:

\begin{equation} \label{eq:RepFormula}
\int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \int_{\Omega} \varphi(X_x(t)) \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) , \quad \textnormal{for} \ t \in [0,T] , \ \varphi \in C_c(\Rn,\R) .
\end{equation}

§.§ Absolutely continuous solutions to the Fokker-Planck equation

Thanks to the regularity of the coefficients $(a,b) \in \C$ which is offered through Theorem <ref>, the solution $\mu$ to FPE is absolutely continuous, i.e., it takes the form

\mu_t(A) = \int_A p(t,y) \; \mathrm{d}y , \quad A \in \Borel ,

for an appropriate $p : [0,T]\times\Rn\to\R$. To elucidate this property, we first introduce broader definitions of FPE and corresponding solutions, which encompass Theorem-Definition <ref> as a sub-case (see Theorem <ref> below) and will be useful in our analysis:

Let $f \in L^2([0,T]\times\Rn,\R)$ and $\bar p \in L^2(\Rn,\R)$. A (regular enough) function $p : [0,T]\times\Rn\to\R$ is said to solve or be solution to the non-homogeneous Fokker-Planck Equation (with coefficients $(a,b) \in \C$), if

\textnormal{FPE}_f \quad \begin{cases}
\displaystyle \frac{\mathrm{d}}{\mathrm{d}t} \int_{\Rn} \varphi(y) p(t,y) \; \mathrm{d}y = \\
\displaystyle \hspace{10ex}= \int_{\Rn} \left( \Kolmo_t \varphi(y) p(t,y) + f(t,y) \varphi(y) \right) \; \mathrm{d}y , \quad \varphi \in C^{\infty}_c(\Rn,\R) , \\[10pt]
p(0,\cdot) = \bar p(\cdot) .
\end{cases}

The next theorem gathers important properties of the solution to $\textnormal{FPE}_f$, additionally showing the solution to FPE is absolutely continuous. We refer the reader to Section <ref> for exhaustive presentation and proof of these results.

For every $(a,b) \in \C$, $f \in L^2([0,T]\times\Rn,\R)$, and $\bar p \in L^2(\Rn,\R)$, there exists a unique mapping $p \in C(0,T;L^2(\Rn,\R)) \cap H^{0,1}([0,T]\times\Rn,\R)$ which solves FPE$_f$ with coefficients $(a,b) \in \C$. In addition, the following estimate holds:

\begin{align} \label{eq:Shortestimate1}
\| p (t,\cdot) \|^2_{L^2} + \int^t_0 &\| p(t,\cdot) \|^2_{H^1} \; \mathrm{d}t \le \nonumber \\
&\le C\big(\|(a,b)\|_{\Cbasis}\big) \left( \| \bar p \|^2_{L^2} + \int^t_0 \| f(s,\cdot) \|^2_{L^2} \; \mathrm{d}s \right) , \quad t \in [0,T] ,
\end{align}

where $C\big(\|(a,b)\|_{\Cbasis}\big) > 0$ is a constant which continuously depends on $\|(a,b)\|_{\C}$.

Assume $f = 0$ and $\bar p = p_0$. Then, it holds that

p(t,\cdot) \ge 0 , \quad \int_{\Rn} p(t,y) \; \mathrm{d}y = 1 , \quad t \in [0,T] ,

and therefore, if for every $t \in [0,T]$ we define

\mu_t(A) \triangleq \int_A p(t,y) \; \mathrm{d}y , \quad A \in \Borel ,

then the curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ is narrowly continuous and solves FPE. Finally, if in addition $p_0 \in H^{2 m + 1}(\Rn,\R)$, then the function $p$ satisfies

p \in H^{m+1,2(m + 1)}([0,T]\times\Rn,\R)

and the following Strong Fokker-Planck Equations (with coefficients $(a,b) \in \C$):

\textnormal{SFPE} \quad \begin{cases}
\displaystyle \frac{\partial p}{\partial t}(t,y) = (\Kolmo_t)^* p(t,y) , \quad \textnormal{a.e.} \quad (t,y) \in [0,T]\times\Rn , \\[10pt]
p(0,\cdot) = p_0(\cdot) ,
\end{cases}

where $(\Kolmo_t)^*$ denotes the dual operator of the Kolmogorov generator $\Kolmo_t$.

Combining Theorem <ref> with the representation formula (<ref>) allows us to introduce criteria to establish satisfactory guarantees for our learning approach in the context of observation and regulation of stochastic differential equations (a more exhaustive presentation is provided in Section <ref>). More specifically, let $f \in L^2([0,T]\times\Rn,\R)$ be an observation/regulation integral metric and $(a,b)\in\C$. By denoting $X^{a,b}$ and $p_{a,b}$ respectively the solution to SDE and to SFPE with coefficients $(a,b)\in\C$, we are interested in studying the accuracy with which the following observation/regulation metric is approximated by our learning approach:

\begin{equation} \label{eq:ShortintegralMetric}
\mathbb{E}_{\mu_0\times\Proba}\left[ \int^T_0 f(t,X^{a,b}_x(t)) \; \mathrm{d}t \right] = \int^T_0 \int_{\Rn} f(t,y) p_{a,b}(t,y) \; \mathrm{d}y \mathrm{d}t .
\end{equation}

To give precise estimates of the approximation error for (<ref>), the following corollary of Theorem <ref> will be crucial (see Section <ref> for a proof):

For every $(a_1,b_1) , (a_2,b_2) \in \C$, it holds that

\begin{align} \label{eq:ShortestimateOperator}
\Bigg| \mathbb{E}_{\mu_0\times\Proba}\Bigg[ \int^T_0 f(t,X^{a_1,b_1}_x(t)) \; \mathrm{d}t \Bigg] - \mathbb{E}_{\mu_0\times\Proba}\Bigg[ \int^T_0 f(t,&X^{a_2,b_2}_x(t)) \; \mathrm{d}t \Bigg] \Bigg| \nonumber \\
&\le \| f \|_{L^2} \| p_{a_1,b_1} - p_{a_2,b_2} \|_{L^2} .
\end{align}

§ USEFUL RESULTS FROM RKHS THEORY

In this section, we list classical results about Sobolev Spaces of functions with scattered zeros and Reproducing Kernel Hilbert Spaces (RKHS), which will be extensively leveraged in the following sections and for which we mainly refer to [3, 48, 42].

Let $r , s \in \mathbb{N}$ such that $r > s / 2$. Every function $u \in H^r(\R^s,\R)$ satisfies

\begin{equation}\label{eq:bessel-sobolev}
C \| u \|_{L^{\infty}} \le \| u \|_{H^r} = \| J_r(\cdot) ({\cal F} u)(\cdot) \|_{L^2} ,
\end{equation}

where $J_r(z) \triangleq (1 + \| z \|^2)^{r/2} / (2 \pi)^{s/2}$, $z \in \R^s$ denotes the Bessel potential and ${\cal F} u$ the Fourier transform of $u$, whereas the constant $C > 0$ depends on $s$ uniquely.

Let $\ell \in \mathbb{N}$, $D \subseteq \R^s$ be an open and bounded domain, and $u \in H^r(D,\R)$ be a function such that $u|_{\widehat{X}} = 0$, where $\widehat{X}_{\ell} \triangleq \{ x_1 , \dots , x_{\ell} \} \subseteq \R^s$ is a finite set of $\ell$ given points. For the fill distance, which is defined by

h_{\widehat{X}_{\ell},D} \triangleq \sup_{x \in D} \ \min_{i=1,\dots,\ell} \ \| x - x_i \| ,

the following inequality holds true, e.g., <cit.>,

\begin{equation} \label{eq:wendland}
% \|u\|_{H^q(\Omega)} \leq C_{q,m,\Omega} h_{\hat{X}, \Omega}^{m-q} \|u\|_{H^m(\Omega)}.
\| u \|_{H^{\nu}} \le C h_{\widehat{X}_{\ell},D}^{r-\nu} \| u \|_{H^r} , \quad 0 \le \nu \le r ,
\end{equation}

where the constant $C > 0$ depends on $\nu$, $r$, and $D$ uniquely. Inequality <ref> may be interpreted as follows: if $u$ is zero on a well distributed set of points over $D$, i.e., $h_{\widehat{X}_{\ell},D}$ is small, and is very regular, i.e., $r$ is large, then any norm $\|\cdot\|_{H^{\nu}}$, $0 \le \nu \le r$, e.g., the $L^2$ norm for $\nu=0$, is small over the whole domain $D$.

Given a set $D$, a RKHS $\mathcal{H}_D$ is a separable Hilbert space of functions $u : D \to \R$ such that the following reproducing property holds true:

For every point $x \in D$ there exists a reproducing function $k_x \in \mathcal{H}_D$  such that

f(x) = \scal{f}{k_x}_{\mathcal{H}_D}, \quad f \in \mathcal{H}_D.

We denote $K_D : D \times D \to \R$, the reproducing kernel associated to $\mathcal{H}_D$, i.e.,

K_D(x_1,x_2) \triangleq \scal{k_{x_1}}{k_{x_2}}_{\mathcal{H}_D}, \quad x_1 , x_2 \in D .

The following crucial result holds true, see, e.g., [42]:

Given a reproducing kernel for a RKHS $\mathcal{H}_D$, by the reproducing property, the reproducing function $k_x \in \mathcal{H}_D$ for any $x \in D$ corresponds to

k_x = K_D(x,\cdot) \in \mathcal{H}_D .

We recall that the Sobolev space $H^r(D,\R)$ is a RKHS for $r > s/2$, and $D \subseteq \R^s$ be either $\R^s$ or an open domain with Lipschitz boundary and $s \in \mathbb{N}$. In this case, the associated reproducing kernel has a known closed form, see, e.g., [48].

§ LEARNING STOCHASTIC DIFFERENTIAL EQUATIONS

In this section, we finally introduce and study the problem of learning drift and diffusion coefficients of a stochastic differential equation. For this, from now on we assume the following hypothesis, which naturally stems from our setting, to hold true (see also Definition <ref>):

$(A)$ Let $m \in \mathbb{N}$ with $m \ge 1$, $\alpha > 0$, and $p_0 \in H^{2 m + 1}(\Rn,\R)$. There exist $(\unknownDiffusion,\unknownDrift) \in \C$ and a (unique) solution $\unknownX$ to SDE with coefficients $(\unknownDiffusion,\unknownDrift) \in \C$.

Thanks to the results we gathered in Section <ref>, one readily checks that Assumption $(A)$ yields the following characterization of the mapping $\unknownX$:

With the notation $\unknownKolmo_t \triangleq \mathcal{L}^{\unknownDiffusion,\unknownDrift}_t$, there exists a unique function $\unknownDensity \in H^{m+1,2(m+1)}([0,T]\times\Rn,\R)$ which is non-negative and such that

\int_A \unknownDensity(t,x) \; \mathrm{d}x = \mu_t(A) \triangleq \int_{\Rn} \int_{\Omega} \mathds{1}_{\{ \unknownX_x(t) \in A \}} \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) , \quad \textnormal{for all} \quad A \in \Borel ,

and which satisfies:

\frac{\partial \unknownDensity}{\partial t}(t,y) = \unknownKolmo_t^* \unknownDensity(t,y) , \quad \textnormal{a.e.} \quad (t,y) \in [0,T]\times\Rn .

To achieve our goal, we proceed along three successive steps:

* First, we approximate the unknown curve of densities $\unknownDensity$ from given samples $\{ X_{x_j}(t_{\ell},\omega_j) \}_{\ell=1,\dots,M , j=1,\dots,N}$ of the unknown solution $X$ to SDE, through a finite-dimensional RKHS-based model $\approxDensity$. Thanks to this, all the quantities appearing in the learning problems we define in the next steps can be actually (tractably) computed. As is customary in many applications, we assume our data are collected by independently sampling $X$ at pre-defined times.

* Second, we approximate the unknown coefficients $(\unknownDiffusion,\unknownDrift) \in \C$ through additional coefficients $(\solDiffusion,\solDrift) \in \C$ which “best” match SFPE when evaluated at $\approxDensity$, through an appropriately well-posed infinite-dimensional learning problem. We then combine appropriate estimates with (<ref>) to show the solution to SFPE with coefficients $(\solDiffusion,\solDrift)$ well-approximates the unknown $\unknownDensity$.

* Third, we make the learning problem at the second step “tractable” by approximating the coefficients $(\solDiffusion,\solDrift)$ through a finite-dimensional RKHS-based model $(\solAppDiffusion,\solAppDrift)$. We then combine the estimates we obtained at the second step with (<ref>) to show the solution to SFPE with coefficients $(\solAppDiffusion,\solAppDrift)$ well-approximates the unknown $p$. Finally, this latter property is combined with (<ref>) to show theoretical error bounds in the context of observation and regulation of stochastic differential equations.

§.§ Approximating solutions to SDE via RKHS-based models

We start by defining our data set. We assume sampling happens at $M \in \mathbb{N}$ fixed times $0 = t_1 < \dots < t_M = T$, which are equally spaced for simplicity, i.e., $t_{\ell} \triangleq T (\ell - 1)/(M - 1)$. Then, we assume    at each time $t_{\ell}$, $\ell=1,\dots,M$, we have access to $N \in \mathbb{N}$ samples $X_{\ell,1} \triangleq X_{x_1}(t_\ell,\omega_1) , \dots , X_{\ell,N} \triangleq X_{x_N}(t_\ell,\omega_N)$ of the solution $X$ to SDE, which have been independently drawn from the same probability $\mu_{t_{\ell}}$, with density $\unknownDensity(t_{\ell},\cdot)$ (see Corollary <ref>). Now, for every $t \in [0,T]$ the family of probability measures given by

\mu_{t,k} \triangleq \underbrace{\mu_t \otimes \dots \otimes \mu_t}_{k-\textnormal{times}} : \mathcal{B}( \underbrace{\Rn \times \dots \times \Rn}_{k-\textnormal{times}} ) \to [0,1] ,\quad k \in \mathbb{N}

may be extended to a unique probability measure $\mu^{\mathbb{N}}_t : \mathcal{B}^{\mathbb{N}} \to [0,1]$ thanks to Kolmogorov (extension) theorem, and by definition, for every $\ell=1,\dots,M$ and $N \in \mathbb{N}$ the samples $\{ X_{\ell,j} \}_{j=1,\dots,N}$ may be seen as independent random variables in the probability space $((\Rn)^{\mathbb{N}},\mathcal{B}^{\mathbb{N}},\mu^{\mathbb{N}}_{t_{\ell}})$, with equal density $\unknownDensity(t_{\ell},\cdot)$. Finally, by one further application of Kolmogorov theorem, we extend the family of probability measures $\{ \mu^{\mathbb{N}}_{s_1} \otimes \dots \otimes \mu^{\mathbb{N}}_{s_k} \}_{0 \le s_1 \le \dots \le s_k \le T }$ to a unique probability measure $\ProbaX : (\mathcal{B}^{\mathbb{N}})^{[0,T]} \to [0,1]$, so that, with an abuse of notation, $\ProbaX|_{t_1,\dots,t_M} = \mu^{\mathbb{N}}_{t_1} \otimes \dots \otimes \mu^{\mathbb{N}}_{t_M}$ for every $M \in \mathbb{N}$, and for every $M , N \in \mathbb{N}$ the samples $\{ X_{\ell,j} \}_{\ell=1,\dots,M,j=1,\dots,N}$ may be seen as (not necessarily i.i.d.) random variables in the probability space $\big(((\Rn)^{\mathbb{N}})^{[0,T]},(\mathcal{B}^{\mathbb{N}})^{[0,T]},\ProbaX\big)$. Below, the assessment “with probability at least” will be meant with respect to $\ProbaX$.

At this step, fix $M, N \in \mathbb{N}$ and define the following model (random) density

\begin{align*}
\approxDensity(t,x) \triangleq \sum_{\ell=1}^M c_{\ell}(t) \widehat{g}_{\ell}(x), \quad \textnormal{where} \quad \widehat{g}_{\ell}(x) \triangleq \frac{1}{N} \sum_{j=1}^N \rho_R(x - X_{\ell,j})
\end{align*}

with $\rho_R(x) \triangleq R^{n/2}\|x\|^{-n/2} B_{n/2}(2\pi R \|x\|)$, $R > 0$ and $B_{n/2}$ is the Bessel $J$ function of order $n/2$, while $c_{\ell}(t) \triangleq e^\top_{\ell} G^{-1} v(t)$ with $\{ e_1 , \dots , e_M \}$ the canonical basis of $\R^M$ and we define $v(t) \triangleq (K_{m+1}(t,t_1),\dots, K_{m+1}(t,t_M))$. The notation $G$ is used for the Gram matrix with elements $G_{j,k} \triangleq K_{m+1}(t_j,t_k)$, where $K_{m+1}$ denotes the Sobolev kernel of smoothness degree $m+1$ (see, e.g., <cit.> for an explicit formula). Finally, for every $u \in H^{1,2}([0,T]\times\Rn,\R)$ we denote

L(u) \triangleq \int^T_0 \bigg( \bigg\| \frac{\partial \unknownDensity}{\partial t}(t,\cdot) - \frac{\partial u}{\partial t}(t,\cdot) \bigg\|^2_{L^2} + \| \unknownDensity(t,\cdot) - u(t,\cdot) \|^2_{H^2} \bigg) \; \mathrm{d}t .

Our result on the approximation of $p$ via $\approxDensity$ is as follows:

There exists a constant $C > 0$, which depends on $n$, $m$, and $T$ uniquely, such that the following learning rate for the random model $\approxDensity$ holds with probability at least $1-\delta$, for every $M , N \in \mathbb{N}$ such that $M \ge 2 T$:

\begin{align*}
L(\approxDensity) &\le C \big( \| \unknownDensity \|^2_{H^{m+1,2}} + \| \unknownDensity \|^2_{H^{1,2(m+1)}} \big) \times \\
&\hspace{30ex}\times \left( M^{-2m} + R^{-4m} + R^n \firstRev{\log\left( \frac{4 M}{\delta} \right)} N^{-1} \right) .
\end{align*}

In particular, up to overloading $C$, for every $\varepsilon > 0$ small enough we have that

\begin{align*}
% L(\approxDensity) &\le C \big( \| \unknownDensity \|^2_{H^{m+1,2}} + \| \unknownDensity \|^2_{H^{1,2(m+1)}} \big) \left( \firstRev{\log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}}} \varepsilon \right)^2 \\
L(\approxDensity) &\le C \big( \| \unknownDensity \|^2_{H^{m+1,2}} + \| \unknownDensity \|^2_{H^{1,2(m+1)}} \big) \left( \firstRev{\log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}}} \varepsilon \right)^2 \\
% &\firstRev{\le C \big( \| \unknownDensity \|^2_{H^{m+1,2}} + \| \unknownDensity \|^2_{H^{1,2(m+1)}} \big) \left( \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2} ,
&\firstRev{\le C \big( \| \unknownDensity \|^2_{H^{m+1,2}} + \| \unknownDensity \|^2_{H^{1,2(m+1)}} \big) \left( \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2} ,
\end{align*}

if $M = \varepsilon^{-1/m}/4$, $N = \varepsilon^{-(2 + n/(2m))}$ (as closest integers), and $R = \varepsilon^{-1/(2m)}$.

For the sake of clarity, we divide the proof in several steps.

1) Preliminaries. With obvious notation, fix $r , s \ge 1$ and define the operator

P_M: H^r([0,T],\R) \to H^r([0,T],\R) , \quad \textnormal{via} \quad (P_M u)(t) = \sum_{\ell=1}^M c_{\ell}(t) u(t_{\ell}) .

For $u \in H^{r,s}([0,T]\times\Rn,\R)$, we denote by $u_x$ the function $u_x(\cdot) = u(\cdot,x)$, $x \in \Rn$, and by $\tilde{u}$ the function $\tilde{u}(t,x) = (P_M u_x)(t)$.
Now, note that the function $v_x := u_x - P_M u_x$ satisfies $v_x(t_\ell) = 0$ for any $\ell=1,\dots, M$. From the results we recalled in Section <ref>, we can bound the norm of functions with scattered zeros as follows: for every $0 \le \sigma \le r$ there exists a constant $C_{r,\sigma} > 0$ such that

\begin{equation} \label{eq:ProofAle1}
\| u_x - P_M u_x \|_{H^{\sigma}} \leq C_{r,\sigma} M^{\sigma - r} \| u_x \|_{H^r},
\end{equation}

for almost every $x \in \Rn$. Below, we will often implicitly overload the constant $C_{r,\sigma}$. Moreover, we recall that, for $\alpha , \beta \ge 0$,

D^\alpha_t D^\beta_x \tilde{u}(t,x) = D^\alpha_t D^\beta_x ( P_M u_x)(t) = D^\alpha_t P_M (D^\beta_x u)(t,x) ,

so that by leveraging (<ref>) with $\sigma = \alpha$, for $0 \leq \alpha \leq r$ and $0 \leq \nu \leq s$, yields

\begin{align} \label{eq:mixed-norm-bound-in-t}
\int_0^T \| &D^{\alpha}_t u(t,\cdot) - D^{\alpha}_t \tilde{u}(t,\cdot) \|^2_{H^\nu} \; \mathrm{d}t = \\
&= \sum_{|\beta| \leq \nu} \int_0^T \int_{\Rn} (D^{\alpha}_t D^{\beta}_x u(t,x) -  D^{\alpha}_t D^{\beta}_x \tilde{u}(t,x))^2 \; \mathrm{d}x \mathrm{d}t \nonumber \\
& = \sum_{|\beta| \leq \nu} \int_{\Rn} \int_0^T (D^{\alpha}_t D^{\beta}_x u(t,x) -  D^{\alpha}_t P_M D^{\beta}_x u(t,x))^2 \; \mathrm{d}x \mathrm{d}t \nonumber \\
& \leq \sum_{|\beta| \leq \nu} \int_{\Rn} \| D^{\beta}_x u(\cdot,x) - P_M D^{\beta}_x u(\cdot,x)\|^2_{H^{\alpha}} \; \mathrm{d}x \nonumber \\
& \leq \sum_{|\beta| \leq \nu} \int_{\Rn} C^2_{r,\alpha} M^{2 (\alpha - r)} \|D^{\beta}_x u(\cdot,x)\|^2_{H^r} \; \mathrm{d}x = C^2_{r,\alpha} M^{2 (\alpha - r)} \| u \|^2_{H^{r,\nu}} . \nonumber
\end{align}

At this step, by decomposing $\unknownDensity - \approxDensity = (\unknownDensity - \widetilde{p}) + (\widetilde{p} - \approxDensity)$ and applying the triangular inequality to $L(\approxDensity)$, we obtain that

\begin{align*}
&L(\approxDensity)^{1/2} \le L(\widetilde{p})^{1/2} + A^{1/2} , \\
&A \triangleq \int^T_0 \bigg( \bigg\| \frac{\partial \widetilde{p}}{\partial t}(t,\cdot) - \frac{\partial \approxDensity}{\partial t}(t,\cdot) \bigg\|^2_{L^2} + \| \widetilde{p}(t,\cdot) - \approxDensity(t,\cdot) \|^2_{H^2} \bigg) \; \mathrm{d}t .
\end{align*}

Thanks to (<ref>) and $\unknownDensity \in H^{m+1,2(m+1)}([0,T]\times\Rn,\R)$, we may bound $L(\widetilde{p})$ by

L(\widetilde{p})^{1/2} \le C_{m+1,1} M^{-m} \| \unknownDensity \|_{H^{m+1,0}} + C_{m+1,0} M^{-(m+1)} \| \unknownDensity \|_{H^{m+1,2}} .

The rest of the proof is devoted to appropriately bounding $A$.

2) Further decomposing the term $\bm{A}$. For this, from the definition of both $\widetilde{p}$ and $\approxDensity$, for every $0 \leq \alpha \leq m+1$ and $0 \leq \nu \leq 2(m+1)$ we have that

\begin{align*}
\bigg(\int_0^T \| D^{\alpha}_t \widetilde{p}(t,\cdot) - D^{\alpha}_t \approxDensity(t,\cdot) &\|^2_{H^\nu} \; \mathrm{d}t \bigg)^{1/2} = \\
&= \left( \int_0^T \left\| \sum_{\ell=1}^M D^{\alpha}_t c_{\ell}(t) ( \unknownDensity(t_{\ell},\cdot) - \widehat{g}_{\ell}(\cdot) ) \right\|^2_{H^\nu} \mathrm{d}t \right)^{1/2} \\
& \leq \sum_{\ell=1}^M \| c_{\ell} \|_{H^{\alpha}} \| \unknownDensity(t_{\ell},\cdot) - \widehat{g}_{\ell}(\cdot) \|_{H^\nu} .
\end{align*}

In, particular, by applying this bound to the two terms in $A$, we infer that

A^{1/2} \leq \sum_{\ell=1}^M \Big( \| c_{\ell} \|_{H^1} \| \unknownDensity(t_{\ell},\cdot) - \widehat{g}_{\ell}(\cdot) \|_{L^2} + \| c_{\ell} \|_{L^2} \| \unknownDensity(t_{\ell},\cdot) - \widehat{g}_{\ell}(\cdot) \|_{H^2} \Big) .

3) Bounding each term $\bm{\| \unknownDensity(t_{\ell},\cdot) - \widehat{g}_{\ell}(\cdot) \|_{H^{\nu}}}$. Fix $\ell=1,\dots,M$ and $0 \le \nu \le 2(m+1)$. As recalled in Section <ref>, the norm of any $u \in H^\nu(\Rn,\R)$, with $\nu \in \mathbb{N}$ writes

\| u \|_{H^{\nu}} = \| J_{\nu}(\cdot) ({\cal F} u)(\cdot) \|_{L^2} ,

where ${\cal F} u$ denotes the Fourier transform of $u$, and $J_{\nu}(z) \triangleq (1 + \| z \|^2)^{\nu/2} / (2 \pi)^{n/2}$, $z \in \Rn$. In particular, since the Fourier transform of $\rho_R$ is $\mathds{1}_{B^{\Rn}_R(0)}$, it holds that

\begin{equation} \label{eq:ProofAle2}
{\cal F} \widehat{g}_{\ell}(z) = \sum_{j=1}^N \mathds{1}_{B^{\Rn}_R(0)}(z) e^{2\pi i z^\top X_{\ell,j}} = \mathds{1}_{B^{\Rn}_R(0)}(z) {\cal F} \widehat{g}_{\ell}(z) , \quad z \in \Rn ,
\end{equation}

so that, thanks to the fact that $(1 - \mathds{1}_{B^{\Rn}_R(0)}) \mathds{1}_{B^{\Rn}_R(0)} = 0$ and $\mathds{1}^2_{B^{\Rn}_R(0)} = \mathds{1}_{B^{\Rn}_R(0)}$, by denoting the function $\unknownDensity_{\ell}(\cdot) \triangleq \unknownDensity(t_{\ell},\cdot)$ we may compute

\begin{align*}
\| \unknownDensity(t_{\ell},\cdot) - \widehat{g}_{\ell}(\cdot) \|_{H^{\nu}} &= \|J_\nu ({\cal F} p_\ell - {\cal F} \widehat{g}_\ell)\|_{L^2} \\
& = \left\| J_\nu (1 - \mathds{1}_{B^{\Rn}_R(0)}){\cal F} \unknownDensity_{\ell} ~+~ J_{\nu} \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} \unknownDensity_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2} \\
& \leq \left\| J_{\nu} (1 - \mathds{1}_{B^{\Rn}_R(0)}) J_{2(m+1)}^{-1} \right\|_{L^\infty} \|J_{2(m+1)} {\cal F} \unknownDensity_{\ell} \|_{L^2} \\
&\hspace{23ex} + \left\| J_{\nu} \mathds{1}_{B^{\Rn}_R(0)} \right\|_{L^\infty} \left\| \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} p_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2} \\
& \leq R^{\nu - 2(m+1)} \| \unknownDensity_{\ell} \|_{H^{2(m+1)}} + 2^{\nu} R^{\nu} \left\| \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} p_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2} ,
\end{align*}

where we used the fact that $J_{2(m+1)}$, $J_{\nu}$, and $\mathds{1}_{B^{\Rn}_R(0)}$ are radial functions, and thus

\begin{align*}
&\left\| J_{\nu} (1 - \mathds{1}_{B^{\Rn}_R(0)}) J_{2(m+1)}^{-1} \right\|_{L^\infty} = \sup_{r > R} (1+R^2)^{-(2(m+1)+\nu)/2} \le R^{\nu-2(m+1)} , \\
&\left\| J_{\nu} \mathds{1}_{B^{\Rn}_R(0)} \right\|_{L^\infty} = \sup_{0< r \le R} (1+R^2)^{\nu/2} \le 2^{\nu} R^{\nu} , \quad \textnormal{as soon as} \quad R \ge 1 .
\end{align*}

4) Bounding each term $\bm{\left\| \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} \unknownDensity_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2}}$. For this, for every integers $\ell=1,\dots,M$ and $j=1,\dots,N$ we define a random mapping $\zeta_{\ell,j} : (\Rn)^{\mathbb{N}} \to L^2(\Rn,\mathbb{C})$ through the following expression, which holds for every $x^{\mathbb{N}} \in (\Rn)^{\mathbb{N}}$,

\zeta_{\ell,j}(x^{\mathbb{N}})(z) \triangleq \mathds{1}_{B^{\Rn}_R(0)}(z) \; e^{2 \pi i z^\top X_{\ell,j}(x^{\mathbb{N}})} , \quad z \in \Rn .

Note that, for every fixed $\ell=1,\dots,M$, since they depend deterministically from $X_{\ell,1},\dots,X_{\ell,M}$, the random mappings $\zeta_{\ell,1},\dots,\zeta_{\ell,N}$ are independent random mappings in the probability space $((\Rn)^{\mathbb{N}},\mathcal{B}^{\mathbb{N}},\mu^{\mathbb{N}}_{t_{\ell}})$, and they are additionally equally distributed with respect to $\unknownDensity_{\ell}(\cdot) \triangleq \unknownDensity(t_{\ell},\cdot)$. In particular, for every $j=1,\dots,N$ we may compute

\mathbb{E}_{\mu^{\mathbb{N}}_{t_{\ell}}}[ \zeta_{\ell,j} ](z) = \int_{\Rn} \mathds{1}_{B^{\Rn}_R(0)}(z) \; e^{2 \pi i z^\top x} \unknownDensity(t_{\ell},x) \; \mathrm{d}x = \mathds{1}_{B^{\Rn}_R(0)}(z) ({\cal F} \unknownDensity_{\ell})(z) , \quad z \in \Rn .

We may also compute

\underset{x^{\mathbb{N}} \in (\Rn)^{\mathbb{N}}}{\textnormal{ess}\sup} \ \left\| \frac{\zeta_{\ell,j}(x^{\mathbb{N}}) - \mathbb{E}_{\mu^{\mathbb{N}}_{t_{\ell}}}[ \zeta_{\ell,j} ]}{N} \right\|^2_{L^2} \le \frac{4}{N^2} \int_{\Rn} \mathds{1}_{B^{\Rn}_R(0)}(z) \; \mathrm{d}z = \frac{4}{N^2} V_n R^n ,

where $V_n$ is the volume of the $n$-dimensional unit ball. At this step, thanks to Pinelis inequality, for every $\eta > 0$ and $\ell=1,\dots,M$, we may compute

\begin{align*}
&\mu_X\left( \left\| \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} \unknownDensity_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2} > \frac{3 (V_n R)^{n/2} \log\left( \frac{4}{\eta} \right)}{\sqrt{N}} \right) \le \\
&\le \mu_X\left( \left\| \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} \unknownDensity_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2} > \frac{2 \sqrt{2} (V_n R)^{n/2} \sqrt{\log\left( \frac{2}{\eta} \right)}}{\sqrt{N}} \right) \\
&= \mu^{\mathbb{N}}_{t_{\ell}}\left( \left\| \sum_{j=1}^N \frac{\zeta_{\ell,j} - \mathbb{E}_{\mu^{\mathbb{N}}_{t_{\ell}}}[ \zeta_{\ell,j} ]}{N} \right\|_{L^2} > \frac{2 \sqrt{2} (V_n R)^{n/2} \sqrt{\log\left( \frac{2}{\eta} \right)}}{\sqrt{N}} \right) \le \eta .
\end{align*}

Therefore, for every $\delta > 0$, by taking the union bound with $\eta = \delta / M$, with probability at least $1 - \delta$ it finally holds that

\underset{\ell=1,\dots,M}{\max} \ \left\| \mathds{1}_{B^{\Rn}_R(0)} ({\cal F} \unknownDensity_{\ell} - {\cal F} \widehat{g}_{\ell}) \right\|_{L^2} \le \frac{3 (V_n R)^{n/2} \firstRev{\log\left( \frac{4 M}{\delta} \right)^{\frac{1}{2}}}}{\sqrt{N}} .

5) Bounding each term $\bm{\| c_{\ell} \|_{H^{\nu}}}$. By construction, each $c_{\ell} \in H^{m+1}([0,T],\R)$ is the function with minimum norm that satisfies $c_{\ell_1}(t_{\ell_2}) = \delta_{\ell_1,\ell_2}$, for $\ell_1 , \ell_2 = 1,\dots,M$. In particular, since the function $z_\ell(t) \triangleq \sinc(M(t-t_\ell)/T)$ is analytic, and thus  $z_{\ell} \in H^{m+1}([0,T],\R)$, and satisfies $z_{\ell_1}(t_{\ell_2}) = \delta_{\ell_1,\ell_2}$, for $\ell_1 , \ell_2 = 1,\dots,M$, it must hold that $\| c_{\ell} \|_{H^{m+1}} \leq \| z_{\ell} \|_{H^{m+1}}$, for every $\ell=1,\dots,M$. Therefore, since

\underset{\ell_1=1,\dots,M}{\max} \ | c_{\ell_2}(t_{\ell_1}) | = 1 , \quad \ell_2=1,\dots,M ,

by applying the bound for Sobolev functions with scattered zeros recalled in <ref> with $h = T/M$, we obtain that, for every $0 \le \nu \le m+1$ there exists a constant $C_{m+1,\nu} > 0$ such that

\| c_{\ell} \|_{H^{\nu}} \leq C_{m+1,\nu} \Big( ( T / M )^{-\nu} + ( T / M )^{m+1-\nu} \| z_{\ell} \|_{H^{m+1}} \Big) , \quad \ell=1,\dots,M .

To conclude, we note that the Fourier transform of the extension of each $z_{\ell}$ to $H^{m+1}(\R,\R)$ is the function $(T / M) \mathds{1}_{\{ M / T \}}$. Combining this latter result with the Fourier characterization of the norm $\| \cdot \|_{H^{m+1}}$ yields, for every $\ell=1,\dots,M$,

\displaystyle \| z_{\ell} \|^2_{H^{m+1}} \leq \frac{T}{M} \int_{-\frac{M}{2 T}}^{\frac{M}{2 T}} (1+ \tau^2)^{m+1} \; \mathrm{d}\tau \leq \left( \frac{M}{T} \right)^{2(m+1)},

where the last step follows from the assumption $M \ge 2 T$[Indeed, under this assumption, $1 \le 3/4 M^2 / T^2$, and thus $\underset{0 \le \tau \le M / (2 T)}{\max} ( 1 + \tau^2) \le M^2/T^2$.]. Therefore, for every $\ell=1,\dots,M$ we finally obtain that (we implicitly overload the constant $C_{m+1,\nu}$)

\| c_{\ell} \|_{H^{\nu}} \leq C_{m+1,\nu} \left( \frac{M}{T} \right)^{\nu} , \quad \nu=0,\dots,m+1 ,

6) Gathering the previous bounds together and end of the proof. Recalling from the results in Section <ref> that there exists a constant $C > 0$ such that

\| u \|_{L^{\infty}} \le C \| u \|_{H^1} , \quad \textnormal{for every} \quad u \in H^{m+1}([0,T],\R) ,

we may compute, for every $\ell=1,\dots,M$,

\| \unknownDensity(t_{\ell},\cdot) \|_{H^{2(m+1)}(\Rn,\R)} \le \| \unknownDensity \|_{L^\infty([0,T],\R) \otimes H^{2(m+1)}(\Rn,\R)} \le C \| \unknownDensity \|_{H^{1,2(m+1)}} .

Combining this latter inequality with all the previous bounds finally yields

\begin{align*}
L(\approxDensity)^{1/2} &\le L(\widetilde{p})^{1/2} + A^{1/2} \\
&\le C_{m+1,1} M^{-m} \| \unknownDensity \|_{H^{m+1,0}} + C_{m+1,0} M^{-(m+1)} \| \unknownDensity \|_{H^{m+1,2}} + A^{1/2} \\
&\le C \big( \| \unknownDensity \|_{H^{m+1,2}} + \| \unknownDensity \|_{H^{1,2(m+1)}} \big) \times \\
&\hspace{29ex}\times \left( M^{-m} + R^{-2m} + R^{n/2} \firstRev{\log\left( \frac{4 M}{\delta} \right)^{\frac{1}{2}}} N^{-1/2} \right) ,
\end{align*}

for some appropriate constant $C > 0$, which holds with probability at least $1 - \delta$, and the sought conclusion may be easily inferred.

The main benefit offered by replacing the unknown density $\unknownDensity$ with the model density $\approxDensity$ consists of the fact that integrals of this latter mapping and of its derivatives can be actually (easily) computed, enabling to correctly instantiate our first learning problem for $(\unknownDiffusion,\unknownDrift) \in \C$, which is our next step.

§.§ The infinite-dimensional learning problem and its fidelity

In this section, our goal consists of instantiating and analyzing the learning problem to identify the coefficients $(\unknownDiffusion,\unknownDrift) \in \C$. This problem computes coefficients which “best” matches SFPE when evaluated at the model density $\approxDensity$. Below, we seek those coefficients in the whole set $\C$, deferring to a later section the problem of learning $(\unknownDiffusion,\unknownDrift) \in \C$ through finite-dimensional models. In particular, the main result contained herein consists of appropriate learning error estimates which pave the way to obtaining learning error estimates for the problem of learning $(\unknownDiffusion,\unknownDrift) \in \C$ through finite-dimensional models. However, for pedagogical purposes, we present complete learning rates for the infinite-dimensional learning problem as well.

From now on, we fix $0 < \varepsilon , \delta < 1$, and select $M , N \in \mathbb{N}$ and $R > 0$ as claimed in Theorem <ref>, so that, with probability at least $1 - \delta$, it holds that:

\begin{align} \label{eq:ErrorDensity}
% L(\approxDensity) &\le C(\unknownDiffusion,\unknownDrift) \left( \firstRev{\log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}}} \varepsilon \right)^2 \\
L(\approxDensity) &\le C(\unknownDiffusion,\unknownDrift) \left( \firstRev{\log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}}} \varepsilon \right)^2 \\
% &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 , \nonumber
&\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 , \nonumber
\end{align}

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$ depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely. To compute accurate learning rates, we leverage classical RKHS approximation theory, which is essentially well-posed for mappings which are defined on bounded domains. For this, we will make use of the following additional assumption:

$(B)$ There exists $R_* > 0$ such that the coefficients $(\unknownDiffusion,\unknownDrift) \in \C$ satisfy:

\textnormal{supp}\big( \unknownDiffusion(t,\cdot) , \unknownDrift(t,\cdot) \big) \subseteq \overline{B^{\Rn}_{R_*}(0)} , \quad \textnormal{for every} \ t \in [0,T] .

Assumption $(B)$ plays a key role in computing learning rates which, on the one hand, leverage classical RKHS approximation theory, and which, on the other hand, depend explicitly on the parameters defining the learning problem. In particular, the latter property can not be generally obtained by just (smoothly) restricting to balls quantities which are defined on unbounded domains. That said, our result can be shown to hold under less restricting assumptions, such as assuming the coefficients $(\unknownDiffusion,\unknownDrift) \in \C$ decay to zero at infinity under specific rates [48], although we opted for Assumption $(B)$ to avoid excessively tedious computations, in turn fostering a smooth exposition. In addition, it is worth mentioning Assumption $(B)$ is often naturally verified in many applications ranging from biology to robotics, where the state space is a bounded domain.

Thanks to Assumption $(B)$, we may restrict ourselves to the closed subspace:

\CR \triangleq \Big\{ (a,b) \in \C : \ \textnormal{supp}\big( a(t,\cdot) , b(t,\cdot) \big) \subseteq \overline{B^{\Rn}_{R_*}(0)} , \ \textnormal{for every} \ t \in [0,T] \Big\} \subseteq \C ,

We are now ready to define our infinite-dimensional problem for learning the coefficients $(\unknownDiffusion,\unknownDrift) \in \CR$, which writes as follows:

For every positive real $\lambda > 0$, the (random) infinite-dimensional Learning Problem to learn stochastic differential equations is defined as:

\LearningProblem \quad \underset{(a,b) \in \CR}{\min} L_{\lambda}(a,b) \triangleq \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\Kolmo_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (a,b) \|^2_{\Cbasis} \ .

The well-posedness of problem is proven in the following proposition:

For every $\lambda > 0$, problem is well-posed and has a unique solution, which is denoted by $(\solDiffusion,\solDrift) \in \CR$.

Since the mapping

\begin{equation} \label{eq:stronglyConvexMapping}
(a,b) \in \Cbasis \mapsto \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\Kolmo_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (a,b) \|^2_{\Cbasis}
\end{equation}

is strictly convex when restricted to the closed convex subset $\CR \subseteq \Cbasis$, we just need to prove the existence of a solution to . For this, if $(a_k,b_k)_{k \in \mathbb{N}} \in \CR$ is any minimizing sequence for , there must exist some constant $C > 0$ such that $\| (a_k,b_k) \|_{\C} \le C$ for every $k \in \mathbb{N}$, and therefore, since $\CR$ is in particular a closed and convex subset of the Hilbert space $\Cbasis$, up to extracting a subsequence there exists $(\solDiffusion,\solDrift) \in \CR$ such that $(a_k,b_k)_{k \in \mathbb{N}}$ converges to $(\solDiffusion,\solDrift)$ for the weak topology of $\Cbasis$. Now, thanks to Theorem <ref>, it is clear the mapping (<ref>) is in addition continuous for the strong topology of $\Cbasis$. Thus, we infer that the mapping (<ref>) is in particular weakly lower semi-continuous, and therefore we finally obtain that

\begin{align*}
&\int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\mathcal{L}^{\solDiffusion,\solDrift}_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (\solDiffusion,\solDrift) \|^2_{\Cbasis} \le \\
&\le \underset{k\to\infty}{\liminf} \ \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\kKolmo_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (a_k,b_k) \|^2_{\Cbasis} \\
&= \underset{(a,b) \in \CR}{\min} \quad \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\Kolmo_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (a,b) \|^2_{\Cbasis} ,
\end{align*}

and the sought conclusion follows.

We now investigate the fidelity of the learning problem . More specifically, through the estimates of Section <ref>, under appropriate norms we compute bounds for the approximation error between solutions to SFPE, associated with the drift and diffusion coefficients solutions to , and the unknown mapping $\unknownDensity$. In turn, these bounds will endow our learning procedure with well-posedness and high fidelity.

We start with the following technical lemma:

For every $\lambda > 0$, with probability at least $1 - \delta$, it holds that:

\begin{align*}
% L_{\lambda}(\solDiffusion,\solDrift) &\le C(\unknownDiffusion,\unknownDrift) \left( \lambda + \left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right) \\
L_{\lambda}(\solDiffusion,\solDrift) &\le C(\unknownDiffusion,\unknownDrift) \left( \lambda + \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \right) \\
% &\le C(\unknownDiffusion,\unknownDrift) \left( \lambda + \left( \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 \right) ,
&\le C(\unknownDiffusion,\unknownDrift) \left( \lambda + \left( \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 \right) ,
\end{align*}

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$ depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely.

Thanks to Theorem <ref> and Corollary <ref>, a routine use of Hölder and Young inequalities allows us to compute

\begin{align*}
L_{\lambda}(\solDiffusion,\solDrift) - &\lambda \| (\unknownDiffusion,\unknownDrift) \|^2_{\Cbasis} \le \\
&\le \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - \unknownKolmo^*_t \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t \\
&\le 2 \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - \frac{\partial \unknownDensity}{\partial t}(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + 2 \int^T_0 \left\| \unknownKolmo^*_t (\approxDensity - \unknownDensity)(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t \\
&\le 2 \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - \frac{\partial \unknownDensity}{\partial t}(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + 2 C(\unknownDiffusion,\unknownDrift) \int^T_0 \| \approxDensity(t,\cdot) - \unknownDensity(t,\cdot) \|^2_{H^2} \; \mathrm{d}t \\
% &\le C(\unknownDiffusion,\unknownDrift) L(\approxDensity) \le C(\unknownDiffusion,\unknownDrift) \left( \log\left ( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 ,
% &\le C(\unknownDiffusion,\unknownDrift) L(\approxDensity) \le C(\unknownDiffusion,\unknownDrift) \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2} ,
&\le C(\unknownDiffusion,\unknownDrift) L(\approxDensity) \le C(\unknownDiffusion,\unknownDrift) \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2} ,
\end{align*}

where the (overloaded) constant $C(\unknownDiffusion,\unknownDrift) > 0$, which depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely, comes from the constant in (<ref>), and the conclusion follows.

We are now ready to compute error bounds between solutions to SFPE, associated with the drift and diffusion coefficients solutions to , and the unknown mappings $\unknownDensity$. It is important to note that, given the nature of our data set, which essentially depends on observations of the law of the state process, the error between the unknown densities and the densities stemming from the learned coefficients is a “good metric” with which the convergence of an identification algorithm for stochastic differential equations which leverage observations of the state process may be tested.

We better formalize this metric as follows. For every $(a,b) \in \C$, let $p_{a,b} \in H^{m+1,2(m+1)}([0,T]\times\Rn,\R)$ denote the unique solution to SFPE with coefficients $(\solDiffusion,\solDrift) \in \C$. Note that the existence and uniqueness of the regular mapping $p_{a,b} \in H^{m+1,2(m+1)}([0,T]\times\Rn,\R)$ as non-negative solution to SFPE, of unitary mass and with coefficients $(a,b) \in \C$ is immediate consequence of Theorem <ref>. We define the following metric to test the accuracy of our method:

\Metric(a,b) \triangleq \| p_{a,b} - \unknownDensity \|^2_{L^2} = \int^T_0 \| p_{a,b}(t,\cdot) - \unknownDensity(t,\cdot) \|^2_{L^2} \; \mathrm{d}t , \quad (a,b) \in \C .

Among other benefits, we will see this metric is also particularly well-suited to estimate the error which is done when computing the observation/regulation metric (<ref>).

Our main result on the accuracy of our learning method writes as follows:

Let the coefficients $(\solDiffusion,\solDrift) \in \CR$ be the unique solution to with

$\firstRev{\lambda = \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2}$. With probability at least $1 - \delta$, it holds that:

\begin{align*}
% E(\solDiffusion,\solDrift) &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2 \\
E(\solDiffusion,\solDrift) &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \\
% &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 ,
&\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 ,
\end{align*}

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$ depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely.

We define

\rho \triangleq \solDensity - \approxDensity \in C(0,T;L^2(\Rn,\R)) \cap H^{0,1}([0,T]\times\Rn,\R) ,

f(t,\cdot) \triangleq -\left( \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\mathcal{L}^{\solDiffusion,\solDrift}_t)^* \approxDensity(t,\cdot) \right) \in L^2([0,T]\times\Rn,\R) .

It is readily seen that

\begin{cases}
\displaystyle \frac{\mathrm{d}}{\mathrm{d}t} \int_{\Rn} \varphi(y) \rho(t,y) \; \mathrm{d}y = \\
\displaystyle \hspace{10ex}= \int_{\Rn} \left( \mathcal{L}^{\solDiffusion,\solDrift}_t \varphi(y) \rho(t,y) + f(t,y) \varphi(y) \right) \; \mathrm{d}y , \quad \varphi \in C^{\infty}_c(\Rn,\R) , \\[10pt]
\rho(0,\cdot) = 0 ,
\end{cases}

and therefore, thanks to Theorem <ref>, we may apply the estimate (<ref>) to $\rho$, which in combination with Lemma <ref> with the choice

$\firstRev{\lambda = \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2}$

\underset{t \in [0,T]}{\sup} \ \| \solDensity(t,\cdot) - \approxDensity(t,\cdot) \|^2_{L^2} \le C\big(\|(\solDiffusion,\solDrift)\|_{\Cbasis}\big) C(\unknownDiffusion,\unknownDrift) %\left( \log\left ( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 ,
% \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2} ,
\firstRev{\left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2} ,

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$ depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely, whereas the constant $C\big(\|(\solDiffusion,\solDrift)\|_{\Cbasis}\big) > 0$ continuously depends on $\| (\solDiffusion,\solDrift) \|_{\Cbasis}$ uniquely. Up to overloading these constants, combined with (<ref>) this latter inequality readily yields

\begin{equation} \label{eq:almostFinalBound}
\int^T_0 \| \solDensity(t,\cdot) - \unknownDensity(t,\cdot) \|^2_{L^2} \; \mathrm{d}t \le C\big(\|(\solDiffusion,\solDrift)\|_{\Cbasis}\big) C(\unknownDiffusion,\unknownDrift) %\left( \log\left ( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 .
% \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2} .
\firstRev{\left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2} .
\end{equation}

At this step, from Lemma <ref> in particular we obtain that

\begin{equation} \label{eq:boundSolCoeff}
\| (\solDiffusion,\solDrift) \|^2_{\Cbasis} \le \frac{C(\unknownDiffusion,\unknownDrift)}{\lambda} %\left( \lambda + \left( \log\left ( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 \right)
% \left( \lambda + \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2} \right) = 2 C(\unknownDiffusion,\unknownDrift) ,
\left( \lambda + \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2} \right) = 2 C(\unknownDiffusion,\unknownDrift) ,
\end{equation}

as soon as

$\firstRev{\lambda = \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2}$. Therefore, up to overloading the constant $C(\unknownDiffusion,\unknownDrift)$, the conclusion follows from combining (<ref>) with (<ref>).

§.§ The finite-dimensional learning problem and its fidelity

The learning problem we introduced in Section <ref> (see Definition <ref>) remains difficult to numerically solve. Here, we discuss appropriate finite-dimensional approximations of and error bounds ensuring the fidelity of this latter approximation, ultimately making our learning approach for stochastic differential equations accurate and tractable.

We start by recalling and adapting the approximation tools we introduced in Section <ref> to our framework.
Let $D = [0,T] \times \overline{B^{\Rn}_{R_*}(0)}$, and consider the RKHS $\mathcal{H}_D = H^{d(m)}(D,\R)$ with associated kernel $K_D$. Without loss of generality, we may assume $K_D$ and all its derivatives equal zero on $[0,T] \times \partial B^{\Rn}_{R_*}(0)$. For any set of $Q \in \mathbb{N}$ points

\widetilde{X}_D \triangleq \Big\{ (t_1,x_1) , \dots , (t_Q, x_Q) \Big\} ,

we consider the following coordinate-wise finite dimensional models to approximate the candidate drift and diffusion coefficient solutions $(\solDiffusion,\solDrift)$:

\begin{align*}%\label{eq:def-tilde-a-tildeb}
\begin{split}
\tilde{a}(t,y) &\triangleq \sum_{\ell, \ell'=1}^Q A_{\ell, \ell'} \, K_D((t,y),(t_{\ell},x_{\ell}))\, K_D((t,y),(t_{\ell'},x_{\ell'})) , \\
&\hspace{40ex} (t,y) \in D , \quad i , j \in \{1,\dots,n\} , \\
\tilde{b}(t,x) &\triangleq \sum_{\ell=1}^Q B_{\ell} K_D((t,y),(t_{\ell},x_{\ell})),  \quad (t,y) \in D , \quad i \in \{1,\dots,n\} ,
\end{split}
\end{align*}

with $A_{\ell,\ell'} \in \R^{n\times n}, B_{\ell} \in \R^n$. Note in particular that, by defining $\Phi_Q: D \to \R^{n \times Qn}$ to be the map $\Phi_Q(t,y) = \big( K_D((t,x),(t_1,x_1)) I_{n\times n} | \dots | K_D((t,y),(t_Q,x_Q)) I_{n\times n} \big)$, then

\begin{align}\label{eq:ab-with-phi}
\tilde{a}(t,y) = \Phi_Q(t,y) A \Phi_Q(t,y)^{\top}, \quad \tilde{b}(t,y) = \Phi_Q(t,y) B,
\end{align}

with $B \triangleq (B_1,\dots,B_Q)^{\top} \in \R^{Qn}$, and $A \in \R^{Qn \times Qn}$ is the $Q \times Q$ block matrix with block entries $A_{\ell,\ell'}$. Therefore, we define the finite dimensional convex subset of $\CR$ in which the tuple $(\solDiffusion,\solDrift)$ is approximated to be (zero is imposed as value outside $D$)

\begin{align*}
\CRh \triangleq \textnormal{span} \ \Big\{ (\tilde a , \tilde b) : \ A \in \R^{Qn \times Qn} , \ A \succeq 0 \ \textnormal{and} \ B \in \R^{Qn} \Big\} \subseteq \CR .
\end{align*}

Before moving to the core of this section, by leveraging the facts we recalled in Section <ref>, we provide a crucial approximation results for coefficients $(a,b) \in \CR$. Specifically, by combining the bounds for Sobolev functions with scattered zeros we listed in Section <ref> with Theorem <ref>, we obtain the following:

Denote the fill distance between $\widetilde{X}_D$ and $D$ with

h_Q \triangleq \sup_{(t,y) \in D} \ \min_{\ell = 1,\dots,Q} \|(t_{\ell}, x_{\ell}) - (t,y) \| .

There exists a constant $C > 0$ such that, for every $(a,b) \in \CR$ with $a(t,y) \succ \beta I$, $(t,y) \in D$, for some constant $\beta > 0$, there exists a tuple $(\proj(a),\proj(b)) \in \CRh$:

\begin{align*}
&\| (a,b) - (\proj(a),\proj(b)) \|_{W^{2,\infty}} \le C (\|\sqrt{a}\|^2_{H^{d(m)}} + \|b\|_{H^{d(m)}})  %h^{2 m - (n+1)/2 - 1}_Q , \\[5pt]
h^{2 m - 1}_Q , \\[5pt]
&\|(\proj(a),\proj(b)) \|_{\Cbasis} \le C (\|\sqrt{a}\|^2_{H^{d(m)}} + \|b\|_{H^{d(m)}}).
\end{align*}

First, by combining the bounds for Sobolev functions with scattered zeros <ref> in Section <ref> with Theorem <ref>, we may readily claim the existence of a constant $C > 0$ (implicitly overloaded below) and $\proj(b) \triangleq \tilde{b}$ in the form <ref> such that

\begin{equation*} %\label{eq:boundWInftyProof}
\|b - \tilde{b}\|_{W^{2,\infty}} \le C_1 \| b \|_{H^{d(m)}} %h^{2 m - (n+1)/2 - 1}_Q .
h^{2 m - 1}_Q .
\end{equation*}

We now turn to the construction of $\proj(a) \triangleq \tilde{a}$. This is slightly more elaborated and is taken from [40, 31, 39, 29], in particular, adapting the construction of [31]. Note that the matrix square root $\sqrt{\cdot}$ has uniformly bounded derivatives of any order on the set $\{A \in \R^{n\times n} | A \succ \beta I\}$. By assumption, it thus follows that $\sqrt{a} \in H^{d(m)}(D,\R^{n\times n})$, and in particular that the element-wise functions $(\sqrt{a})_{i,j} \triangleq e_i \ \sqrt{a} \ e_j \in H^{d(m)}(D,\R)$. Now, we can build our approximation $\tilde{a}$ as follows

\tilde{a}(t,x) \triangleq v(t,y)^{\top} v(t,y) \succeq 0 , \quad (t,y) \in D ,

where $v: D \to \R^{n\times n}$ is obtained by element-wise applying the bound for Sobolev functions with scattered zeros <ref> in Section <ref>. In particular, we obtain that

\begin{equation*} %\label{eq:boundWInftyProof-forsqrta}
\|(\sqrt{a})_{i,j} - v_{i,j}\|_{W^{2,\infty}} \le C \| \sqrt{a}_{i,j} \|_{H^{d(m)}} %h^{2 m - (n+1)/2 - 1}_Q , \quad i , j \in \{ 1 , \dots , n \} .
h^{2 m - 1}_Q , \quad i , j \in \{ 1 , \dots , n \} .
\end{equation*}

Therefore, since $\sqrt{a}^{\top} \sqrt{a} - v^{\top} v = \sqrt{a}^{\top} (\sqrt{a} - v) + (\sqrt{a} - v)^{\top} \sqrt{a} - (\sqrt{a} - v)^{\top} (\sqrt{a} - v)$, and $W^{2,\infty}$ is in particular a Banach algebra,
we may infer that

\|a - \tilde{a}\|_{W^{2,\infty}} \leq  2\|\sqrt{a} - v\|_{W^{2,\infty}}\|\sqrt{a}\|_{W^{2,\infty}} + \|\sqrt{a} - v\|_{W^{2,\infty}}^2 .

At this step, the inequalities

\|\sqrt{a} - v\|^2_{W^{2,\infty}} \leq \sum_{i,j=1}^n \|(\sqrt{a})_{ij} - v_{ij}\|^2_{W^{2,\infty}}

and $\|\cdot\|_{W^{2,\infty}} \leq C \|\cdot\|_{H^{d(m)}}$ may be combined to readily derive that

\|a - \tilde{a}\|_{W^{2,\infty}} \leq  C n^2 \|\sqrt{a}\|_{H^{d(m)}}^2 %h^{2 m - (n+1)/2 - 1}_Q ,
h^{2 m - 1}_Q ,

that is the first inequality claimed in the theorem. In particular, note that

v(t,y) = \sum^Q_{\ell=1} R_\ell \ K_D((t,y),(t_\ell,x_\ell)) , \quad (t,y) \in D ,

for some $R_1,\dots, R_Q \in \R^{n\times n}$. By denoting $R = (R_1, \dots, R_Q) \in \R^{n\times Qn}$, we have that

\tilde{a}(t,y) = \Phi_Q(t,y) A \Phi_Q(t,y)^{\top} , \quad (t,y) \in D ,

with $A \triangleq R R^{\top} \in \R^{Qn \times Qn}$ semi-positive-definite. We thus infer that $(\tilde{a},\tilde{b}) \in \CRh$.

To conclude, according to the results we gathered in Section <ref>, it holds that $\tilde{b} = \Pi_Q(b)$ and $v = \Pi_Q(\sqrt{a})$, where $\Pi_Q$ denotes the corresponding projection operator of each involved RKHS, and thus $\tilde{a} = \Pi_Q(\sqrt{a})^{\top} \Pi_Q(\sqrt{a})$. This implies that

\|\Pi_Q(b)\|_{H^{d(m)}} \leq \| b \|_{H^{d(m)}} \quad \textnormal{and} \quad \| \Pi_Q(\sqrt{a}) \|_{H^{d(m)}} \leq \| \sqrt{a} \|_{H^{d(m)}} .

The proof is concluded considering that $H^{d(m)}$ is in particular a Banach algebra.

In the rest of the manuscript, we adopt the notation we introduced in Theorem <ref>. We are now ready to define our finite-dimensional problem for learning the coefficients $(\unknownDiffusion,\unknownDrift) \in \CR$, which writes as follows:

For every positive real $\lambda > 0$, the (random) finite-dimensional Learning Problem to learn stochastic differential equations is defined as:

\LearningProblemFinite \quad \underset{(a,b) \in \CRh}{\min} L_{\lambda}(a,b) \triangleq \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - (\Kolmo_t)^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t + \lambda \| (a,b) \|^2_{\Cbasis} \ .

Since $\CRh$ is a finite-dimensional convex set, the well-posedness of may be proven similarly to the well-posedness of the original problem , i.e., by replicating the proof of Proposition <ref>. We thus report this result below without proof:

For every $\lambda > 0$, problem is well-posed and has a unique solution, which is denoted by $(\solAppDiffusion,\solAppDrift) \in \CRh$.

Thanks to Assumption $(A)$, up to re-scaling $\alpha$ and overloading $\unknownDiffusion$, one may always assume that $\unknownDiffusion(t,y) \succ \alpha I$, $(t,y) \in D$. This property would enable applying Theorem <ref> directly to $(\unknownDiffusion,\unknownDrift)$. Unfortunately, the setting $\unknownDiffusion(t,y) \succ \alpha I$, $(t,y) \in D$, does not fit Assumption $(B)$, and in turn the problem formulation of . Nevertheless, one may appropriately tailor the definition of $\CRh$ to allow for $\unknownDiffusion(t,y) \succ \alpha I$, $(t,y) \in D$. Indeed, in such setting Assumption $(B)$ would rather require that $\textnormal{supp}\big( a(t,\cdot) - \alpha I , b(t,\cdot) \big) \subseteq \overline{B^{\Rn}_{R_*}(0)}$, for every $t \in [0,T]$. Therefore, we should consider a larger domain $D$ in which learning is pursued, e.g., $D \triangleq [0,T] \times \overline{B^{\Rn}_{R_*+1}}$, selecting $K_D$ to equal 1 on $[0,T] \times \partial B^{\Rn}_{R_*+1}(0)$ and such that all its derivatives equal zero on $[0,T] \times \partial B^{\Rn}_{R_*+1}(0)$ (this selection is always possible under mild assumptions). Thanks to this choice, $\CRh$ may be replaced by the following subset

\begin{align*}
\textnormal{span} \ \Big\{ (\chi \tilde a + (1 - \chi) \alpha I , \tilde b) : \ &A \in \R^{Qn \times Qn} , \ A \succeq \alpha I , \ \textnormal{and} \ B \in \R^{Qn} \Big\} \subseteq \CROne ,
\end{align*}

where the definition of $\CROne$ corresponds to $\CR$'s with updated requirement $\textnormal{supp}\big( a(t,\cdot) - \alpha I , b(t,\cdot) \big) \subseteq \overline{B^{\Rn}_{R_*+1}(0)}$. Here, $\chi \in C^{\infty}([0,T]\times\R^n,[0,1])$ is some fixed cut-off function satisfying $\chi|_{\overline{B^{\Rn}_{R_*}(0)}} = 1$ and $\textnormal{supp} \; \chi \subseteq \overline{B^{\Rn}_{R_*+1}(0)}$. One readily check this updated definition does not affect the validity of Theorem <ref>, and in turn of the rates of convergence we will develop shortly.
From a numerical point of view, the computational cost of solving with this updated definition of $\CRh$ remains unchanged (see Section <ref>). Therefore, without loss of generality, from now on we assume we may apply Theorem <ref> to $(\unknownDiffusion,\unknownDrift) \in \CR$ without further ado.

The next result is a natural extension of Lemma <ref> to the setting of problem , and it represents the main result of this section.

$Q \in \mathbb{N}$ with $h_Q \leq 1$, with probability at least $1 - \delta$, it holds that:

L_{\lambda}(\solAppDiffusion,\solAppDrift) \le C(\unknownDiffusion,\unknownDrift) %\left( \left( \log\left ( \frac{1}{\delta \varepsilon} \right) \varepsilon \right)^2 + h^{2 m - 1}_Q \right) ,
% \left( \firstRev{\left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2} + h^{2 m - {\color{red} (n+1)/2 - 1}}_Q \right) ,
\left( \firstRev{\lambda + \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 + h^{2(2 m - 1)}_Q} \right) ,

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$ depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely.

Thanks to Lemma <ref> (actually, its proof), we may compute

\begin{align*}
&L_{\lambda}(\solAppDiffusion,\solAppDrift) - \lambda \| \big( \proj(\unknownDiffusion) , \proj(\unknownDrift) \big) \|^2_{\Cbasis} \le \\
&\hspace{35ex} \le \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - ( \projKolmoUnk_t )^* \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t \\
&\le 2 \int^T_0 \left\| \frac{\partial \approxDensity}{\partial t}(t,\cdot) - \unknownKolmo^*_t \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t \\
&\hspace{35ex} + 2 \int^T_0 \left\| \left( ( \projKolmoUnk_t )^* - \unknownKolmo^*_t \right) \approxDensity(t,\cdot) \right\|^2_{L^2} \; \mathrm{d}t \\
&\le C(\unknownDiffusion,\unknownDrift) \Bigg( \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \\
&\hspace{25ex} + \| (\unknownDiffusion,\unknownDrift) - \big( \proj(\unknownDiffusion),\proj(\unknownDrift) \big) \|^2_{W^{2,\infty}} \int^T_0 \| \approxDensity(t,\cdot) \|^2_{H^2} \; \mathrm{d}t \Bigg) ,
\end{align*}

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$, which we will overload below, depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely. Theorem <ref> and the choice $0 < \varepsilon , \delta < 1$ readily yield

\int^T_0 \| \approxDensity(t,\cdot) \|^2_{H^2} \; \mathrm{d}t \le C(\unknownDiffusion,\unknownDrift) .

By combining this inequality with Theorem <ref> (and Remark <ref>), we infer that

\| (\unknownDiffusion,\unknownDrift) - \big( \proj(\unknownDiffusion),\proj(\unknownDrift) \big) \|^2_{W^{2,\infty}} \int^T_0 \| \approxDensity(t,\cdot) \|^2_{H^2} \; \mathrm{d}t \le C(\unknownDiffusion,\unknownDrift) h^{2(2 m - 1)}_Q ,

yielding that

\begin{align} \label{eq:proofFirstIneq}
&L_{\lambda}(\solAppDiffusion,\solAppDrift) - \lambda \| \big( \proj(\unknownDiffusion) , \proj(\unknownDrift) \big) \|^2_{\Cbasis} \le \\
&\hspace{25ex} \le C(\unknownDiffusion,\unknownDrift) \Bigg( \lambda + \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 + h^{2( 2 m - 1 )}_Q \Bigg) . \nonumber
\end{align}

Finally, observe that Theorem <ref> additionally provides that

\| \big( \proj(\unknownDiffusion) , \proj(\unknownDrift) \big) \|^2_{\Cbasis} \le C(\unknownDiffusion,\unknownDrift) ,

which combined with (<ref>) leads to the conclusion.

Thanks to Lemma <ref>, Theorem <ref> may be straightforwardly extended to the context of the finite-dimensional learning problem (just by replicating its proof). We thus report this result without proof in the proposition below.

Let the coefficients $(\solAppDiffusion,\solAppDrift) \in \CRh$ be the unique solution to with

$\firstRev{\lambda = \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2}$ and $Q \in \mathbb{N}$ so that

$\firstRev{h_Q = \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^{\frac{1}{2 m - 1}}} \le 1$. With probability at least $1 - \delta$, it holds that:

\begin{align*}
% E(\solAppDiffusion,\solAppDrift) &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \right)^2 \\
E(\solAppDiffusion,\solAppDrift) &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2 \\
% &\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 ,
&\le C(\unknownDiffusion,\unknownDrift) \left( \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} \right)^2 ,
\end{align*}

where the constant $C(\unknownDiffusion,\unknownDrift) > 0$ depends on $\unknownDiffusion$ and $\unknownDrift$ uniquely.

We conclude this section with a result which summarizes our essential contributions, showing how our learning approach may be leveraged for efficient observation/regulation of stochastic differential equations. First, we recall the setting introduced at the end of Section <ref>. Specifically, for every coefficients $(a,b) \in \C$, we denote by $X^{a,b}$ and $p_{a,b}$ respectively the (unique) solutions to SDE and SFPE with coefficients $(a,b) \in \C$. By combining Theorem <ref> with Corollary <ref> (in particular, compare with (<ref>); more details about this remark are provided in Section <ref>), we readily obtain the following summarizing result:

Let $m \in \mathbb{N}$, $\alpha > 0$, $R_* > 0$, $p_0 \in H^{2 m + 1}(\Rn,\R)$, and $(\unknownDiffusion,\unknownDrift) \in \CR$ satisfy Assumptions $(A)$ and $(B)$, and denote by $X$ and $p$ respectively the (unique) solutions to SDE and SFPE with coefficients $(\unknownDiffusion,\unknownDrift) \in \CR$. There exists a constant $C(\unknownDiffusion,\unknownDrift) > 0$ which only depends on $\unknownDiffusion$ and $\unknownDrift$, such that by choosing the following learning parameters for the fixed precision parameters $0 < \varepsilon , \delta < 1$:

$M = \varepsilon^{-1/m}/4$, $N = \varepsilon^{-(2 + n/(2m))}$ (as closest integers), and $R = \varepsilon^{-1/(2m)}$,

* $\firstRev{\lambda = \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^2}$, and $Q \in \mathbb{N}$ so that $\firstRev{h_Q = \min\left( 1 , \left( \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \right)^{\frac{1}{2 m - 1}} \right)}$,

with probability at least $1 - \delta$, the following estimate holds:

\begin{align} \label{eq:finalRates}
% \| p - p_{\solAppDiffusion,\solAppDrift} \|_{L^2} &\le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \\
% &\le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} , \nonumber
\| p - p_{\solAppDiffusion,\solAppDrift} \|_{L^2} &\le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \\
&\le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} , \nonumber
\end{align}

where $(\solAppDiffusion,\solAppDrift) \in \CRh$ is the unique solution to the finite-dimensional learning problem , where the learning parameters have been selected as above. Therefore, for every $f \in L^2([0,T]\times\Rn,\R)$ the following observation/regulation estimate holds:

\begin{align*}
&\Bigg| \mathbb{E}_{\mu_0\times\Proba}\Bigg[ \int^T_0 f(t,X_x(t)) \; \mathrm{d}t \Bigg] - \mathbb{E}_{\mu_0\times\Proba}\Bigg[ \int^T_0 f(t,X^{\solAppDiffusion,\solAppDrift}_x(t)) \; \mathrm{d}t \Bigg] \Bigg| \le \\
% &\le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{1}{\delta \varepsilon^{\frac{1}{m}}} \right)^{\frac{1}{2}} \varepsilon \le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{(M N)^{\frac{2}{n + 2 ( 2 m + 1 )}}}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} .
&\le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{1}{\delta \varepsilon} \right)^{\frac{1}{2}} \varepsilon \le C(\unknownDiffusion,\unknownDrift) \log\left( \frac{M N}{\delta} \right)^{\frac{1}{2}} (M N)^{-\frac{2 m}{n + 2 ( 2 m + 1 )}} .
\end{align*}

To the best of our knowledge, the learning rates (<ref>) have not yet appeared in the literature on non-parametric learning of SDE. Specifically, although in recent works, see, e.g., [1, 35, 2, 28], learning rates that improve as the regularity of the drift and diffusion coefficients increases are proposed, for the first time our method offers non-asymptotic learning rates when considering estimation of multi-dimensional SDE with non-constant diffusion coefficient, under discrete-time observations of $X$.

For the sake of completeness, one may want to compare (<ref>) with existing learning rates, e.g., the rates offered by [1, 35], though as we mentioned the estimation methods provided in these works apply to less general settings. If we were to do such comparison, we would infer that, up to a log factor, (<ref>) would be “almost optimal” in that their would be sub-optimal by a factor $\displaystyle (NM)^{\frac{1}{n + 2 (2 m + 1)}}$. However, this factor exponentially converges to $1$ when $m$ tends to infinity.

§ COMPUTATIONAL CONSIDERATIONS

From a numerical viewpoint, we can solve problem exactly in closed form. Indeed, such a problem amounts to a finite semi-definite program in the variables $A \in \R^{Q n \times Q n}, B \in \R^{Qn \times 1}$, subject to $A \succeq 0$.

Problem reformulation.

Recall that we are naturally, though implicitly extending the mapping $\Phi_Q$ to zero outside $D$, smoothly. Thanks to the characterization form <ref>, which is valid for every $(a,b) \in \CRh$, one easily shows that

\Kolmo_t \approxDensity(t,y) = \tr(V(t,y) \, A) - U(t,y) B + r(t,y) , \quad (t,y) \in [0,T]\times\R^n ,

\displaystyle r(t,y) \triangleq \frac{\alpha}{2} \Delta \approxDensity(t,y) ,

and $U: [0,T]\times\R^n \to \R^{1 \times Q n}$ and $V: [0,T]\times\R^n \to \R^{Q n \times Q n}$ are defined as

\begin{align*}
% U_j(t,x) &\triangleq \sum_{i=1}^n \frac{\partial}{\partial x_i}\left[ \approxDensity(t,x) e_j^\top \Phi_Q(t,x) \right] , \quad j = 1 , \dots , Q , \\
&U_{\ell}(t,y) \triangleq \sum_{i=1}^n \frac{\partial}{\partial x_i}\left[ \approxDensity(t,y) e^{\top}_i K_D((t,y),(t_{\ell},x_{\ell})) \right] \in \R^n , \quad \ell , \ell' = 1 , \dots , Q , \\
% V(t,x) &\triangleq \frac{1}{2} \sum_{i,j=1}^n \frac{\partial^2}{\partial y_j \partial y_i}\left[ \approxDensity(t,y) \Phi_Q(t,y)^\top e_j e_i^\top \Phi_Q(t,y) \right] .
&V_{\ell,\ell'}(t,x) \triangleq \frac{1}{2} \sum_{i,j=1}^n \frac{\partial^2}{\partial y_j \partial y_i}\big[ \approxDensity(t,y) K_D((t,y),(t_{\ell},x_{\ell})) \times \\
&\hspace{45ex} \times K_D((t,y),(t_{\ell'},x_{\ell'})) e_i e^{\top}_j \big] \in \R^{n \times n} .
\end{align*}

Therefore, by denoting $\displaystyle q \triangleq \frac{\partial \approxDensity}{\partial t}$, the cost in thus writes

\begin{align*}
L_\lambda(a,b) &= \int_{[0,T]\times\R^n} \left(q(t,y) - \tr(V(t,y) A) + U(t,y) B - r(t,y) \right)^2 \mathrm{d}t \; \mathrm{d}y \\
& = \int_{[0,T]\times\R^n} (1, B^\top, \vec(A)^\top) W(t,y) (1, B^\top, \vec(A)^\top) \; \mathrm{d}t \; \mathrm{d}y \\
& = (1, B^\top, \vec(A)^\top) H (1, B^\top, \vec(A)^\top)^\top.
\end{align*}

where, by denoting $\widetilde{q} \triangleq q - r$, we define $W: [0,T]\times\R^n \to \R^{1+Qn + Qn^2 \times 1+Qn + Qn^2}$ as

W(t,y) \triangleq \begin{pmatrix} \widetilde{q}^2(t,y) & -\widetilde{q}(t,y) U(t,y) & -\widetilde{q}(t,y) \vec(V(t,y))^\top \\
-\widetilde{q}(t,y) U(t,y) & U(t,y) U(t,y)^\top & U(t,y)^\top \vec(V(t,y))^\top\\
-\widetilde{q}(t,y) \vec(V(t,y)) & \vec(V(t,y)) U(t,y) & \vec(V(t,y))\vec(V(t,y))^\top ,
\end{pmatrix}

whereas $H \in \R^{1+Qn + Q^2n^2 \times 1+Qn + Q^2n^2}$ is defined as

H \triangleq \int_{[0,T]\times\R^n} W(t,y) \; \mathrm{d}t \; \mathrm{d}y .

Summing up, equals the following semi-definite program

\underset{\begin{cases}
A \in \R^{Qn \times Qn} , B \in \R^{Qn} \\
v \triangleq (1, B, \vec(A)) \in \R^{1 + Qn + Q^2n^2} \\
A \succeq 0
\end{cases}}{\min} v^{\top} H v .

Such semi-definite programs enjoy a particularly simple form. They can be efficiently solved via damped Newton methods, with computational cost $O((Qn)^{3.5})$ [34].

Computing the integral $\bm{H}$. The computation of each entry of the matrix $H$ can be done in closed form, given that they correspond to the integral of products of functions, that are linear combinations of elementary functions. As a matter of example, below we explicitly compute the first element of $H$, i.e., $H_{11}$, the same argument holding for the other elements. We have that

\int_{[0,T]\times\R^n} W_{11}(t,y) \; \mathrm{d}t \; \mathrm{d}y = \int_{[0,T]\times\R^n} ( q(t,y)^2 - 2 q(t,y) r(t,y) + r(t,y)^2 ) \; \mathrm{d}t \; \mathrm{d}y .

Due to their similarity, we proceed with computations only for the integral of the first term in the right-hand side of this equality. For this, we may compute

\begin{align*}
&\int_{[0,T]\times\R^n} q(t,y)^2 \; \mathrm{d}t \; \mathrm{d}y \\
& = \sum_{\ell,\ell'=1}^M \sum_{j,j'=1}^N \frac{1}{N^2} \int_{[0,T]\times\R^n} \dot{c}_\ell(t) \dot{c}_{\ell'}(t) \rho_R(y-x_{\ell,j}) \rho_R(y-x_{\ell',j'}) \; \mathrm{d}t \; \mathrm{d}y \\
& = \sum_{i,i',\ell,\ell'=1}^M \sum_{j,j'=1}^N \frac{\alpha_{i\ell} \alpha_{i'\ell'}}{N^2} \int_0^T \dot{k}_m(t-t_i) \dot{k}_m(t-t_{i'}) \; \mathrm{d}t \ \times \\
&\hspace{45ex} \times \int_{\R^n} \rho_R(y-x_{\ell,j}) \rho_R(y-x_{\ell',j'}) \; \mathrm{d}y .
\end{align*}

Above we used the definition of $\widehat{p}$, and the fact that $c_\ell$ are minimum-norm kernel interpolators [48], i.e., they are given by $c_\ell(t) = \sum_{i=1}^N \alpha_{i,\ell} k_m(t-t_i)$, where $\alpha_{i,\ell} = (K^{-1})_{i,\ell}$, whereas $K \in \R^{N\times N}$ is given by $K_{i,i'} = k_m(t_i, t_{i'})$. By denoting

R_{\ell, \ell'} \triangleq \int_0^T \dot{k}_m(t-t_i) \dot{k}_m(t-t_{i'}) \; \mathrm{d}t , \quad S_{\ell, j, \ell', j'} \triangleq \int_{\R^n} \rho_R(y-x_{\ell,j})\rho_R(y-x_{\ell',j'}) \; \mathrm{d}y ,

the computations above thus amount to

\int_{[0,T]\times\R^n} q(t,y)^2 \; \mathrm{d}t \; \mathrm{d}y = \sum_{\ell,\ell'=1}^M \sum_{j,j'=1}^N (K^{-1} R K^{-1})_{\ell, \ell'} S_{\ell, j, \ell', j'} .

Assuming that the computational cost of analytically computing such integrals of elementary functions is $O(1)$ per integral, the total computational cost to compute $H_{11}$ is thus $O(M^3 + M^2 N^2)$. Under the same assumption and iterating the same argument for all the elements of $H$, since $U$ is the linear combination of $Q n^2$ elementary functions, whereas $V$ is the linear combination of $Q^2 n^4$ elementary functions, the total computational cost to compute the whole matrix $H$ is thus $O(M^3 + M^2 N^2 + Q^4 n^8)$.

Considerations on the computational complexity. Since to achieve a desired value for the fill distance $Q = O(h_Q^{-(n+1)})$ number of centers are required, taken, e.g. on a grid of stepsize $h_Q$ [40], we thus need $Q = O\left(\varepsilon^{-\frac{n + 1}{m - (n-1)/4}}\right)$ to meet the rates of Theorem <ref>. In particular, by dimensioning $M$ and $N$ according to Theorem <ref> to achieve the desired precision $\varepsilon$, it is required that $Q^{4} \leq M^2 N^2$. Therefore, the proposed algorithm enjoy the following performance:

\begin{align*}
&\textrm{1. Approximation error of $(a^*, b^*)$:} ~~~O(\varepsilon) , \\
&\textrm{2. Total number of required samples:} ~~~O(\varepsilon^{-2 - \frac{n+1/2}{2m}}) , \\
&\textrm{3. Total computational cost:} ~~~O(\varepsilon^{-4 - \frac{2n+1}{2m}} n^8) .
\end{align*}

In particular, under regular enough settings, i.e., $m \geq n+1/2$, we infer that our identification method achieves approximation error $O(\varepsilon)$ by requiring

$O(\varepsilon^{-2.5})$ samples and a total computational cost of $O(\varepsilon^{-5} n^8)$.

§ CONCLUSION AND PERSPECTIVES

In this paper, we propose a Reproducing Kernel Hilbert Space-based learning paradigm for the identification of drift and diffusion coefficients of non-linear stochastic differential equations, which relies upon discrete-time observation of the state. Under assumptions of smoothness for the unknown drift and diffusion coefficients, we provide theoretical estimates of learning rates which become increasingly tighter when both the number of observations of the state and the regularity of the unknown drift and diffusion coefficients grow.

Some possible improvements and perspectives are in order. Since our learning rates essentially apply to the laws of the state process, it would be interesting to understand whether and under what conditions our method may be extended to derive stronger $L^p$ norm-based learning rates. Finally, it would be interesting to investigate extensions of our work for the identification of controlled stochastic differential equations: although these models are crucial for the control of complex systems, e.g., in aerospace and robotics, methods which offer relevant guarantees of accuracy and efficiency of the identification process still require extensive investigation.

§ DETAILS ON STOCHASTIC DIFFERENTIAL AND FOKKER-PLANCK EQUATIONS

In this section, we provide a more structured exposition of the concepts we previously introduced in Section <ref>. For this, we chronologically retrace in more details every definition and result of Section <ref> step by step.

§.§ The Fokker-Planck equation

We recall we fixed $m \in \mathbb{N}$ and a constant $\alpha > 0$, and since we worked with diffusion coefficients which are never trivial, for every $(a,b) \in \C$, the coefficient $a$ has been replaced with the mapping

%a + \alpha I : [0,T]\times\Rn \to \SymPP \quad \textnormal{with} \quad a + \alpha I = \sqrt{a + \alpha I} \sqrt{a + \alpha I}^{\top} .
a + \alpha I : [0,T]\times\Rn \to \SymPP \quad \textnormal{with} \quad a + \alpha I = \sqrt{a + \alpha I} \sqrt{a + \alpha I} .

Also, we fixed a non-negative density $p_0 \in L^2(\Rn,\R)$ which served as appropriate initial condition, and we denoted by $\mu_0 \in \ProbaSpace(\Rn)$ the associate probability measure. We recall the following notions of stochastic differential equation and its solutions:

A measurable mapping $X : \Rn\times\Omega\to\DistSpace$ solves the Stochastic Differential Equation with coefficients $(a,b) \in \C$ if each process $X_x(t,\omega) \triangleq X(x,\omega)(t)$ is $\F$–progressively measurable for every $x \in \Rn$, and

\textnormal{SDE}_x \quad \begin{cases}
\mathrm{d}X_x(t) = b(t,X_x(t)) \; \mathrm{d}t + \sqrt{(a + \alpha I)(t,X_x(t))} \; \mathrm{d}W_t , \\[5pt]
\Proba\big( X_x(0) = x \big) = 1 ,
\end{cases}

holds in $(\Omega,\G,\F,\Proba)$ for $\mu_0$-almost every $x \in \Rn$. A solution $X$ to SDE is unique if, for every measurable mapping $Y : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$, it holds that $X(x,\cdot) = Y(x,\cdot)$ a.s., for $\mu_0$-almost every $x \in \Rn$.

The well-posedness of Definition <ref> is contained in the following theorem, together with other useful properties on solutions to SDE:

For every $(a,b) \in \C$, there exists a unique measurable mapping  $X : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$. In addition, the following properties hold true for the mapping $X$:

* The following mapping is measurable:

(x,\omega,t) \in \Rn\times\Omega\times[0,T] \mapsto X_x(t,\omega) \in \Rn .

* For every $\varphi \in C_b([0,T]\times\Rn,\R)$, the following mapping is continuous:

t \in [0,T] \mapsto \int_{\Rn} \int_{\Omega} \varphi(t,X_x(t)) \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) \in \R .

Thanks to Theorem <ref>, for every $x \in \Rn$ there exists a unique (up to stochastic indistinguishability) $\F$–adapted process $X_x : [0,T]\times\Omega \to \Rn$ with continuous sample paths which solves $\textnormal{SDE}_x$, and which satisfies the following inequality

\begin{equation} \label{eq:Burkholder}
\mathbb{E}\Big[ \Dist( X_{x_1} , X_{x_2} )^2 \Big] \le C\big( \| (a,b) \|_{\C} \big) \| x_1 - x_2 \|^2 ,
\end{equation}

where the constant $C\big( \| (a,b) \|_{\C} \big) > 0$ depends on $(a,b) \in \C$ uniquely (see, e.g., [22])[Here, we use the fact that the mapping $y \mapsto \sqrt{a(t,y) + \alpha I}$ is Lipschitz, for every $t \in [0,T]$. This is a straightforward consequence of Theorem <ref> and the fact that the mapping $A \in \SymPP \mapsto \sqrt{A}$ is Lipschitz on $\{ A \in \SymPP : \ \| A \| \le \| a \|_{L^{\infty}} + \alpha , \ y^{\top} A y \ge \alpha \| y \|^2 , y \in \Rn \}$. Indeed, the latter set is compact and the mapping $A \in \SymPP \mapsto \sqrt{A}$ is continuously differentiable.]. Therefore, we define the mapping $X : \Rn\times\Omega\to\DistSpace$ by $X(x,\omega)(t) \triangleq X_x(t,\omega)$. From the continuity of the sample paths of each $X_x$, one may show that

X(x,\cdot)^{-1}\left( \overline{B^{\DistSpace}_{\varepsilon}(w_0)} \right) = \underset{t \in [0,T] \cap \mathbb{Q}}{\bigcap} X_x(t)^{-1}\left( \overline{B^{\Rn}_{\varepsilon}(w_0(t))} \right) , \quad x \in \Rn ,

from which, by leveraging a routine monotone class argument, we easily infer the measurability of the process $(X(x,\cdot))_{x \in \Rn}$.

To prove the measurability of $X : \Rn\times\Omega\to\DistSpace$, we rather build a measurable mapping $\tilde X : \Rn\times\Omega\to\DistSpace$ such that the process $(\tilde X(x,\cdot))_{x \in \Rn}$ is a modification of $(X(x,\cdot))_{x \in \Rn}$. In particular, such property would imply for every $x \in \Rn$ the existence of a subset $N_x \in \F_T$ with $\Proba(N_x) = 1$, and such that

\begin{equation} \label{eq:modification}
\tilde X_x(t,\omega) = X_x(t,\omega) , \quad t \in [0,T] , \quad \omega \in N_x .
\end{equation}

From (<ref>), together with the completeness of $\F$, we would infer each process $\tilde X_x$ is $\F$–adapted and has continuous sample paths (the latter property being trivially true by definition). Moreover, thanks to Theorem <ref>, a routine application of Burkholder-Davis-Gundy inequality, and the fact that each process $X_x$ satisfies $\textnormal{SDE}_x$, one may straightforwardly compute, for every $x \in \Rn$,

\begin{align*}
&\mathbb{E}\left[ \underset{t \in [0,T]}{\sup} \ \left\| \tilde X_x(t) - x - \int^t_0 b(s,\tilde X_x(s)) \; \mathrm{d}r - \int^t_0 \sqrt{(a + \alpha I)(s,\tilde X_x(s))} \; \mathrm{d}W_s \right\|^2 \right] \le \\
&\le C \mathbb{E}\left[ \underset{t \in [0,T]}{\sup} \ \| \tilde X_x(t) - X_x(t) \|^2 \right] + C \mathbb{E}\left[ \left( \int^T_0 \| b(t,\tilde X_x(t)) - b(t,X_x(t)) \| \; \mathrm{d}t \right)^2 \right] \\
&\hspace{17.5ex}+ C \mathbb{E}\left[ \int^T_0 \left\| \sqrt{(a + \alpha I)(t,\tilde X_x(t))} - \sqrt{(a + \alpha I)(t,X_x(t))} \right\|^2 \; \mathrm{d}t \right] \\
&\le C \mathbb{E}\Big[ \Dist( \tilde X_x , X_x )^2 \Big] ,
\end{align*}

for some (overloaded) constant $C > 0$, and therefore by combining (<ref>) with (<ref>) yields that each process $\tilde X_x$ satisfies SDE$_x$, $x \in \Rn$. In turn, we showed the existence of a measurable mapping $X : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$. By leveraging the same argument (and a routine application of Gronwäll's inequality), one also shows the uniqueness of this mapping as in Definition <ref>.

At this step, we build the aforementioned mapping $\tilde X : \Rn\times\Omega\to\DistSpace$ via Kolmogorov's lemma. More precisely, combining (<ref>) with Kolmogorov's lemma yields the existence of a modification $(\tilde X(x,\cdot))_{x \in \Rn}$ of $(X(x,\cdot))_{x \in \Rn}$ such that, for every $\omega \in \Omega$, the mapping $x \in \Rn \mapsto \tilde X_x(\cdot,\omega) \in \DistSpace$ is continuous. Therefore, the mapping $\tilde X : \Rn\times\Omega\to\DistSpace$ is Caratheodory, and thus measurable (see, e.g., <cit.>).

At this step, note that the mapping

\pi : [0,T]\times\DistSpace \to \Rn : (t,w) \mapsto w(t)

\begin{align*}
\| \pi(t_1,w_1) - \pi(t_2,w_2) \| \le \| w_1(t_1) - w_1(t_2) \| + \Dist(w_1,w_2) ,
\end{align*}

for every $t_1 , t_2 \in [0,T]$, $w_1 , w_2 \in \DistSpace$. Hence, $\pi$ is continuous and property 1. follows from $X_x(t,\omega) = \pi\big( t , X(x,\omega) \big)$, for $(x,\omega,t) \in \Rn \times \Omega\times [0,T]$. Moreover, if $\varphi \in C_b([0,T]\times\Rn,\R)$ and $(t_k)_{k \in \mathbb{N}} \subseteq [0,T]$ satisfies $t_k \to t$, from what we just proved:

\varphi\big( t_n , \pi\big( t_n , X(x,\omega) \big) \big) \to \varphi\big( t , \pi\big( t , X(x,\omega) \big) \big) , \quad \textnormal{a.e. in} \quad \Rn\times\Omega ,

and a routine application of the dominated convergence theorem yields property 2.

At this step, for every $(a,b) \in \C$, we denote the Kolmogorov generator by

\Kolmo_t \varphi(y) \triangleq \frac{1}{2} \sum^n_{i,j=1} (a + \alpha I)_{ij}(t,y) \frac{\partial^2 \varphi}{\partial y_i \partial y_j}(y) + \sum^n_{i=1} b_i(t,y) \frac{\partial \varphi}{\partial y_i}(y) , \quad \varphi \in C^2(\Rn,\R) .

Fix $(a,b) \in \C$, and assume we are given a measurable mapping  $X : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$. For $\varphi \in C^{\infty}_c(\Rn,\R)$, a straightforward application of Itô's formula to SDE$_x$ yields

\begin{equation} \label{eq:Ito}
\int_{\Omega} \varphi(X_x(t)) \; \mathrm{d}\Proba = \varphi(x) + \int_{\Omega} \int^t_0 \Kolmo_s \varphi(X_x(s)) \; \mathrm{d}s \; \mathrm{d}\Proba , \quad t \in [0,T] ,
\end{equation}

which holds for $\mu_0$-almost every $x \in \Rn$. Thanks to Theorem <ref>, we may define the curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ of probability measures

\begin{equation*} %\label{eq:mesureFokker}
\mu_t(A) \triangleq \int_{\Rn} \int_{\Omega} \mathds{1}_{\{ X_x(t) \in A \}} \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) , \quad A \in \Borel ,
\end{equation*}

and note that $\mu$ is narrowly continuous, i.e., for every $\varphi \in C_b(\Rn,\R)$, the mapping

t \in [0,T] \mapsto \int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \int_{\Omega} \varphi(X_x(t)) \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) \in \R

is continuous. By combining this latter property with (<ref>) and Theorem <ref>
, one readily checks that the curve of probabilities $\mu$ satisfies:

A narrowly continuous curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ is said to solve the Fokker-Planck Equation with coefficients $(a,b) \in \C$ if

\textnormal{FPE} \quad \begin{cases}
\displaystyle \frac{\mathrm{d}}{\mathrm{d}t} \int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \Kolmo_t \varphi(y) \mu_t(\mathrm{d}y) , \quad \varphi \in C^{\infty}_c(\Rn,\R) , \\[5pt]
\mu_{t=0} = \mu_0 .
\end{cases}

Importantly, thanks to the regularity of the coefficients $(a,b) \in \C$, FPE can have one narrowly continuous solution $\mu$ at most, as we state in the following:

Given any tuple of coefficients $(a,b) \in \C$, at most one narrowly continuous curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ can solve FPE with coefficients $(a,b) \in \C$.

Our previous computations show that solutions to FPE may be obtained from solutions to SDE, and we now establish this process may be inverted. Specifically, motivated by the results in [43, 14], we prove that any narrowly continuous curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ solution to FPE with coefficients $(a,b) \in \C$ is associated with a unique (in the sense of Definition <ref>) measurable mapping $X : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$. We gather such result in the following theorem, which is a natural extension of <cit.> to our setting:

Let a narrowly continuous curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ be solution to FPE with coefficients $(a,b) \in \C$ (which is unique thanks to Proposition <ref>). There exists a unique measurable mapping $X : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$, and which satisfies the representation formula:

\int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \int_{\Omega} \varphi(X_x(t)) \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) , \quad \textnormal{for} \quad t \in [0,T] , \quad \varphi \in C_c(\Rn,\R) .

Thanks to Theorem <ref>, we already know there exists a unique measurable mapping  $X : \Rn\times\Omega\to\DistSpace$ which solves SDE with coefficients $(a,b) \in \C$.

To conclude we just need to show the representation formula holds true. For this, we define the curve $\bar \mu : [0,T] \to \ProbaSpace(\Rn)$ of probability measures

\bar \mu_t(A) \triangleq \int_{\Rn} \int_{\Omega} \mathds{1}_{\{ X_x(t) \in A \}} \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) , \quad A \in \Borel ,

which is well-defined and narrowly continuous thanks to Theorem <ref>. In addition, by combining this latter property with (<ref>) and Theorem <ref>, we see that the curve $\bar \mu : [0,T] \to \ProbaSpace(\Rn)$ solves FPE, and thus Proposition <ref> yields

\int_{\Rn} \varphi(y) \mu_t(\mathrm{d}y) = \int_{\Rn} \varphi(y) \bar \mu_t(\mathrm{d}y) = \int_{\Rn} \int_{\Omega} \varphi(X_x(t)) \; \mathrm{d}\Proba \; \mu_0(\mathrm{d}x) ,

for every $t \in [0,T]$ and $\varphi \in C_c(\Rn,\R)$, and the conclusion follows.

§.§ Absolutely continuous solutions to the Fokker-Planck equation

From what we showed, solutions to SDE may be found by solving FPE. In this section, we show the existence of narrowly continuous curves $\mu : [0,T] \to \ProbaSpace(\Rn)$ of type

\mu_t(A) = \int_A p(t,y) \; \mathrm{d}y , \quad A \in \Borel ,

for appropriate densities $p : [0,T]\times\Rn\to\R$, which are solutions to FPE. Note that, if such solutions to FPE exist, then they are unique thanks to Proposition <ref>. Although such existence result is classic (see, e.g., [14, 8]), we retrace its proof in Appendix <ref> to characterize the constants appearing in some appropriate estimates and regularity properties which have been paramount to derive the results in Section <ref>.

For this, let us first recall the broader definition of solution to FPE, which in particular encompasses Definition <ref> as a sub-case (see also Theorem <ref> below):

Let $f \in L^2([0,T]\times\Rn,\R)$ and $\bar p \in L^2(\Rn,\R)$. A (regular enough) function $p : [0,T]\times\Rn\to\R$ is said to solve the non-homogeneous Fokker-Planck Equation with coefficients $(a,b) \in \C$, if

\textnormal{FPE}_f \quad \begin{cases}
\displaystyle \frac{\mathrm{d}}{\mathrm{d}t} \int_{\Rn} \varphi(y) p(t,y) \; \mathrm{d}y = \\
\displaystyle \hspace{10ex}= \int_{\Rn} \left( \Kolmo_t \varphi(y) p(t,y) + f(t,y) \varphi(y) \right) \; \mathrm{d}y , \quad \varphi \in C^{\infty}_c(\Rn,\R) , \\[10pt]
p(0,\cdot) = \bar p(\cdot) .
\end{cases}

We gather results on the existence, uniqueness, and energy-type estimates for solutions to FPE$_f$, and therefore for solutions to FPE, in the following:

For every $f \in L^2([0,T]\times\Rn,\R)$ and every $\bar p \in L^2(\Rn,\R)$, there exists a unique mapping $p \in C(0,T;L^2(\Rn,\R)) \cap L^2(0,T;H^1(\Rn,\R))$ which solves FPE$_f$ with coefficients $(a,b) \in \C$, and with $\displaystyle \frac{\partial p}{\partial t} \in L^2(0,T;H^{-1}(\Rn,\R))$. In addition, the following first parabolic estimate holds:

\begin{align} \label{eq:estimate1}
\| p (t,\cdot) \|^2_{L^2} + \int^t_0 &\| p(t,\cdot) \|^2_{H^1} \; \mathrm{d}t \le \nonumber \\
&\le C\big(\|(a,b)\|_{\Cbasis}\big) \left( \| \bar p \|^2_{L^2} + \int^t_0 \| f(s,\cdot) \|^2_{L^2} \; \mathrm{d}s \right) , \quad t \in [0,T] ,
\end{align}

where $C\big(\|(a,b)\|_{\Cbasis}\big) > 0$ is a constant which continuously depends on $\|(a,b)\|_{\Cbasis}$. Finally, if $f = 0$ and $\bar p$ is a non-negative density in $L^2(\Rn,\R)$, then

p(t,\cdot) \ge 0 , \quad \int_{\Rn} p(t,y) \; \mathrm{d}y = 1 , \quad t \in [0,T] ,

and therefore, if for every $t \in [0,T]$ we define

\mu_t(A) \triangleq \int_A p(t,y) \; \mathrm{d}y , \quad A \in \Borel ,

then the curve $\mu : [0,T] \to \ProbaSpace(\Rn)$ is narrowly continuous and solves FPE.

The regularity of the mapping $f$ may be weakened (see, e.g., [25, 8]), although the requirement $f \in L^2([0,T]\times\Rn,\R)$ already fits our purpose.

The proof of Theorem <ref> is reported in Appendix <ref> and is based on the classical Lions scheme (see, e.g., [25, 10, 13]). Among straightforward benefits, we recall Theorem <ref> enables introducing rigorous criteria to establish satisfactory guarantees for our learning approach in many circumstances. Let us better introduce this concept below. As a matter of example, when dealing with stochastic differential equations in applications such as observation and regulation, one must often manipulate metrics

\begin{equation} \label{eq:integralMetric}
\mathbb{E}_{\mu_0\times\Proba}\left[ \int^T_0 f(t,X_x(t)) \; \mathrm{d}t \right] = \int^T_0 \left( \int_{\R^{2n}} f(t,y) \; \Proba_{X_x(t)}(\mathrm{d}y) \; \mu_0(\mathrm{d}x) \right) \; \mathrm{d}t ,
\end{equation}

where the measurable mapping $X : \Rn\times\Omega \to \DistSpace$ solves SDE for some coefficients $(a,b) \in \C$, and the mapping $f : [0,T]\times\Rn\to\R$ is regular enough. In the case the coefficients $(a,b) \in \C$ of SDE are to be learned, one does not have perfect knowledge of (<ref>), and therefore the error between (<ref>) and its counterpart in which $(X_x)_{x\in\Rn}$ is replaced with the solution to SDE stemming from rather learned coefficients must be estimated. This gap is filled with the following:

For any coefficients $(a,b)\in\C$, let $\XCoeff : \Rn\times\Omega \to \DistSpace$ denote the unique measurable mapping which solves SDE with coefficients $(a,b) \in \C$ (see Theorem <ref>). There exists an operator

\begin{align*}
\OperatorSDE : \ &\C \to L^{\infty}(0,T;L^2(\Rn,\R)^*) \cong L^{\infty}(0,T;L^2(\Rn,\R)) \\
&(a,b) \mapsto \OperatorSDECoeff_{\cdot}(\cdot)
\end{align*}

which, for every $t \in [0,T]$ and every $\varphi \in C_c(\Rn,\R)$, satisfies

\OperatorSDECoeff_t(\varphi) = \int_{\R^{2n}} \varphi(y) \; \Proba_{\XCoeff_x(t)}(\mathrm{d}y) \; \mu_0(\mathrm{d}x) .

In addition, if $\densityCoeff \in C(0,T;L^2(\Rn,\R))$ denotes the unique solution to FDE$_0$ with coefficients $(a,b)\in\C$ and $\bar p = p_0$ (which uniquely exists thanks to Theorem <ref>), for every $f \in L^2([0,T]\times\Rn,\R)$ the mapping

t \in [0,T] \mapsto \OperatorSDECoeff_t\big( f(t,\cdot) \big) \in \R

is measurable and in $L^1([0,T],\R)$, and it satisfies, for every $(a_1,b_1) , (a_2,b_2) \in \C$,

\begin{equation} \label{eq:estimateOperator}
\int^T_0 \big| \OperatorSDE^{a_1,b_1}_t\big( f(t,\cdot) \big) - \OperatorSDE^{a_2,b_2}_t\big( f(t,\cdot) \big) \big| \; \mathrm{d}t \le \| f \|_{L^2} \| p_{a_1,b_1} - p_{a_2,b_2} \|_{L^2} .
\end{equation}

Thanks to the previous Corollary, we may provide (<ref>) with a rigorous meaning by defining, for every $f \in L^2([0,T]\times\Rn,\R)$,

\mathbb{E}_{\mu_0\times\Proba}\left[ \int^T_0 f(t,\XCoeff_x(t)) \; \mathrm{d}t \right] \triangleq \int^T_0 \OperatorSDECoeff_t\big( f(t,\cdot) \big) \; \mathrm{d}t .

As we showed in Section <ref>, the estimate (<ref>) enables the control of any estimation error occurring during the computation of the metric (<ref>). Note that more refined estimates than (<ref>) may be easily obtained by requiring more regularity on the function $f$ (e.g., see the proof of Corollary <ref> below).

Fix any $(a,b)\in\C$. For $t \in [0,T]$ and $\varphi \in L^2(\Rn,\R)$, we define

\OperatorSDECoeff_t(\varphi) = \underset{k\to\infty}{\lim} \ \int_{\R^{2n}} \varphi_k(y) \; \Proba_{\XCoeff_x(t)}(\mathrm{d}y) \; \mu_0(\mathrm{d}x) ,

where $(\varphi_k)_{k\in\mathbb{N}} \subseteq C_c(\Rn,\R)$ is any sequence which converges to $\varphi$ for the strong topology of $L^2(\Rn,\R)$. This definition is well-posed thanks to Theorem <ref>: indeed, the limit above uniquely exists, given that if $(\varphi^1_k)_{k\in\mathbb{N}} , (\varphi^2_k)_{k\in\mathbb{N}} \subseteq C_c(\Rn,\R)$ are two sequences which converge to $\varphi$ for the strong topology of $L^2(\Rn,\R)$, we may compute

\begin{align*}
\bigg| \int_{\R^{2n}} \varphi^1_k(y) \; \Proba_{\XCoeff_x(t)}(\mathrm{d}y) \; \mu_0(\mathrm{d}x) - &\int_{\R^{2n}} \varphi^2_k(y) \; \Proba_{\XCoeff_x(t)}(\mathrm{d}y) \; \mu_0(\mathrm{d}x) \bigg| \le \\
&\le \| \densityCoeff(t,\cdot) \|_{L^2} \| \varphi^1_k - \varphi^2_k \|_{L^2} \to 0 , \quad k\to\infty .
\end{align*}

Similarly, one may easily prove that $\OperatorSDECoeff_t(\cdot) \in L^2(\Rn,\R)^* \cong L^2(\Rn,\R)$ for every $t \in [0,T]$. In particular, note that thanks to (<ref>), for $t \in [0,T]$ and $\varphi \in L^2(\Rn,\R)$,

\begin{equation} \label{eq:boundOperatorSDE}
|\OperatorSDECoeff_t(\varphi)| \le C(a,b) \| p_0 \|_{L^2} \| \varphi \|_{L^2} ,
\end{equation}

for some appropriate constant $C(a,b) > 0$.

Finally, let $g \in L^2([0,T]\times\Rn,\R)$ and let $(g_k)_{k\in\mathbb{N}} \subseteq C_c([0,T]\times\Rn,\R)$ converge to $g$ for the strong topology of $L^2([0,T]\times\Rn,\R)$. In particular, up to some subsequence, we infer that each sequence $\big( g_k(t,\cdot) \big)_{k\in\mathbb{N}} \subseteq C_c([0,T]\times\Rn,\R)$ converges to $g(t,\cdot) \in L^2(\Rn,\R)$ for the strong topology of $L^2(\Rn,\R)$, for almost every $t \in [0,T]$. Hence, the definition of $\OperatorSDECoeff_t(\cdot)$ and Theorem <ref> yield

\begin{equation} \label{eq:limitOperatorSDE}
\OperatorSDECoeff_t\big( g(t,\cdot) \big) = \underset{k\to\infty}{\lim} \ \int_{\Rn} g_k(t,y) \densityCoeff(t,y) \; \mathrm{d}y , \quad \textnormal{for almost every $t \in [0,T]$} .
\end{equation}

On the one hand, combined with Theorem <ref> and Pettis' theorem, (<ref>) yields the Bochner-measurability of the mapping $t \in [0,T] \mapsto \OperatorSDECoeff_t(\cdot) \in L^2(\Rn,\R)^*$, and therefore from (<ref>) we deduce that $\OperatorSDECoeff_{\cdot}(\cdot) \in L^{\infty}(0,T;L^2(\Rn,\R)^*)$. On the other hand, (<ref>) yields the measurability of the mapping $t \in [0,T] \mapsto \OperatorSDECoeff_t\big( f(t,\cdot) \big) \in \R$, which together with (<ref>) makes the mapping $t \in [0,T] \mapsto \OperatorSDECoeff_t\big( f(t,\cdot) \big) \in \R$ measurable and in $L^1([0,T],\R)$. The conclusion of the proof follows from the fact that (<ref>) is a straightforward consequence of (<ref>) and Fatou's lemma.

§.§ Additional regularity of solutions to the Fokker-Planck equation

In Section <ref>, we dealt with solutions $p : [0,T]\times\Rn\to\R$ of FPE$_0$ which enjoy higher regularity properties, and specifically for which $p \in H^{m+1,2(m+1)}([0,T]\times\Rn,\R)$, and for which FPE$_0$ holds pointwise. Below, we are going to show such additional attributes stem from the properties of coefficients $(a,b) \in \C$ listed in Theorem <ref>.

First, for every $(a,b) \in \C$ and almost every $t \in [0,T]$ we introduce the notation:

(\Kolmo_t)^* u(t,y) \triangleq \frac{1}{2} \sum^n_{i,j=1} \frac{\partial^2 \big( a^{\alpha}_{ij} u \big)}{\partial y_j \partial y_i}(t,y) - \sum^n_{i=1} \frac{\partial \big( b_i u \big)}{\partial y_i}(t,y) , \quad u \in L^2(0,T;H^2(\Rn,\R)) ,

which denotes the dual operator of $\Kolmo_t$. This operator is well-defined with image in $L^2([0,T]\times\Rn,\R)$. Indeed, thanks to Theorem <ref>, one readily shows that

\begin{align*}
&b_i u \in L^2(0,T;H^1(\Rn,\R)) , \quad \textnormal{with} \quad \frac{\partial \big( b_i u \big)}{\partial y_r} = b_i \frac{\partial u}{\partial y_r} + u \frac{\partial b_i}{\partial y_r} \in L^2([0,T]\times\Rn,\R) , \\
&a_{ij} u \in L^2(0,T;H^2(\Rn,\R)) , \quad \textnormal{with} \quad \frac{\partial \big( a_{ij} u \big)}{\partial y_r} = a_{ij} \frac{\partial u}{\partial y_r} + u \frac{\partial a_{ij}}{\partial y_r} \in L^2([0,T]\times\Rn,\R) , \\
&\hspace{30.5ex}\textnormal{and} \quad \frac{\partial^2 \big( a^{\alpha}_{ij} u \big)}{\partial y_r \partial y_s} = a^{\alpha}_{ij} \frac{\partial^2 u}{\partial y_r \partial y_s} + \frac{\partial a^{\alpha}_{ij}}{\partial y_s} \frac{\partial u}{\partial y_r} \\
&\hspace{37.5ex}+ \frac{\partial a^{\alpha}_{ij}}{\partial y_r} \frac{\partial u}{\partial y_s} + u \frac{\partial^2 a^{\alpha}_{ij}}{\partial y_r \partial y_s} \in L^2([0,T]\times\Rn,\R) ,
\end{align*}

for $u \in L^2(0,T;H^2(\Rn,\R))$, $i,j,r,s=1,\dots,n$, thus the well-posedness of $(\Kolmo_t)^*$.

Thanks to Theorem <ref>, the following higher regularity result holds:

Under the setting and notation of Theorem <ref>, if furthermore $\bar p \in H^{2 \expo + 1}(\Rn,\R)$, then the unique solution $p : [0,T]\times\Rn\to\R$ to FPE$_0$ with coefficients $(a,b) \in \C$ is additionally such that

p \in H^{m+1}(0,T;H^{2(m + 1)}(\Rn,\R)) .

In particular, the mapping $p$ additionally satisfies the following Strong Fokker-Planck Equations with coefficients $(a,b) \in \C$:

\textnormal{SFPE} \quad \begin{cases}
\displaystyle \frac{\partial p}{\partial t}(t,y) = (\Kolmo_t)^* p(t,y) , \quad \textnormal{a.e.} \quad (t,y) \in [0,T]\times\Rn , \\[10pt]
p(0,\cdot) = \bar p(\cdot) .
\end{cases}

This result is classically proved under either time-independent coefficients or bounded domains (e.g., [13, 26]). Although in classical reference which consider time-dependent coefficients such as [26] it is explicitly mentioned therein that their results may be extended to unbounded domains, we found such extension non-straightforward. We therefore decided to revisit and extend the regularity results for parabolic equations which are contained in <cit.> to time-dependent coefficients and unbounded domains, and to report the proof in Appendix <ref>.

§ PROOFS OF SECTIONS <REF> AND <REF>

§.§ Proofs of Section <ref>

The first and second statements in Theorem <ref> stem from Morrey theorem as soon as we show the existence of a linear and bounded operator:

E : H^1([0,T]\times\Rn,\R) \to H^1(\R^{n+1},\R) ,

for which there exists a constant $C > 0$ such that:

* $E u|_{[0,T]\times\Rn} = u$, for every $u \in H^1([0,T]\times\Rn,\R)$,

* $\| E u \|_{L^2} \le C \| u \|_{L^2}$ and $\| E u \|_{H^1} \le C \| u \|_{H^1}$, for every $u \in H^1([0,T]\times\Rn,\R)$,

for instance, see [9]. For this, we may follow the proof of <cit.>. More specifically, let $\eta \in C^{\infty}(\R,[0,1])$ be such that

\eta(t) = \left\{\begin{array}{cc}
1 , & \displaystyle t < \frac{T}{4} , \\
0 , & \displaystyle t > \frac{3}{4}T ,
\end{array}\right.

and for every $u : [0,T]\times\Rn\to\R$ define the mappings

u^-(t,y) \triangleq \left\{\begin{array}{cc}
u(t,y) , & 0 < t \le T , \\
0 , & t \le 0
\end{array}\right. \quad \textnormal{and} \quad u^+(t,y) \triangleq \left\{\begin{array}{cc}
u(t,y) , & 0 \le t < T , \\
0 , & t \ge T .
\end{array}\right.

If $u \in H^1([0,T]\times\Rn,\R)$, one sees that $\eta u^+ \in H^1([0,\infty)\times\Rn,\R)$ with

\frac{\partial \eta u^+}{\partial t} = \eta \left( \frac{\partial u}{\partial t}\right)^+ + u^+ \frac{\partial \eta}{\partial t} \quad \textnormal{and} \quad \frac{\partial \eta u^+}{\partial y_i} = \eta \left( \frac{\partial u}{\partial y_i} \right)^+ , \quad i=1,\dots,n ,

and that $(1 - \eta) u^- \in H^1((-\infty,T]\times\Rn,\R)$ with, for every $i=1,\dots,n$,

\frac{\partial (1 - \eta) u^-}{\partial t} = (1 - \eta) \left( \frac{\partial u}{\partial t}\right)^- + u^- \frac{\partial \eta}{\partial t} \quad \textnormal{and} \quad \frac{\partial (1 - \eta) u^-}{\partial y_i} = (1 - \eta) \left( \frac{\partial u}{\partial y_i} \right)^-  .

At this step, since $u = \eta u + (1 - \eta) u$, we first extend $\eta u^+$ by reflection at $t = 0$ through a mapping $v^1 \in H^1(\R^{n+1},\R)$ which thus satisfies

\| v^1 \|_{L^2} \le C \| u \|_{L^2} \quad \textnormal{and} \quad \| v^1 \|_{H^1} \le C \| u \|_{H^1} ,

and then we extend $(1 - \eta) u^-$ by reflection at $t = T$ through another mapping $v^2 \in H^1(\R^{n+1},\R)$ which thus satisfies

\| v^2 \|_{L^2} \le C \| u \|_{L^2} \quad \textnormal{and} \quad \| v^2 \|_{H^1} \le C \| u \|_{H^1} ,

where the constant $C > 0$ only depends on $\eta$. The conclusion easily follows if we define the linear and bounded operator $E$ as $E u \triangleq v^1 + v^2$.

§.§ Proofs of Section <ref>

Below, we will make use of the notation $a^{\alpha} \triangleq a + \alpha I$.

The proof is standard and based on the classical Lions scheme (see, e.g., [25, 10, 13]), therefore we mainly focus on deriving the constant $C\big(\|(a,b)\|_{\Cbasis}\big) = C\big(\alpha,\|(a,b)\|_{\Cbasis}\big)$ in (<ref>) and the properties of $p $ when $f = 0$ and $\bar p$ is a non-negative density in $L^2(\Rn,\R)$. For this, consider the Gelfand triple

V \triangleq H^1(\Rn,\R) = H^1_0(\Rn,\R) \hookrightarrow H \triangleq L^2(\Rn,\R) \hookrightarrow V^* = H^{-1}(\Rn,\R)

and define the $t$-measurable bilinear form

\begin{align*}
\Bil : \ &[0,T] \times V \times V \to \R \\
&(t;u,v) \mapsto \sum^n_{i,j=1} \int_{\Rn} \frac{1}{2} a^{\alpha}_{ij}(t,y) \frac{\partial u}{\partial y_j}(y) \frac{\partial v}{\partial y_i}(y) \; \mathrm{d}y \\
&\hspace{20ex}+ \sum^n_{i=1} \int_{\Rn} \left( \sum^n_{j=1} \frac{1}{2} \frac{\partial a^{\alpha}_{ij}}{\partial y_j}(t,y) - b_i(t,y) \right) \frac{\partial v}{\partial y_i}(y) u(y) \; \mathrm{d}y .
\end{align*}

Thanks to Theorem <ref>, one may prove that the form $\Bil$ is continuous and semi-coercive (see also the computations for the estimate (<ref>) below). Therefore, thanks to a straightforward modification of the proof of <cit.>, there exists a unique solution $p \in C(0,T;H) \cap L^2(0,T;V)$ to the variational problem

\begin{equation} \label{eq:VarProblem}
\begin{cases}
\displaystyle \frac{\mathrm{d}}{\mathrm{d}t} ( p(t,\cdot) , \varphi )_H + \Bil(t;p(t,\cdot),\varphi) = ( f(t,\cdot) , \varphi )_H , \quad \varphi \in V , \\[10pt]
p(0,\cdot) = \bar p(\cdot) ,
\end{cases}
\end{equation}

which in addition satisfies $\displaystyle \frac{\partial p}{\partial t} \in L^2(0,T;V^*)$. At this step, thanks to Theorem <ref>

a^{\alpha}_{ij}(t,\cdot) p(t,\cdot) \in H^1(\Rn,\R) , \quad \textnormal{with} \quad \frac{\partial (a^{\alpha}_{ij} p)}{\partial y_j}(t,\cdot) = p(t,\cdot) \frac{\partial a^{\alpha}_{ij}}{\partial y_j}(t,\cdot) + a^{\alpha}_{ij}(t,\cdot) \frac{\partial p}{\partial y_j}(t,\cdot) ,

for every $i,j=1,\dots,n$ and almost every $t \in [0,T]$. Therefore, for every $\varphi \in C^{\infty}_c(\Rn;\R)$, $i,j=1,\dots,n$, and almost every $t \in [0,T]$, it holds that

\begin{align*}
\int_{\Rn} a^{\alpha}_{ij}(t,y) p(t,y) &\frac{\partial^2 \varphi}{\partial y_j \partial y_i}(y) \; \mathrm{d}y = \\
&= -\int_{\Rn} \left( a^{\alpha}_{ij}(t,y) \frac{\partial p}{\partial y_j}(t,y) + p(t,y) \frac{\partial a^{\alpha}_{ij}}{\partial y_j}(t,y) \right) \frac{\partial \varphi}{\partial y_i}(y) \; \mathrm{d}y ,
\end{align*}

showing that $p$ solves FPE$_f$ with coefficients $(a,b) \in \C$, $\alpha > 0$.

Next, since $p \in L^2(0,T;V)$ and $\displaystyle \frac{\partial p}{\partial t} \in L^2(0,T;V^*)$, from (<ref>) we may compute

\begin{align*}
\frac{\mathrm{d}}{\mathrm{d}t} \frac{\| p(t,\cdot) \|^2_{H}}{2} = \left\langle \frac{\partial p}{\partial t}(t,\cdot) , p(t,\cdot) \right\rangle_{V^*} = ( f(t,\cdot) , p(t,\cdot) )_H - \Bil(t;p(t,\cdot),p(t,\cdot)) ,
\end{align*}

and therefore Young's inequality yields

\begin{align*}
\frac{\mathrm{d}}{\mathrm{d}t} \frac{\| p(t,\cdot) \|^2_{H}}{2} &\le \| f(t,\cdot) \|_{H} \| p(t,\cdot) \|_{H} - \frac{\alpha}{2} \| \nabla p(t,\cdot) \|^2_{H} \\
&\hspace{35ex} + \bar C(a,b) \sum^n_{i=1} \int_{\Rn} \left| \frac{\partial p(t,y)}{\partial y_i} p(t,y) \right| \; \mathrm{d}y \\
&\le \frac{\| f(t,\cdot) \|^2_{H}}{2} + \frac{\| p(t,\cdot) \|^2_{H}}{2} - \frac{\alpha}{2} \| \nabla p(t,\cdot) \|^2_{H} \\
&\hspace{28ex} + n M \bar C(a,b)^2 \frac{\| p(t,\cdot) \|^2_{L^2}}{2} + \frac{\| \nabla p(t,\cdot) \|^2_{H}}{2 M} \\
&\le \big(1 + n M \bar C(a,b)^2 \big) \frac{\| p(t,\cdot) \|^2_{H}}{2} + \frac{\| f(t,\cdot) \|^2_{H}}{2} - \frac{\alpha}{4} \| \nabla p(t,\cdot) \|^2_{H} ,
\end{align*}

where $M > 2 / \alpha$ is some large enough constant which stems from applying Young's inequality, whereas $\bar C(a,b) > 0$ is a constant which stems from Theorem <ref> and continuously depends on the $L^{\infty}$ norms of $b$ and of the derivatives of $a$ uniquely, and a routine application of Gronwäll's inequality together with Theorem <ref> yield (<ref>).

Finally, assume $f = 0$ and that $\bar p$ is a non-negative density in $L^2(\Rn,\R)$. Thanks to the fact that $u^+ , u^- \in H^1(\Rn,\R)$ for every $u \in H^1(\Rn,\R)$, with $\nabla (u^+) = \nabla u \mathds{1}_{\{ u > 0 \}}$ and $\nabla (u^-) = \nabla u \mathds{1}_{\{ u < 0 \}}$, and that $u^- = (-u)^+$, leveraging the notation we introduced previously and applying <cit.> with $u = -p$ yield

\begin{align*}
\frac{\mathrm{d}}{\mathrm{d}t} &\frac{\| p(t,\cdot)^- \|^2_{H}}{2} = -\left\langle \frac{\partial p}{\partial t}(t,\cdot) , p(t,\cdot)^- \right\rangle_{V^*} \\
&= -\sum^n_{i,j=1} \int_{\Rn} \frac{1}{2} a^{\alpha}_{ij}(t,y) \frac{\partial (p^+ - p^-)}{\partial y_j}(t,y) \frac{\partial (-p^-)}{\partial y_i}(t,y) \; \mathrm{d}y \\