and $\|\psi_{m}\|_{W^{1,\infty}(\Omega_{m})}$ as follows

$\displaystyle\|\phi_{m}\|_{L^{\infty}(\Omega_{m})}$
$\displaystyle\leq\|\phi_{m}\|_{L^{\infty}([0,1]^{d})}\leq\|g_{m}\|_{L^{\infty}([0,1]^{d})}+\|\phi_{m}-g_{m}\|_{L^{\infty}([0,1]^{d})}$
$\displaystyle\leq 1+50d^{5/2}\lesssim d^{5/2},$
$\displaystyle\|\phi_{m}\|_{W^{1,\infty}(\Omega_{m})}$
$\displaystyle\leq\|\phi_{m}\|_{W^{1,\infty}([0,1]^{d})}\leq\|g_{m}\|_{W^{1,\infty}([0,1]^{d})}+\|\phi_{m}-g_{m}\|_{W^{1,\infty}([0,1]^{d})}$
$\displaystyle\leq 4\lfloor N^{1/d}\rfloor^{2}\lfloor
L^{2/d}\rfloor+50d^{5/2},$
$\displaystyle\|\psi_{m}\|_{L^{\infty}(\Omega_{m})}\leq\|f\|_{L^{\infty}(\Omega_{m})}+\|\psi_{m}-f\|_{L^{\infty}(\Omega_{m})}\lesssim
1,$
$\displaystyle\|\psi_{m}\|_{W^{1,\infty}(\Omega_{m})}\leq\|f\|_{W^{1,\infty}([0,1]^{d})}+\|\psi_{m}-f\|_{W^{1,\infty}([0,1]^{d})}\lesssim
1.$

Let
$B_{1}:=\max_{m\in\\{1,2\\}^{d}}\\{\|\phi_{m}\|_{L^{\infty}(\Omega_{m})},\|\psi_{m}\|_{L^{\infty}(\Omega_{m})}\\}$,
then it yields that $B_{1}\lesssim d^{5/2}$ by the estimates of
$\|\phi_{m}\|_{L^{\infty}(\Omega_{m})}$ and
$\|\psi_{m}\|_{W^{1,\infty}(\Omega_{m})}$. Let
$B_{2}:=\max_{m\in\\{1,2\\}^{d}}\\{\|\phi_{m}\|_{W^{1,\infty}(\Omega_{m})},\|\psi_{m}\|_{W^{1,\infty}(\Omega_{m})}\\}$.
Similarly, it yields that $B_{2}\lesssim(NL)^{2/d}+d^{5/2}$. By Lemma F.3, for
any $N,L\in\mathbb{N}$, there exists a deep ReLU network $\phi_{\times,B_{1}}$
with width $15(N+1)$ and depth $16L$ such that
$\|\phi_{\times,B_{1}}\|_{W^{1,\infty}((-B_{1},B_{1})^{2})}\leq 12B_{1}^{2}$
and

$\displaystyle\|\phi_{\times,B_{1}}(x,y)-xy\|_{W^{1,\infty}((-B_{1},B_{1})^{2})}\leq
6B_{1}^{2}(N+1)^{-8L}.$

To obtain a global estimate on $[0,1]^{d}$, we combine the local estimate
$\\{\psi_{m}\\}_{m\in\\{1,2\\}^{d}}$ and the approximate partition of unity
$\\{\phi_{m}\\}_{m\in\\{1,2\\}^{d}}$. Let us construct the global
approximation function $\phi$ by

$\displaystyle\phi(x):=\sum_{m\in\\{1,2\\}^{d}}\phi_{\times,B_{1}}(\phi_{m}(x),\psi_{m}(x)).$
(B.3)

Next, we bound the error of the global approximation estimate by

$\displaystyle\|f-\phi\|_{L^{\infty}([0,1]^{d})}=$
$\displaystyle\|\sum\nolimits_{m\in\\{1,2\\}^{d}}g_{m}f-\phi\|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle\leq$
$\displaystyle\underbrace{\|\sum\nolimits_{m\in\\{1,2\\}^{d}}[g_{m}f-\phi_{m}\psi_{m}]\|_{L^{\infty}([0,1]^{d})}}_{=:\mathcal{R}_{1}}$
$\displaystyle+$
$\displaystyle\underbrace{\|\sum\nolimits_{m\in\\{1,2\\}^{d}}[\phi_{m}\psi_{m}-\phi_{\times,B_{1}}(\phi_{m}(x),\psi_{m}(x))]\|_{L^{\infty}([0,1]^{d})}}_{=:\mathcal{R}_{2}}$

and

$\displaystyle\|f-\phi\|_{W^{1,\infty}((0,1)^{d})}=$
$\displaystyle\|\sum\nolimits_{m\in\\{1,2\\}^{d}}g_{m}f-\phi\|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle\leq$
$\displaystyle\underbrace{\|\sum\nolimits_{m\in\\{1,2\\}^{d}}[g_{m}f-\phi_{m}\psi_{m}]\|_{W^{1,\infty}((0,1)^{d})}}_{=:\mathcal{R}_{3}}$
$\displaystyle+$
$\displaystyle\underbrace{\|\sum\nolimits_{m\in\\{1,2\\}^{d}}[\phi_{m}\psi_{m}-\phi_{\times,B_{1}}(\phi_{m}(x),\psi_{m}(x))]\|_{W^{1,\infty}((0,1)^{d})}}_{=:\mathcal{R}_{4}}.$

It remains to bound $\mathcal{R}_{1},\mathcal{R}_{2},\mathcal{R}_{3}$, and
$\mathcal{R}_{4}$, respectively. For the term $\mathcal{R}_{1}$, it holds

$\displaystyle\mathcal{R}_{1}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\|g_{m}f-\phi_{m}\psi_{m}\|_{L^{\infty}([0,1]^{d})}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\left[\|(g_{m}-\phi_{m})f\|_{L^{\infty}([0,1]^{d})}+\|\phi_{m}(f-\psi_{m})\|_{L^{\infty}([0,1]^{d})}\right]$
$\displaystyle=\sum_{m\in\\{1,2\\}^{d}}\left[\|(g_{m}-\phi_{m})f\|_{L^{\infty}([0,1]^{d})}+\|\phi_{m}(f-\psi_{m})\|_{L^{\infty}(\Omega_{m})}\right]$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\left[\|g_{m}-\phi_{m}\|_{L^{\infty}([0,1]^{d})}\|f\|_{L^{\infty}([0,1]^{d})}+\|\phi_{m}\|_{L^{\infty}(\Omega_{m})}\|f-\psi_{m}\|_{L^{\infty}(\Omega_{m})}\right]$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\left[\|g_{m}-\phi_{m}\|_{W^{1,\infty}([0,1]^{d})}\|f\|_{W^{1,\infty}([0,1]^{d})}+\|\phi_{m}\|_{L^{\infty}(\Omega_{m})}\|f-\psi_{m}\|_{L^{\infty}(\Omega_{m})}\right]$
$\displaystyle\leq 2^{d}[50d^{5/2}(N+1)^{-4dL}+(1+50d^{5/2})(NL)^{-2/d}]$
$\displaystyle\lesssim(NL)^{-2/d},$

where we use $(NL)^{2/d}\leq(N+1)^{4dL}$ to derive the last inequality and
omit a prefactor in $d$. For the term $\mathcal{R}_{3}$, it holds

$\displaystyle\mathcal{R}_{3}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\|g_{m}f-\phi_{m}\psi_{m}\|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\left[\|(g_{m}-\phi_{m})f\|_{W^{1,\infty}((0,1)^{d})}+\|\phi_{m}(f-\psi_{m})\|_{W^{1,\infty}((0,1)^{d})}\right]$
$\displaystyle=\sum_{m\in\\{1,2\\}^{d}}\left[\|(g_{m}-\phi_{m})f\|_{W^{1,\infty}((0,1)^{d})}+\|\phi_{m}(f-\psi_{m})\|_{W^{1,\infty}(\Omega_{m})}\right]$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\left[\|g_{m}-\phi_{m}\|_{W^{1,\infty}((0,1)^{d})}\|f\|_{W^{1,\infty}((0,1)^{d})}\right.$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}+\left.\|\phi_{m}\|_{W^{1,\infty}(\Omega_{m})}\|f-\psi_{m}\|_{L^{\infty}(\Omega_{m})}+\|\phi_{m}\|_{L^{\infty}(\Omega_{m})}\|f-\psi_{m}\|_{W^{1,\infty}(\Omega_{m})}\right]$
$\displaystyle\lesssim 2^{d}[50d^{5/2}(N+1)^{-4dL}+(4\lfloor
N^{1/d}\rfloor^{2}\lfloor L^{2/d}\rfloor+50d^{5/2})(NL)^{-2/d}+(1+50d^{5/2})]$
$\displaystyle\lesssim 1,$

where we use $(NL)^{2/d}\leq(N+1)^{4dL}$ to derive the last inequality and
omit a prefactor in $d$. For the terms $\mathcal{R}_{2}$ and
$\mathcal{R}_{4}$, it holds

$\displaystyle\mathcal{R}_{2}$ $\displaystyle\leq\mathcal{R}_{4}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\|[\phi_{m}\psi_{m}-\phi_{\times,B_{1}}(\phi_{m}(x),\psi_{m}(x))]\|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}\|[\phi_{m}\psi_{m}-\phi_{\times,B_{1}}(\phi_{m}(x),\psi_{m}(x))]\|_{W^{1,\infty}(\Omega_{m})}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}2\sqrt{d}\max\left\\{\|\phi_{\times,B_{1}}(x,y)-xy\|_{L^{\infty}((-B_{1},B_{1})^{2})},\right.$
$\displaystyle~{}~{}~{}\left.|\phi_{\times,B_{1}}(x,y)-xy|_{W^{1,\infty}((-B_{1},B_{1})^{2})}\times\max\\{|\phi_{m}|_{W^{1,\infty}(\Omega_{m})},|\psi_{m}|_{W^{1,\infty}(\Omega_{m})}\\}\right\\}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}2\sqrt{d}\|\phi_{\times,B_{1}}(x,y)-xy\|_{W^{1,\infty}((-B_{1},B_{1})^{2})}$
$\displaystyle~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}~{}\times\max\\{\|\phi_{m}\|_{W^{1,\infty}(\Omega_{m})},\|\psi_{m}\|_{W^{1,\infty}(\Omega_{m})}\\}$
$\displaystyle\leq\sum_{m\in\\{1,2\\}^{d}}12\sqrt{d}B_{1}^{2}(N+1)^{-8L}B_{2}$
$\displaystyle\lesssim 2^{d}\sqrt{d}d^{5}(N+1)^{-8L}((NL)^{2/d}+d^{5/2})$
$\displaystyle\lesssim 2^{d}\sqrt{d}d^{5}(d^{5/2}(NL)^{2/d})(N+1)^{-8L}$
$\displaystyle\lesssim(NL)^{2/d}(N+1)^{-8L}$
$\displaystyle\lesssim(NL)^{-2/d},$

where we use $(NL)^{2/d}\leq(N+1)^{4L}$ in the last inequality and omit
constants depending only on $d$. Combining the estimates of
$\mathcal{R}_{1},\mathcal{R}_{2},\mathcal{R}_{3}$, and $\mathcal{R}_{4}$, we
have

$\displaystyle\|f-\phi\|_{L^{\infty}([0,1]^{d})}\leq\mathcal{R}_{1}+\mathcal{R}_{2}\lesssim(NL)^{-2/d}$

and

$\displaystyle\|f-\phi\|_{W^{1,\infty}([0,1]^{d})}\leq\mathcal{R}_{3}+\mathcal{R}_{4}\lesssim
1+(NL)^{-2/d}\lesssim 1.$

It is easy to see

$\displaystyle\|\phi\|_{W^{1,\infty}([0,1]^{d})}\leq\|f\|_{W^{1,\infty}([0,1]^{d})}+\|f-\phi\|_{W^{1,\infty}([0,1]^{d})}\lesssim
1.$

Lastly, we calculate the complexity of the constructed deep ReLU network
$\phi$ in (B.3). By the definition of $\phi$ in (B.3), we know that $\phi$
consists of ${\mathcal{O}}(2^{d})$ parallel subnetworks listed as follows:

  * •

$\phi_{\times,B_{1}}$ with width $\mathcal{O}(N)$ and depth $\mathcal{O}(L)$;

  * •

$\phi_{m}$ with width $\mathcal{O}(dN)$ and depth $\mathcal{O}(d^{2}L)$;

  * •

$\psi_{m}$ with width ${\mathcal{O}}(N\log N)$ and depth ${\mathcal{O}}(L\log
L)$.

Hence, the deep ReLU network implementing the function $\phi$ has width
${\mathcal{O}}(2^{d}dN\log N)$ and depth ${\mathcal{O}}(d^{2}L\log L)$. ∎

###### Proof of Corollary B.2.

The proof is completed by employing Lemma B.1 on
$\bar{f}:=f/\|f\|_{W^{1,\infty}((0,1)^{d})}$. ∎

Figure 2: Functions $\phi_{m}(t)$ and $\phi_{m+1}(t)$ for defining a partition
of unity.

###  B.2 Approximation in time with Lipschitz regularity

To handle the singularity of the velocity field in time, we develop a new
approximation result to approximate the velocity field in time.

######  Lemma B.8.

Given any $f\in W^{1,\infty}((0,1))$ with
$\|f\|_{W^{1,\infty}((0,1))}\leq\infty$, for any $M\in\mathbb{N}$, there
exists a function $\xi$ implemented by a deep ReLU network with width
$\mathcal{O}(M)$ and depth $\mathcal{O}(1)$ such that
$|\xi|_{W^{1,\infty}((0,1))}\lesssim|f|_{W^{1,\infty}((0,1))}$ and

$\displaystyle\|\xi-f\|_{L^{\infty}([0,1])}\lesssim|f|_{W^{1,\infty}((0,1))}/M.$

###### Proof.

The proof consists of two steps. We start with the construction of a
continuous piecewise linear function for approximating $1$-Lipschitz
functions, which shall be implemented by a deep ReLU network. After that, we
establish the global Lipschitz continuity of the constructed deep ReLU
network, in addition to the approximation bounds in the $L^{\infty}([0,1])$
norm.

Step 1. We construct a partition of unity following Yarotsky (2017, Proof of
Theorem 1). Let $M\in\mathbb{N}$ and $m\in\\{0,1,\cdots,M\\}$. We collect a
set of functions $\\{\phi_{m}\\}_{m=0}^{M}$ that are defined as follows: for
any $t\in[0,1]$, let

$\displaystyle\phi_{m}(t):=\psi\left(3M\left(t-\frac{m}{M}\right)\right)\quad\text{with}\quad\psi(z)=\begin{cases}1,&|z|<1,\\\
0,&|z|>2,\\\ 2-|z|,&1\leq|z|\leq 2,\end{cases}$ (B.4)

that satisfies $\sum_{m=0}^{M}\phi_{m}(t)=1$. It implies that
$\\{\phi_{m}\\}_{m=0}^{M}$ forms a partition of unity on the domain $[0,1]$.
We plot $\phi_{m}$ and $\phi_{m+1}$ in Figure 2. As in Chen et al. (2020,
Proof of Lemma 10), for each $m\in\\{0,1,\cdots,M\\}$, we consider a piecewise
constant function $f_{m}:=f(m/M)$. Actually, the piecewise constant
approximation is specially the zero-degree Taylor polynomial for the function
$f$ at $x=m/M$ in Yarotsky (2017, Proof of Theorem 1). We claim that

$\displaystyle\tilde{f}(t):=\sum_{m=0}^{M}\phi_{m}(t)f_{m}$ (B.5)

provides an approximation of $f$, and the approximation error is evaluated by

$\displaystyle\|\tilde{f}-f\|_{L^{\infty}([0,1])}$
$\displaystyle=\sup_{t\in[0,1]}\Big{|}\sum\nolimits_{m=0}^{M}\phi_{m}(t)[f_{m}-f(t)]\Big{|}$
$\displaystyle=\sup_{t\in[0,1]}\Big{|}\sum\nolimits_{|t-\frac{m}{M}|\leq\frac{2}{3M}}\phi_{m}(t)[f(m/M)-f(t)]\Big{|}$
$\displaystyle\leq\frac{2}{3M}|f|_{W^{1,\infty}((0,1))},$

where the Lipschitz continuity of $f$ is used in the inequality. It is clear
that $\tilde{f}$ can be implemented with a deep ReLU network.

Step 2. We establish the global Lipschitz continuity of $\tilde{f}$. Notice
that for any $t,s\in[0,1]$,

$\displaystyle|\tilde{f}(t)-\tilde{f}(s)|$
$\displaystyle\leq|\tilde{f}(t)-f(t)|+|f(t)-f(s)|+|f(s)-\tilde{f}(s)|$
$\displaystyle\leq
2\|\tilde{f}-f\|_{L^{\infty}([0,1])}+|f|_{W^{1,\infty}((0,1))}|t-s|$
$\displaystyle\leq\frac{4}{3M}|f|_{W^{1,\infty}((0,1))}+|f|_{W^{1,\infty}((0,1))}|t-s|.$

(1) If $|t-s|\geq\frac{1}{3M}$, it is clear that
$|\tilde{f}(t)-\tilde{f}(s)|\leq 5|f|_{W^{1,\infty}((0,1))}|t-s|$.
(2) If $|t-s|<\frac{1}{3M}$, we try to directly bound the difference

$\displaystyle|\tilde{f}(t)-\tilde{f}(s)|$
$\displaystyle=\Big{|}\sum\nolimits_{m=0}^{M}[\phi_{m}(t)-\phi_{m}(s)]f_{m}\Big{|}$
$\displaystyle=\Big{|}\sum\nolimits_{m=0}^{M}\left[\psi\left(3Mt-3m\right)-\psi\left(3Ms-3m\right)\right]f_{m}\Big{|}=:\mathcal{E}.$

Next, we focus on bounding $\mathcal{E}$. Without loss of generality, we
assume $s>t$. Considering $|t-s|<\frac{1}{3M}$, we deduce that
$s\in(t,t+\frac{1}{3M})$. From Figure 2, we can observe that there exist at
most two numbers $m=\tilde{m}\in\\{0,1,\cdots,M\\}$ or
$m=\bar{m}:=\tilde{m}+1$ such that $\psi(3Mt-3m)\not\equiv 0$ or
$\psi(3Ms-3m)\not\equiv 0$. It follows that

$\displaystyle\mathcal{E}$
$\displaystyle=\left|\left[\psi\left(3Mt-3\tilde{m}\right)-\psi\left(3Ms-3\tilde{m}\right)\right]f_{\tilde{m}}\right.$
$\displaystyle~{}~{}~{}~{}+\left.\left[\psi\left(3Mt-3\bar{m}\right)-\psi\left(3Ms-3\bar{m}\right)\right]f_{\bar{m}}\right|$
$\displaystyle=\left|\left[\psi\left(3Mt-3\tilde{m}\right)-\psi\left(3Ms-3\tilde{m}\right)\right]f_{\tilde{m}}\right.$
$\displaystyle~{}~{}~{}~{}+\left.\left[(1-\psi\left(3Mt-3\tilde{m}\right))-(1-\psi\left(3Ms-3\tilde{m}\right))\right]f_{\bar{m}}\right|$
$\displaystyle=\left|\left[\psi\left(3Mt-3\tilde{m}\right)-\psi\left(3Ms-3\tilde{m}\right)\right](f_{\tilde{m}}-f_{\bar{m}})\right|$
$\displaystyle\leq\left|f_{\tilde{m}}-f_{\bar{m}}\right|\cdot\left|\psi\left(3Mt-3\tilde{m}\right)-\psi\left(3Ms-3\tilde{m}\right)\right|$
$\displaystyle\leq\frac{1}{M}|f|_{W^{1,\infty}((0,1))}\left|\psi\left(3Mt-3\tilde{m}\right)-\psi\left(3Ms-3\tilde{m}\right)\right|$
$\displaystyle=3|f|_{W^{1,\infty}((0,1))}|t-s|.$

Hence, if $|t-s|<\frac{1}{3M}$, it holds that $|\tilde{f}(t)-\tilde{f}(s)|\leq
3|f|_{W^{1,\infty}((0,1))}|t-s|$.

To sum up, for any $t,s\in[0,1]$, it holds that
$|\tilde{f}(t)-\tilde{f}(s)|\leq 5|f|_{W^{1,\infty}((0,1))}|t-s|$. It is easy
to see from (B.5) that the deep ReLU network implementing $\tilde{f}$ has
width $\mathcal{O}(M)$ and depth $\mathcal{O}(1)$. Then we complete the proof.
∎

###  B.3 Time-space approximation

In the subsection, we construct a time-space approximation while keeping the
Lipschitz regularity both in the space variable and in the time variable.

Figure 3: The clipping function $\beta_{A}$.

######  Lemma B.9 (Clipping functions).

Given $A>0$, we define $\beta_{A}:{\mathbb{R}}\to[-A,A]$ by

$\displaystyle\beta_{A}(z):=\begin{cases}-A,&z\in(-\infty,-A),\\\
z,&z\in[-A,A],\\\ A,&z\in(A,\infty).\end{cases}$

There exists a clipping function
$\mathcal{C}_{A}:{\mathbb{R}}^{d}\to[-A,A]^{d}$ at level $A$ implemented by a
deep ReLU network with width $\mathcal{O}(d)$ and depth $\mathcal{O}(1)$ such
that for any $x=[x_{1},x_{2},\cdots,x_{d}]^{\top}\in{\mathbb{R}}^{d}$,

$\displaystyle\mathcal{C}_{A}(x)=[\beta_{A}(x_{1}),\beta_{A}(x_{2}),\cdots,\beta_{A}(x_{d})]^{\top}.$

###### Proof.

It is clear that
$\mathcal{C}_{A}(x)=\varrho(x+A\mathbf{1}_{d})-\varrho(x-A\mathbf{1}_{d})-A\mathbf{1}_{d}$
where $\varrho:{\mathbb{R}}^{d}\to{\mathbb{R}}^{d}$ is the ReLU function. This
expression implies that the clipping function $\mathcal{C}_{A}$ can be
implemented by a deep ReLU network with width $\mathcal{O}(d)$ and depth
$\mathcal{O}(1)$. ∎

The main idea of the time-space approximation on
$[0,1-\underline{t}]\times{\mathbb{R}}^{d}$ is based on Lemmas B.1, B.8, and
B.9.

###### Proof of Theorem 5.4.

We derive a time-space approximation $\bar{v}$ of the velocity field $v^{*}$
on the domain $\Omega_{\underline{t},A}=[0,1-\underline{t}]\times[-A,A]^{d}$
and bound the Lipschitz constants of $\bar{v}$ on the domain
$[0,1-\underline{t}]\times{\mathbb{R}}^{d}$.

First of all, we use the clipping function $\mathcal{C}$ defined in Lemma B.9
to clip the support of the space variable, that is, for each
$x\in{\mathbb{R}}^{d}$, we have $\mathcal{C}(x)\in[-A,A]^{d}$. We only need to
consider approximation in $x$ on the domain $[-A,A]^{d}$.

Then we can employ the mappings
$\tilde{t}=\mathcal{T}_{1}(t):=t/(1-\underline{t})$ and
$\tilde{x}=\mathcal{T}_{2}(x):=(x+A\mathbf{1}_{d})/(2A)$ to transform the
domain $\Omega_{\underline{t},A}$ into the domain $[0,1]^{d+1}$. When the
domain $[0,1-\underline{t}]\times{\mathbb{R}}^{d}$ is considered, the
transformed domain is $[0,1]\times{\mathbb{R}}^{d}$. Notice that both mappings
are invertible and can be implemented by deep ReLU networks. We denote their
inverse functions as $t=\mathcal{T}_{1}^{-1}(\tilde{t})$ and
$x=\mathcal{T}_{2}^{-1}(\tilde{x})$. We further define a new velocity field
$v^{\diamond}$ by
$v^{\diamond}(\tilde{t},\tilde{x}):=v^{*}(\mathcal{T}_{1}^{-1}(\tilde{t}),\mathcal{T}_{2}^{-1}(\tilde{x}))$
for any $(\tilde{t},\tilde{x})\in[0,1]\times{\mathbb{R}}^{d}$. It is clear
that $v^{*}(t,x)=v^{\diamond}(\mathcal{T}_{1}(t),\mathcal{T}_{2}(x))$ for any
$(t,x)\in[0,1-\underline{t}]\times{\mathbb{R}}^{d}$. According to Theorem 5.1,
the new velocity field $v^{\diamond}$ satisfies

  1. (1)

For any $\tilde{s},\tilde{t}\in[0,1]$ and $\tilde{x}\in{\mathbb{R}}^{d}$,
$\|v^{\diamond}(\tilde{t},\tilde{x})-v^{\diamond}(\tilde{s},\tilde{x})\|_{\infty}\lesssim\underline{t}^{-2}|\tilde{t}-\tilde{s}|$;

  2. (2)

For any $\tilde{x},\tilde{y}\in{\mathbb{R}}^{d}$ and $\tilde{t}\in[0,1]$,
$\|v^{\diamond}(\tilde{t},\tilde{x})-v^{\diamond}(\tilde{t},\tilde{y})\|_{\infty}\lesssim
A\|\tilde{x}-\tilde{y}\|_{\infty}$;

  3. (3)

$\|v^{\diamond}\|_{L^{\infty}([0,1]^{d+1})}\lesssim A$,

where we omit constants in $d,\kappa,\beta,\sigma,D,R$.

In the following, we construct a time-space approximation of the new velocity
field $v^{\diamond}$ on the transformed domain $[0,1]^{d+1}$ using deep ReLU
networks. Let
$v^{\diamond}=[v^{\diamond}_{1},v^{\diamond}_{2},\cdots,v^{\diamond}_{d}]^{\top}$.
Given $M\in\mathbb{N}$, we uniformly partition the unit interval $[0,1]$ into
$M$ non-overlapping sub-intervals with length $1/M$. Let
$\\{\phi_{j}(\tilde{t})\\}_{j=0}^{M}$ form a partition of unity on $[0,1]$
with the same definition as (B.4) in the proof of Lemma B.8. For each
$i\in\\{1,2,\cdots,d\\}$, we define a time approximation of $v^{\diamond}_{i}$
by

$\displaystyle\tilde{v}_{i}(\tilde{t},\tilde{x}):=\sum\nolimits_{j=0}^{M}v^{\diamond}_{i}\left(j/M,\tilde{x}\right)\phi_{j}\left(\tilde{t}\right).$

Let $\tilde{v}:=[\tilde{v}_{1},\tilde{v}_{2},\cdots,\tilde{v}_{d}]^{\top}$.
Due to Lemma B.8, for any $\tilde{x}\in[0,1]^{d}$, it holds that

$\displaystyle|\tilde{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}\lesssim|v^{\diamond}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}\lesssim\underline{t}^{-2}.$

For $i=1,2,\cdots,d$ and $j=0,1,\cdots,M$, let $\zeta_{ij}(\tilde{x})$ be a
space approximation of $v^{\diamond}_{i}\left(j/M,\tilde{x}\right)$
implemented by a deep ReLU network constructed in Lemma B.1. Then it holds
that $\max_{i,j}\|\zeta_{ij}\|_{W^{1,\infty}((0,1)^{d})}\lesssim A$. By Lemma
F.4, we can construct a deep ReLU network $\phi_{\times,(B_{3},B_{4})}$ with
width $15(N+1)$ and depth $8L$ to approximate the product function such that
$\|\phi_{\times,(B_{3},B_{4})}\|_{W^{1,\infty}((-B_{3},B_{3})\times(-B_{4},B_{4}))}\leq
12B_{3}B_{4}$,

$\displaystyle\|\phi_{\times,(B_{3},B_{4})}(x,y)-xy\|_{W^{1,\infty}((-B_{3},B_{3})\times(-B_{4},B_{4}))}\leq
6B_{3}B_{4}(N+1)^{-4L},$ (B.6)

and

$\displaystyle\phi_{\times,(B_{3},B_{4})}(x,0)=\frac{\partial\phi_{\times,(B_{3},B_{4})}(x,0)}{\partial
x}=0\text{ for $x\in(-B_{3},B_{3})$}.$ (B.7)

Using the same partition of unity $\\{\phi_{j}(\tilde{t})\\}_{j=0}^{M}$ on
$[0,1]$, we define a time-space approximation of $v^{\diamond}_{i}$ for each
$i\in\\{1,2,\cdots,d\\}$ by

$\displaystyle
v^{\natural}_{i}(\tilde{t},\tilde{x}):=\sum\nolimits_{j=0}^{M}\phi_{\times,(B_{3},B_{4})}\left(\zeta_{ij}(\tilde{x}),\phi_{j}\left(\tilde{t}\right)\right),$
(B.8)

which can be implemented with a deep ReLU network. We choose the parameters
$B_{3},B_{4}$ such that
$B_{3}\asymp\max_{i,j}\|\zeta_{ij}\|_{L^{\infty}([0,1]^{d})}\lesssim A$ and
$B_{4}\asymp\max_{j}\|\phi_{j}\|_{L^{\infty}([0,1])}\lesssim 1$.

######  Claim B.10.

There are at most two nonzero terms in the summation (B.8) defining the time-
space approximation function $v^{\natural}_{i}$.

This claim holds because for any $\tilde{t}\in[0,1]$, there are at most two
indexes $j$’s from $\\{0,1,2,\cdots,M\\}$ such that $\phi_{j}(\tilde{t})$ is
nonzero according to the definition of the partition of unity
$\\{\phi_{j}(\tilde{t})\\}_{j=0}^{M}$. Then our claim follows from the
property (B.7) of the approximation product function
$\phi_{\times,(B_{3},B_{4})}$.

Before we study the properties of $v^{\natural}_{i}$, we introduce a surrogate
function $\check{v}_{i}$ defined by

$\displaystyle\check{v}_{i}(\tilde{t},\tilde{x}):=\sum\nolimits_{j=0}^{M}\zeta_{ij}(\tilde{x})\phi_{j}\left(\tilde{t}\right).$

The function $\check{v}_{i}$ will be useful to study the approximation
capacity and the regularity of $v^{\natural}_{i}$. We derive the approximation
rate and the regularity properties of $\check{v}_{i}$ in the following. Due to
Lemma B.1, for any $\tilde{x}\in[0,1]^{d}$, $i=1,2,\cdots,d$, and
$j=0,1,\cdots,M$, we have

$\displaystyle|\zeta_{ij}(\tilde{x})-v^{\diamond}_{i}\left(j/M,\tilde{x}\right)|\lesssim
A(NL)^{-2/d}.$

We evaluate the approximation error of $\check{v}_{i}$ by the following error
decomposition:

$\displaystyle\|\check{v}_{i}-v^{\diamond}_{i}\|_{L^{\infty}([0,1]^{d+1})}\leq\underbrace{\|\check{v}_{i}-\tilde{v}_{i}\|_{L^{\infty}([0,1]^{d+1})}}_{=:\mathcal{E}_{i}^{1}}+\underbrace{\|\tilde{v}_{i}-v^{\diamond}_{i}\|_{L^{\infty}([0,1]^{d+1})}}_{=:\mathcal{E}_{i}^{2}}.$
(B.9)

By Lemma B.1, we bound $\mathcal{E}_{i}^{1}$ by

$\displaystyle\mathcal{E}_{i}^{1}$
$\displaystyle\leq\Big{\|}\sum\nolimits_{j=0}^{M}\left[\zeta_{ij}(\tilde{x})-v^{\diamond}_{i}\left(j/M,\tilde{x}\right)\right]\phi_{j}\left(\tilde{t}\right)\Big{\|}_{L^{\infty}([0,1]^{d+1})}$
$\displaystyle\leq\max\nolimits_{0\leq j\leq
M}\|\zeta_{ij}(\tilde{x})-v^{\diamond}_{i}\left(j/M,\tilde{x}\right)\|_{L^{\infty}([0,1]^{d})}$
$\displaystyle\lesssim\max\nolimits_{0\leq j\leq
M}\|v^{\diamond}_{i}(j/M,\tilde{x})\|_{W^{1,\infty}((0,1)^{d})}(NL)^{-2/d}$
$\displaystyle\lesssim A(NL)^{-2/d}.$ (B.10)

By Lemma B.8, we bound $\mathcal{E}_{i}^{2}$ by

$\displaystyle\mathcal{E}_{i}^{2}\lesssim\sup_{\tilde{x}\in[0,1]^{d}}|v^{\diamond}_{i}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1))}/M\
\lesssim\underline{t}^{-2}M^{-1}.$ (B.11)

Combining (B.9), (B.10), and (B.11), we have

$\displaystyle\|\check{v}_{i}-v^{\diamond}_{i}\|_{L^{\infty}([0,1]^{d+1})}\lesssim
A(NL)^{-2/d}+\underline{t}^{-2}M^{-1}.$

Suppose that $(NL)^{2/d}\asymp\underline{t}^{2}M$, and it yields that for
$i=1,2,\cdots,d$,

$\displaystyle\|\check{v}_{i}-v^{\diamond}_{i}\|_{L^{\infty}([0,1]^{d+1})}\lesssim
A(NL)^{-2/d}.$

Let $\check{v}:=[\check{v}_{1},\check{v}_{2},\cdots,\check{v}_{d}]^{\top}$. We
have the approximation power of $\check{v}$ evaluated by

$\displaystyle\|\check{v}-v^{\diamond}\|_{L^{\infty}([0,1]^{d+1})}\lesssim
A(NL)^{-2/d}.$ (B.12)

Moreover, the Lipschitz continuity of $\check{v}$ in $\tilde{t}$ and
$\tilde{x}$ can be verified. Concretely, we have the Lipschitz estimate in the
space variable $\tilde{x}$: for any $\tilde{x},\tilde{y}\in[0,1]^{d}$ and
$\tilde{t}\in[0,1]$,

$\displaystyle\|\check{v}(\tilde{t},\tilde{x})-\check{v}(\tilde{t},\tilde{y})\|_{\infty}$
$\displaystyle\leq\max\nolimits_{1\leq i\leq
d}\Big{\|}\sum\nolimits_{j=0}^{M}[\zeta_{ij}(\tilde{x})-\zeta_{ij}(\tilde{y})]\phi_{j}\left(\tilde{t}\right)\Big{\|}_{\infty}$
$\displaystyle\leq\max\nolimits_{1\leq i\leq d,\ 0\leq j\leq
M}\|\zeta_{ij}(\tilde{x})-\zeta_{ij}(\tilde{y})\|_{\infty}$
$\displaystyle\leq\max\nolimits_{1\leq i\leq d,\ 0\leq j\leq
M}\|\zeta_{ij}\|_{W^{1,\infty}((0,1)^{d})}\cdot\|\tilde{x}-\tilde{y}\|_{\infty}$
$\displaystyle\lesssim\|v^{\diamond}\|_{W^{1,\infty}((0,1)^{d};{\mathbb{R}}^{d})}\cdot\|\tilde{x}-\tilde{y}\|_{\infty}$
$\displaystyle\lesssim A\|\tilde{x}-\tilde{y}\|_{\infty},$

It is somewhat tedious to derive the Lipschitz estimate in the time variable
$\tilde{t}$. For any $\tilde{s},\tilde{t}\in[0,1]$ and
$\tilde{x}\in[0,1]^{d}$,

$\displaystyle~{}~{}~{}~{}~{}\|\check{v}(\tilde{t},\tilde{x})-\check{v}(\tilde{s},\tilde{x})\|_{\infty}$
$\displaystyle\leq\|\check{v}(\tilde{t},\tilde{x})-\tilde{v}(\tilde{t},\tilde{x})\|_{\infty}+\|\tilde{v}(\tilde{t},\tilde{x})-\tilde{v}(\tilde{s},\tilde{x})\|_{\infty}+\|\tilde{v}(\tilde{s},\tilde{x})-\check{v}(\tilde{s},\tilde{x})\|_{\infty}$
$\displaystyle\leq
2\sup_{\vartheta\in[0,1]}\|\check{v}(\vartheta,\tilde{x})-\tilde{v}(\vartheta,\tilde{x})\|_{\infty}+|\tilde{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}|\tilde{t}-\tilde{s}|$
$\displaystyle\leq 2\max\nolimits_{1\leq i\leq
d}\mathcal{E}_{i}^{1}+|\tilde{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}|\tilde{t}-\tilde{s}|$
$\displaystyle\lesssim A(NL)^{-2/d}+\underline{t}^{-2}|\tilde{t}-\tilde{s}|.$

Considering $(NL)^{2/d}\asymp\underline{t}^{2}M$, we deduce that

$\displaystyle\|\check{v}(\tilde{t},\tilde{x})-\check{v}(\tilde{s},\tilde{x})\|_{\infty}\lesssim
A\underline{t}^{-2}M^{-1}+\underline{t}^{-2}|\tilde{t}-\tilde{s}|.$

Then we consider two cases for bounding
$\|\check{v}(\tilde{t},\tilde{x})-\check{v}(\tilde{s},\tilde{x})\|_{\infty}$.
Case 1. If $|\tilde{t}-\tilde{s}|\geq\frac{1}{3M}$, it is clear that
$\|\check{v}(\tilde{t},\tilde{x})-\check{v}(\tilde{s},\tilde{x})\|_{\infty}\lesssim
A\underline{t}^{-2}|\tilde{t}-\tilde{s}|$.
Case 2. If $|\tilde{t}-\tilde{s}|<\frac{1}{3M}$, for any
$i\in\\{1,2,\cdots,d\\}$, we try to bound the difference

$\displaystyle~{}~{}~{}~{}~{}|\check{v}_{i}(\tilde{t},\tilde{x})-\check{v}_{i}(\tilde{s},\tilde{x})|$
$\displaystyle=\Big{|}\sum\nolimits_{j=0}^{M}\zeta_{ij}(\tilde{x})\left[\phi_{j}\left(\tilde{t}\right)-\phi_{j}\left(\tilde{s}\right)\right]\Big{|}$
$\displaystyle=\Big{|}\sum\nolimits_{j=0}^{M}\zeta_{ij}(\tilde{x})\left[\psi\left(3M\tilde{t}-3j\right)-\psi\left(3M\tilde{s}-3j\right)\right]\Big{|}=:\mathcal{E}_{3}.$

Then, we focus on bounding $\mathcal{E}_{3}$. Without loss of generality, we
assume $\tilde{t}<\tilde{s}$. The remaining calculation is similar to the
proof of Lemma B.8. Let $m=\tilde{m}\in\\{0,1,\cdots,M\\}$ or
$m=\bar{m}:=\tilde{m}+1$ be two possible numbers satisfying
$\psi(3M\tilde{t}-3m)\not\equiv 0$ or $\psi(3M\tilde{s}-3m)\not\equiv 0$. Then
it holds that

$\displaystyle\mathcal{E}_{3}$
$\displaystyle=\left|\left[\psi\left(3M\tilde{t}-3\tilde{m}\right)-\psi\left(3M\tilde{s}-3\tilde{m}\right)\right]\zeta_{i\tilde{m}}(\tilde{x})\right.$
$\displaystyle~{}~{}~{}~{}+\left.\left[\psi\left(3M\tilde{t}-3\bar{m}\right)-\psi\left(3M\tilde{s}-3\bar{m}\right)\right]\zeta_{i\bar{m}}(\tilde{x})\right|$
$\displaystyle=\left|\left[\psi\left(3M\tilde{t}-3\tilde{m}\right)-\psi\left(3M\tilde{s}-3\tilde{m}\right)\right]\zeta_{i\tilde{m}}(\tilde{x})\right.$
$\displaystyle~{}~{}~{}~{}+\left.\left[(1-\psi\left(3M\tilde{t}-3\tilde{m}\right))-(1-\psi\left(3M\tilde{s}-3\tilde{m}\right))\right]\zeta_{i\bar{m}}(\tilde{x})\right|$
$\displaystyle=\left|\left[\psi\left(3M\tilde{t}-3\tilde{m}\right)-\psi\left(3M\tilde{s}-3\tilde{m}\right)\right][\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})]\right|$
$\displaystyle\leq\left|\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})\right|\left|\psi\left(3M\tilde{t}-3\tilde{m}\right)-\psi\left(3M\tilde{s}-3\tilde{m}\right)\right|$
$\displaystyle\leq
3M\left|\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})\right|\cdot|\tilde{t}-\tilde{s}|.$

We bound the term
$|\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})|$ by

$\displaystyle~{}~{}~{}~{}~{}|\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})|$
$\displaystyle\leq|\zeta_{i\tilde{m}}(\tilde{x})-v^{\diamond}_{i}\left(\tilde{m}/M,\tilde{x}\right)|+|v^{\diamond}_{i}\left(\tilde{m}/M,\tilde{x}\right)-v^{\diamond}_{i}\left(\bar{m}/M,\tilde{x}\right)|$
$\displaystyle~{}~{}~{}~{}+|v^{\diamond}_{i}\left(\bar{m}/M,\tilde{x}\right)-\zeta_{i\bar{m}}(\tilde{x})|$
$\displaystyle\lesssim A(NL)^{-2/d}+\underline{t}^{-2}M^{-1}.$

Recall that $(NL)^{2/d}\asymp\underline{t}^{2}M$. It implies that
$|\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})|\lesssim
A\underline{t}^{-2}M^{-1}.$ Therefore, if
$|\tilde{t}-\tilde{s}|<\frac{1}{3M}$, it holds that

$\displaystyle|\check{v}_{i}(\tilde{t},\tilde{x})-\check{v}_{i}(\tilde{s},\tilde{x})|\leq
3M\left|\zeta_{i\tilde{m}}(\tilde{x})-\zeta_{i\bar{m}}(\tilde{x})\right||\tilde{t}-\tilde{s}|\lesssim
A\underline{t}^{-2}|\tilde{t}-\tilde{s}|.$

We summarize the Lipschitz properties of $\check{v}$ as follows:

$\displaystyle|\check{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}\lesssim
A\underline{t}^{-2},\quad|\check{v}(\tilde{t},\cdot)|_{W^{1,\infty}((0,1)^{d};{\mathbb{R}}^{d})}\lesssim
A.$

Let
$v^{\natural}:=[v^{\natural}_{1},v^{\natural}_{2},\cdots,v^{\natural}_{d}]^{\top}$.
We use the approximation rate of $\check{v}$ to derive that of $v^{\natural}$.
By the triangle inequality, it holds that

$\displaystyle\|v^{\natural}-v^{\diamond}\|_{L^{\infty}([0,1]^{d+1})}$
$\displaystyle\leq$
$\displaystyle\|v^{\natural}-\check{v}\|_{L^{\infty}([0,1]^{d+1})}+\|\check{v}-v^{\diamond}\|_{L^{\infty}([0,1]^{d+1})}$
$\displaystyle\lesssim$ $\displaystyle A(N+1)^{-4L}+A(NL)^{-2/d}\quad\text{(By
Claim \ref{claim:nonzero-terms}, Eq. (\ref{eq:approx-rate-times-nn}), and Eq.
(\ref{eq:vf-surrogate-cube-approx}))}$ $\displaystyle\lesssim$ $\displaystyle
A(NL)^{-2/d}\quad\text{(By $(NL)^{2/d}\leq(N+1)^{4L}$ for any
$N,L\in{\mathbb{N}}$)}.$

Thus, the approximation rate of $v^{\natural}$ is given by

$\displaystyle\|v^{\natural}-v^{\diamond}\|_{L^{\infty}([0,1]^{d+1})}\lesssim
A(NL)^{-2/d}.$ (B.13)

Then we study the Lipschitz properties of $v^{\natural}$. By Lemma F.1 and
Claim B.10, it holds that

$\displaystyle~{}|v^{\natural}(\cdot,\tilde{x})-\check{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}$
$\displaystyle=$
$\displaystyle\max_{i}|v_{i}^{\natural}(\cdot,\tilde{x})-\check{v}_{i}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1))}$
$\displaystyle=$
$\displaystyle\max_{i}\Big{|}\sum\nolimits_{j=0}^{M}\phi_{\times,(B_{3},B_{4})}\left(\zeta_{ij}(\tilde{x}),\phi_{j}(\cdot)\right)-\sum\nolimits_{j=0}^{M}\zeta_{ij}(\tilde{x})\phi_{j}(\cdot)\Big{|}_{W^{1,\infty}((0,1))}$
$\displaystyle\lesssim$
$\displaystyle\max_{i}|\phi_{\times,(B_{3},B_{4})}(x,y)-xy|_{W^{1,\infty}((-B_{3},B_{3})\times(-B_{4},B_{4}))}\cdot|\phi_{j}|_{W^{1,\infty}((0,1))}$
$\displaystyle\lesssim$ $\displaystyle AM(N+1)^{-4L}\lesssim
A\underline{t}^{-2}(NL)^{2/d}(N+1)^{-4L}\quad\text{(By
$(NL)^{2/d}\asymp\underline{t}^{2}M$)}$ $\displaystyle\lesssim$ $\displaystyle
A\underline{t}^{-2}\quad\text{(By $(NL)^{2/d}\leq(N+1)^{4L}$ for any
$N,L\in{\mathbb{N}}$)}.$

By the triangle inequality, the Lipschitz property of $v^{\natural}$ in the
time variable $\tilde{t}$ is evaluated by

$\displaystyle|v^{\natural}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}$
$\displaystyle\leq$
$\displaystyle|v^{\natural}(\cdot,\tilde{x})-\check{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}+|\check{v}(\cdot,\tilde{x})|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}$
$\displaystyle\lesssim$ $\displaystyle
A\underline{t}^{-2}+A\underline{t}^{-2}\lesssim A\underline{t}^{-2}.$

By Lemma F.1 and Claim B.10, we derive the Lipschitz property of
$v^{\natural}$ in the space variable $\tilde{x}$ as follows

$\displaystyle~{}|v^{\natural}(\tilde{t},\cdot)|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle=$
$\displaystyle\max_{i}|v_{i}^{\natural}(\tilde{t},\cdot)|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle=$
$\displaystyle\max_{i}\Big{|}\sum\nolimits_{j=0}^{M}\phi_{\times,(B_{3},B_{4})}\left(\zeta_{ij}(\cdot),\phi_{j}(\tilde{t})\right)\Big{|}_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle\lesssim$
$\displaystyle\max_{i}\Big{\\{}|\phi_{\times,(B_{3},B_{4})}(\cdot,y)|_{W^{1,\infty}((-B_{3},B_{3}))}\cdot\max_{j}|\zeta_{ij}|_{W^{1,\infty}((0,1)^{d})}\Big{\\}}$
$\displaystyle\lesssim$ $\displaystyle A^{2}.$

We claim that
$\bar{v}(t,x):=v^{\natural}(\mathcal{T}_{1}(t),\mathcal{T}_{2}\circ\mathcal{C}_{A}(x))$
provides a good approximation of the velocity field $v^{*}$ on the domain
$\Omega_{\underline{t},A}$, and $\bar{v}$ can be implemented by a deep ReLU
network. According to the error bound (B.13), the approximation rate of
$\bar{v}$ is given by

$\displaystyle\|\bar{v}(t,x)-v^{*}(t,x)\|_{L^{\infty}(\Omega_{\underline{t},A})}\lesssim
A(NL)^{-2/d},$

where we omit constants in $d,\kappa,\beta,\sigma,D,R$. Furthermore, we need
to estimate the Lipschitz constants of $\bar{v}$. Here, we use Lemma F.1 to
calculate the Sobolev semi-norms of the composite functions:

$\displaystyle|\bar{v}(\cdot,x)|_{W^{1,\infty}((0,1-\underline{t});{\mathbb{R}}^{d})}\lesssim|v^{\natural}(\cdot,\mathcal{T}_{2}\circ\mathcal{C}_{A}(x))|_{W^{1,\infty}((0,1);{\mathbb{R}}^{d})}|\mathcal{T}_{1}|_{W^{1,\infty}((0,1-\underline{t});(0,1))}\lesssim
A\underline{t}^{-2},$
$\displaystyle|\bar{v}(t,\cdot)|_{W^{1,\infty}({\mathbb{R}}^{d};{\mathbb{R}}^{d})}\lesssim|v^{\natural}(\mathcal{T}_{1}(t),\cdot)|_{W^{1,\infty}((0,1)^{d};{\mathbb{R}}^{d})}|\mathcal{T}_{2}|_{W^{1,\infty}((-A,A)^{d};{\mathbb{R}}^{d})}|\mathcal{C}_{A}|_{W^{1,\infty}({\mathbb{R}}^{d};[-A,A]^{d})}\lesssim
A.$

In addition, we have the $L^{\infty}$ bound
$\|\bar{v}\|_{L^{\infty}([1-\underline{t}]\times{\mathbb{R}}^{d})}\lesssim A$.

In the end, it remains to calculate the complexity of the deep ReLU network
implementing $\bar{v}$. By the definition of $v^{\natural}$ in (B.8), we know
that $v^{\natural}$ consists of ${\mathcal{O}}(\underline{t}^{-2}d(NL)^{2/d})$
parallel subnetworks listed as follows:

  * •

$\phi_{\times,(B_{3},B_{4})}$ with width $\mathcal{O}(N)$ and depth
$\mathcal{O}(L)$;

  * •

$\zeta_{ij}$ with width ${\mathcal{O}}(2^{d}dN\log N)$ and depth
${\mathcal{O}}(d^{2}L\log L)$;

  * •

$\phi_{j}$ with width ${\mathcal{O}}(1)$ and depth ${\mathcal{O}}(1)$.

Hence, the deep ReLU network implementing $v^{\natural}$ has width
${\mathcal{O}}(\underline{t}^{-2}2^{d}d^{2}(NL)^{2/d}N\log N)$ and depth
${\mathcal{O}}(d^{2}L\log L)$. By omitting polynomial prefactors in $d$, we
obtain that the deep ReLU network implementing the function $\bar{v}$ has
width ${\mathcal{O}}(\underline{t}^{-2}2^{d}(NL)^{2/d}N\log N)$, depth
${\mathcal{O}}(L\log L)$, and size
${\mathcal{O}}(\underline{t}^{-2}4^{d}(NL)^{2/d}(N\log N)^{2}L\log L)$. ∎

##  Appendix C Error analysis of flow matching

In the appendix, we present proofs for error analyses of flow matching.

###  C.1 Basic error decomposition

We present the proof of Lemma 5.2.

###### Proof of Lemma 5.2.

We follow the proof of Jiao et al. (2023, Lemma 3.1). Due to that $v^{*}$ is
the minimizer of $\mathcal{L}$, direct calculation implies

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|\hat{v}_{n}(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}=\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}(\hat{v}_{n})-\mathcal{L}(v^{*})].$

Since $\hat{v}_{n}$ is the minimizer of the empirical risk, for any
$v^{{\dagger}}\in\arg\inf_{v\in{\mathcal{F}}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|v(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}$,
it holds that

$\displaystyle\mathcal{L}_{n}(\hat{v}_{n})-\mathcal{L}_{n}(v^{*})\leq\mathcal{L}_{n}(v^{{\dagger}})-\mathcal{L}_{n}(v^{*}).$

Taking expectations over $\mathbb{D}_{n}$ on both sides, it yields that

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}_{n}(\hat{v}_{n})-\mathcal{L}(v^{*})]\leq\mathcal{L}(v^{{\dagger}})-\mathcal{L}(v^{*})=\inf_{v\in{\mathcal{F}}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|v(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}.$
(C.1)

Using the inequality (C.1), we deduce that

$\displaystyle~{}\mathbb{E}_{\mathbb{D}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|\hat{v}_{n}(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}=\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}(\hat{v}_{n})-\mathcal{L}(v^{*})]$
$\displaystyle\leq$
$\displaystyle~{}\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}(\hat{v}_{n})-\mathcal{L}(v^{*})]-2\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}_{n}(\hat{v}_{n})-\mathcal{L}(v^{*})]+2\inf_{v\in{\mathcal{F}}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|v(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}$
$\displaystyle\leq$
$\displaystyle~{}\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}(v^{*})-2\mathcal{L}_{n}(\hat{v}_{n})+\mathcal{L}(\hat{v}_{n})]+2\inf_{v\in{\mathcal{F}}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|v(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}.$

This completes the proof. ∎

###  C.2 Truncation error

The truncation error is well controlled by the fast-decaying tail probability
of $\mathsf{X}_{t}\sim p_{t}$. We bound the tail probability in Lemma 5.7 and
the truncation error in Lemma 5.8. For a sub-Gaussian random variable
$\mathsf{X}$, we use $\|\mathsf{X}\|_{\psi_{2}}$ to denote its sub-Gaussian
norm.

###### Proof of Lemma 5.7.

Let
$\mathsf{X}_{t}=[\mathsf{X}_{t}^{1},\mathsf{X}_{t}^{2},\cdots,\mathsf{X}_{t}^{d}]^{\top}$.
Similarly, let
$\mathsf{Z}=[\mathsf{Z}^{1},\mathsf{Z}^{2},\cdots,\mathsf{Z}^{d}]^{\top}$ and
$\mathsf{X}_{1}=[\mathsf{X}_{1}^{1},\mathsf{X}_{1}^{2},\cdots,\mathsf{X}_{1}^{d}]^{\top}$.
By the general Hoeffding inequality (Vershynin, 2018, Theorem 2.6.3), for any
$1\leq i\leq d$, we bound the tail probability of $\mathsf{X}_{t}^{i}$ by

$\displaystyle\mathbb{P}(|\mathsf{X}_{t}^{i}|>A)=\mathbb{P}(|(1-t)\mathsf{Z}^{i}+t\mathsf{X}_{1}^{i}|>A)\leq
2\exp\left(-\frac{C_{1}A^{2}}{K_{1}^{2}}\right),$

where $C_{1}$ is a universal constant and
$K_{1}:=\|\mathsf{Z}^{1}\|_{\psi_{2}}\vee\max_{1\leq i\leq
d}\|\mathsf{X}_{1}^{i}\|_{\psi_{2}}$ with $\mathsf{Z}^{1}\sim\gamma_{1}$.
According to Remark 5, $K_{1}\asymp\sqrt{C_{\mathrm{LSI}}}$ is finite with
dependence on parameters in $\mathcal{S}_{1}$. By the union bound, it further
yields

$\displaystyle\mathbb{P}(\mathsf{X}_{t}\in\Omega_{A}^{c})=\mathbb{P}(\exists
1\leq i\leq
d:|\mathsf{X}_{t}^{i}|>A)\leq\sum_{i=1}^{d}\mathbb{P}(|\mathsf{X}_{t}^{i}|>A)\leq
2d\exp\left(-\frac{C_{2}A^{2}}{C_{\mathrm{LSI}}}\right),$

where $C_{2}$ is a universal constant and $C_{\mathrm{LSI}}$ depends on
parameters in $\mathcal{S}_{1}$. This tail probability bound holds uniformly
for $t\in[0,1]$. ∎

###### Proof of Lemma 5.8.

We decompose the truncation error by

$\displaystyle\mathcal{E}_{\mathrm{trunc}}$
$\displaystyle=\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|[\bar{v}(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})]\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{\mathsf{t}})\|_{2}^{2}$
$\displaystyle\lesssim\underbrace{\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|\bar{v}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{\mathsf{t}})\|_{2}^{2}}_{=:\mathcal{E}_{\mathrm{trunc}}^{1}}+\underbrace{\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{\mathsf{t}})\|_{2}^{2}}_{=:\mathcal{E}_{\mathrm{trunc}}^{2}}.$
(C.2)

First, we bound $\mathcal{E}_{\mathrm{trunc}}^{1}$. For any $A>0$ and
$t\in[0,1-\underline{t}]$, it holds that

$\displaystyle\mathbb{E}_{\mathsf{X}_{t}}\|\bar{v}(t,\mathsf{X}_{t})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{t})\|_{2}^{2}$
$\displaystyle=\mathbb{E}_{\mathsf{X}_{t}}[\|\bar{v}(t,\mathsf{X}_{t})\|_{2}^{2}\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{t})]$
$\displaystyle\leq\left(\mathbb{E}_{\mathsf{X}_{t}}[\|\bar{v}(t,\mathsf{X}_{t})\|_{2}^{4}]\cdot\mathbb{P}(\mathsf{X}_{t}\in\Omega_{A}^{c})\right)^{1/2}$
$\displaystyle\lesssim
A^{2}\mathbb{P}(\mathsf{X}_{t}\in\Omega_{A}^{c})^{1/2},$ (C.3)

where the first inequality follows from the Cauchy-Schwarz inequality, and the
second inequality is due to
$\|\bar{v}(t,x)\|_{L^{\infty}([0,1-\underline{t}]\times{\mathbb{R}}^{d})}\lesssim
A$ given in Theorem 5.4. Combining (5.3) in Lemma 5.7 and (C.3) above, it
follows

$\displaystyle\mathcal{E}_{\mathrm{trunc}}^{1}=\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|\bar{v}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{\mathsf{t}})\|_{2}^{2}\lesssim\sqrt{d}A^{2}\exp\left(-\frac{C_{3}A^{2}}{C_{\mathrm{LSI}}}\right),$
(C.4)

where $C_{3}$ is a universal constant.
Then, we bound $\mathcal{E}_{\mathrm{trunc}}^{2}$. Due to
$v^{*}(t,x)=\mathbb{E}[\mathsf{X}_{1}-\mathsf{Z}|\mathsf{X}_{t}=x]$, it holds

$\displaystyle\mathbb{E}_{\mathsf{X}_{t}}\|v^{*}(t,\mathsf{X}_{t})\|_{2}^{4}$
$\displaystyle=\mathbb{E}_{\mathsf{X}_{t}}\|\mathbb{E}[\mathsf{X}_{1}-\mathsf{Z}|\mathsf{X}_{t}=x]\|_{2}^{4}$
$\displaystyle\leq\mathbb{E}_{\mathsf{X}_{t}}\mathbb{E}[\|\mathsf{X}_{1}-\mathsf{Z}\|_{2}^{4}|\mathsf{X}_{t}=x]$
$\displaystyle\leq\mathbb{E}[\|\mathsf{X}_{1}-\mathsf{Z}\|_{2}^{4}]$
$\displaystyle\leq
8\mathbb{E}[\|\mathsf{Z}\|_{2}^{4}]+8\mathbb{E}[\|\mathsf{X}_{1}\|_{2}^{4}],$

where the fourth moments in the last expression are finite by the property of
the Gaussian distribution and the sub-Gaussian property of $\mathsf{X}_{1}$.
For any $A>0$ and $t\in[0,1-\underline{t}]$, we further bound
$\mathbb{E}_{\mathsf{X}_{t}}\|v^{*}(t,\mathsf{X}_{t})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{t})\|_{2}^{2}$
by

$\displaystyle\mathbb{E}_{\mathsf{X}_{t}}\|v^{*}(t,\mathsf{X}_{t})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{t})\|_{2}^{2}$
$\displaystyle=\mathbb{E}_{\mathsf{X}_{t}}[\|v^{*}(t,\mathsf{X}_{t})\|_{2}^{2}\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{t})]$
$\displaystyle\leq\left(\mathbb{E}_{\mathsf{X}_{t}}\|[v^{*}(t,\mathsf{X}_{t})\|_{2}^{4}]\cdot\mathbb{P}(\mathsf{X}_{t}\in\Omega_{A}^{c})\right)^{1/2}$
$\displaystyle\lesssim\mathbb{E}[\|\mathsf{X}_{1}\|_{2}^{4}]^{1/2}\cdot\mathbb{P}(\mathsf{X}_{t}\in\Omega_{A}^{c})^{1/2}.$
(C.5)

Combining (5.3) in Lemma 5.7 and (C.5) above, it follows

$\displaystyle\mathcal{E}_{\mathrm{trunc}}^{2}=\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\mathds{1}_{\Omega_{A}^{c}}(\mathsf{X}_{\mathsf{t}})\|_{2}^{2}\lesssim\sqrt{d}\exp\left(-\frac{C_{3}A^{2}}{C_{\mathrm{LSI}}}\right),$
(C.6)

where we omit the dependence on the fourth moment of the target
$\mathsf{X_{1}}$.
Finally, combining (C.2), (C.4), and (C.6), we get

$\displaystyle\mathcal{E}_{\mathrm{trunc}}\lesssim\sqrt{d}A^{2}\exp\left(-\frac{C_{3}A^{2}}{C_{\mathrm{LSI}}}\right),$

where we omit the dependence on the fourth moment of the target
$\mathsf{X_{1}}$. This completes the proof. ∎

###  C.3 Stochastic error

The stochastic error is known as generalization error in statistical machine
learning. In this part, we study the stochastic error of flow matching with
techniques in empirical processes and present the proof of Lemma 5.10. Before
that, we show necessary definitions from the content of empirical processes
for establishing bounds of the stochastic error.

######  Definition C.1 (Uniform and empirical covering numbers).

Given the samples $\mathbb{X}_{n}:=\\{\mathsf{X}_{i}\\}_{i=1}^{n}$, we define
the empirical $L^{\infty}$ pseudometric
$\|\cdot\|_{L^{\infty}(\mathbb{X}_{n})}$ on the samples $\mathbb{X}_{n}$ by

$\|f\|_{L^{\infty}(\mathbb{X}_{n})}:=\max_{1\leq i\leq n}|f(\mathsf{X}_{i})|.$

A set ${\mathcal{F}}_{\delta}$ is called an empirical $L^{\infty}$
$\delta$-cover of the function class ${\mathcal{F}}$ on the samples
$\mathbb{X}_{n}$ if for each $f\in{\mathcal{F}}$, there exists
$f^{\prime}\in{\mathcal{F}}_{\delta}$ such that
$\|f-f^{\prime}\|_{L^{\infty}(\mathbb{X}_{n})}\leq\delta$. Furthermore,

$\mathcal{N}_{\infty}(\delta,{\mathcal{F}},\mathbb{X}_{n}):=\inf\big{\\{}|{\mathcal{F}}_{\delta}|:{\mathcal{F}}_{\delta}\text{
is an empirical $L^{\infty}$ $\delta$-cover of ${\mathcal{F}}$ on
$\mathbb{X}_{n}$}\big{\\}}$

is called the empirical $L^{\infty}$ $\delta$-covering number of
${\mathcal{F}}$ on $\mathbb{X}_{n}$. Given $n$, the largest $L^{\infty}$
$\delta$-covering number over samples $\mathbb{X}_{n}$ is referred to as the
uniform $L^{\infty}$ $\delta$-covering number
$\mathcal{N}_{\infty}(\delta,{\mathcal{F}},n):=\sup_{\mathbb{X}_{n}}\mathcal{N}_{\infty}(\delta,{\mathcal{F}},\mathbb{X}_{n})$.

######  Definition C.2.

Let ${\mathcal{F}}$ be a class of functions from a set ${\mathcal{Z}}$ to
${\mathbb{R}}$. A set $\\{z_{1},\cdots,z_{m}\\}\subset{\mathcal{X}}$ is said
to be shattered by ${\mathcal{F}}$ if there exist
$t_{1},t_{2},\cdots,t_{m}\in{\mathbb{R}}$ such that, for each
$b\in\\{0,1\\}^{m}$, there exist a function $f_{b}\in{\mathcal{F}}$ satisfying
$\mathrm{sgn}\left(f_{b}\left(z_{i}\right)-t_{i}\right)=b_{i}$ for $1\leq
i\leq m$. We say that the threshold values $t_{1},t_{2},\cdots,t_{m}$ witness
the shattering.

######  Definition C.3 (Pseudo-dimension).

Let ${\mathcal{F}}$ be a class of functions from a set $\Omega$ to
${\mathbb{R}}$. The pseudo-dimension of ${\mathcal{F}}$, denoted by
$\mathrm{Pdim}({\mathcal{F}})$, is the maximum cardinality of a subset of
$\Omega$ shattered by ${\mathcal{F}}$.

###### Proof of Lemma 5.10.

Let
$\mathbb{D}_{n}=\\{\mathsf{S}_{i}:=(\mathsf{Z}_{i},\mathsf{X}_{1,i},\mathsf{t}_{i})\\}_{i=1}^{n}$
be a random sample from the distribution of
$\mathsf{Z},\mathsf{X}_{1},\mathsf{t}$ and
$\mathbb{D}_{n}^{\prime}:=\\{\mathsf{S}_{i}^{\prime}:=(\mathsf{Z}_{i}^{\prime},\mathsf{X}_{1,i}^{\prime},\mathsf{t}_{i}^{\prime})\\}_{i=1}^{n}$
be another ghost sample independent of $\mathbb{D}_{n}$. We denote that
$\mathsf{X}_{\mathsf{t}_{i}}:=(1-\mathsf{t}_{i})\mathsf{Z}_{i}+\mathsf{t}_{i}\mathsf{X}_{1,i}$,
$\mathsf{X}_{\mathsf{t}_{i}}^{\prime}:=(1-\mathsf{t}_{i}^{\prime})\mathsf{Z}_{i}^{\prime}+\mathsf{t}_{i}^{\prime}\mathsf{X}_{1,i}^{\prime}$,
$\mathsf{Y}_{i}:=\mathsf{X}_{1,i}-\mathsf{Z}_{i}$, and
$\mathsf{Y}_{i}^{\prime}:=\mathsf{X}_{1,i}^{\prime}-\mathsf{Z}_{i}^{\prime}$.
Define
${\mathcal{D}}(v,\mathsf{S}_{i}):=\|v(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-\mathsf{Y}_{i}\|_{2}^{2}-\|v^{*}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-\mathsf{Y}_{i}\|_{2}^{2}$
for any $v\in{\mathcal{F}}_{n}$ and $\mathsf{S}_{i}$. Notice that

$\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}(v^{*})-2\mathcal{L}_{n}(\hat{v}_{n})+\mathcal{L}(\hat{v}_{n})]=\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\frac{1}{n}\sum\nolimits_{i=1}^{n}\left(\mathbb{E}_{\mathbb{D}_{n}^{\prime}}{\mathcal{D}}(\hat{v}_{n},\mathsf{S}_{i}^{\prime})-2{\mathcal{D}}(\hat{v}_{n},\mathsf{S}_{i})\right)\Big{]}.$
(C.7)

It is clear that the right-hand side of (C.7) defines an asymmetric empirical
process. Let
${\mathcal{G}}(v,\mathsf{S}_{i}):=\mathbb{E}_{\mathbb{D}_{n}^{\prime}}{\mathcal{D}}(v,\mathsf{S}_{i}^{\prime})-2{\mathcal{D}}(v,\mathsf{S}_{i})$
for any $v\in{\mathcal{F}}_{n}$. Then we have

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}[\mathcal{L}(v^{*})-2\mathcal{L}_{n}(\hat{v}_{n})+\mathcal{L}(\hat{v}_{n})]=\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}(\hat{v}_{n},\mathsf{S}_{i})\Big{]}.$

Let $B_{n}\geq\mathtt{B}\geq B\geq 1$ be a positive number that may depend on
the sample size $n$. We construct a clipping function ${\mathcal{C}}_{B_{n}}$
at level $B_{n}$ following the definition of clipping functions in Lemma B.9.
Let
$v_{B_{n}}(t,x):=\mathbb{E}[{\mathcal{C}}_{B_{n}}(\mathsf{Y})|\mathsf{X}_{t}=x]$
be the regression function of the truncated $\mathsf{Y}$. Similar to the
definitions of ${\mathcal{D}}(v,\mathsf{S}_{i})$ and
${\mathcal{G}}(v,\mathsf{S}_{i})$, we define
${\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i}):=\|v(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})\|_{2}^{2}-\|v_{B_{n}}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})\|_{2}^{2}$
and
${\mathcal{G}}_{B_{n}}(v,\mathsf{S}_{i}):=\mathbb{E}_{\mathbb{D}_{n}^{\prime}}{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i}^{\prime})-2{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})$.
Then for any $v\in{\mathcal{F}}_{n}$ we have

$\displaystyle~{}|{\mathcal{D}}(v,\mathsf{S}_{i})-{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})|$
$\displaystyle=$ $\displaystyle\big{|}2\langle
v(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-v^{*}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}}),{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})-\mathsf{Y}_{i}\rangle$
$\displaystyle+\|v_{B_{n}}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})\|_{2}^{2}-\|v^{*}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})\|_{2}^{2}\big{|}$
$\displaystyle\leq$ $\displaystyle
2\big{|}\big{\langle}v(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-v^{*}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}}),{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})-\mathsf{Y}_{i}\big{\rangle}\big{|}$
$\displaystyle+\big{|}\big{\langle}v_{B_{n}}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-v^{*}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}}),v_{B_{n}}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})+v^{*}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-2{\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i})\big{\rangle}\big{|}.$

By considering coordinate-wise scalar expressions of the risks, we get

$\displaystyle~{}|{\mathcal{D}}(v,\mathsf{S}_{i})-{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})|$
$\displaystyle\leq$
$\displaystyle\sum\nolimits_{j=1}^{d}\Big{\\{}2\big{|}[v_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-v^{*}_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})]\cdot[({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}-(\mathsf{Y}_{i})_{j}]\big{|}$
$\displaystyle+\big{|}[(v_{B_{n}})_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-v^{*}_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})]\cdot[(v_{B_{n}})_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})+v^{*}_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-2({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}]\big{|}\Big{\\}}$
$\displaystyle\leq$
$\displaystyle\sum\nolimits_{j=1}^{d}\Big{\\{}4\mathtt{B}\big{|}({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}-(\mathsf{Y}_{i})_{j}\big{|}+4B_{n}\big{|}(v_{B_{n}})_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-v^{*}_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})\big{|}\Big{\\}}$
$\displaystyle\leq$
$\displaystyle\sum\nolimits_{j=1}^{d}\Big{\\{}4\mathtt{B}\big{|}({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}-(\mathsf{Y}_{i})_{j}\big{|}+4B_{n}\mathbb{E}[|({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}-(\mathsf{Y}_{i})_{j}|\big{|}\mathsf{X}_{t_{i}}=\mathsf{X}_{\mathsf{t}_{i}}]\Big{\\}}.$

Note that
$|({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}-(\mathsf{Y}_{i})_{j}|\leq|(\mathsf{Y}_{i})_{j}|\mathds{1}_{\\{|(\mathsf{Y}_{i})_{j}|\geq
B_{n}\\}}$ and $B_{n}\geq\mathtt{B}$. Then it follows that

$\displaystyle~{}\mathbb{E}_{\mathbb{D}_{n}}|{\mathcal{D}}(v,\mathsf{S}_{i})-{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})|$
$\displaystyle\leq$
$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\sum\nolimits_{j=1}^{d}\Big{\\{}4\mathtt{B}|(\mathsf{Y_{i}})_{j}|\mathds{1}_{\\{|(\mathsf{Y_{i}})_{j}|\geq
B_{n}\\}}+4B_{n}\mathbb{E}[|(\mathsf{Y}_{i})_{j}|\mathds{1}_{\\{|(\mathsf{Y}_{i})_{j}|\geq
B_{n}\\}}\big{|}\mathsf{X}_{t_{i}}=\mathsf{X}_{\mathsf{t}_{i}}]\Big{\\}}\Big{]}$
$\displaystyle\leq$
$\displaystyle\sum\nolimits_{j=1}^{d}8B_{n}\mathbb{E}_{\mathbb{D}_{n}}\big{[}|(\mathsf{Y}_{i})_{j}|\mathds{1}_{\\{|(\mathsf{Y}_{i})_{j}|\geq
B_{n}\\}}\big{]}$ $\displaystyle\leq$
$\displaystyle\sum\nolimits_{j=1}^{d}8B_{n}\mathbb{E}_{\mathbb{D}_{n}}\big{[}|(\mathsf{Y}_{i})_{j}|^{2}\big{]}\cdot\mathbb{P}\\{|(\mathsf{Y}_{i})_{j}|\geq
B_{n}\\}.$

By Assumption 2 and Remark 5, the law of $\mathsf{Y}=\mathsf{X}-\mathsf{Z}$ is
sub-Gaussian. Then there exist two constants $K_{1}$ and $K_{2}$ such that for
each $i=1,2,\cdots,n$ and $j=1,2,\cdots,d$,

$\displaystyle\mathbb{P}\\{|(\mathsf{Y}_{i})_{j}|\geq B_{n}\\}\leq
2\exp\left(-\frac{B_{n}^{2}}{K_{1}^{2}}\right),\quad\mathbb{E}_{\mathbb{D}_{n}}\big{[}|(\mathsf{Y}_{i})_{j}|^{2}\big{]}\leq
2K_{2}^{2}.$

The bounds above further imply that

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}{\mathcal{D}}(v,\mathsf{S}_{i})\leq$
$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})+\sum\nolimits_{j=1}^{d}8B_{n}\mathbb{E}_{\mathbb{D}_{n}}\big{[}|(\mathsf{Y}_{i})_{j}|^{2}\big{]}\mathbb{P}(|(\mathsf{Y}_{i})_{j}|\geq
B_{n})$ $\displaystyle\leq$
$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})+32dB_{n}K_{2}^{2}\exp\left(-\frac{B_{n}^{2}}{K_{1}^{2}}\right).$

Therefore, we conclude that

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}(\hat{v}_{n},\mathsf{S}_{i})\Big{]}\leq\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(\hat{v}_{n},\mathsf{S}_{i})\Big{]}+K_{3}B_{n}\exp\left(-\frac{B_{n}^{2}}{K_{1}^{2}}\right),$
(C.8)

where the constant $K_{3}$ does not depend on $n$ and $B_{n}$.

Next, we consider bounding a tail probability of the empirical process. Before
proceeding, we define
$({\mathcal{D}}_{B_{n}})_{j}(v,\mathsf{S}_{i}):=[v_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}]^{2}-[(v_{B_{n}})_{j}(\mathsf{t}_{i},\mathsf{X}_{\mathsf{t}_{i}})-({\mathcal{C}}_{B_{n}}(\mathsf{Y}_{i}))_{j}]^{2}$
for any $j\in\\{1,2,\cdots,d\\}$. It is clear that
${\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})=\sum_{j=1}^{d}({\mathcal{D}}_{B_{n}})_{j}(v,\mathsf{S}_{i})$,
and we have the following tail probability bounds

$\displaystyle\mathbb{P}\Big{\\{}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(\hat{v}_{n},\mathsf{S}_{i})>t\Big{\\}}$
$\displaystyle\leq$ $\displaystyle\mathbb{P}\Big{\\{}\exists
v\in{\mathcal{F}}_{n}:\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(v,\mathsf{S}_{i})>t\Big{\\}}$
$\displaystyle=$ $\displaystyle\mathbb{P}\Big{\\{}\exists
v\in{\mathcal{F}}_{n}:\mathbb{E}_{\mathbb{D}_{n}^{\prime}}{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i}^{\prime})-\frac{2}{n}\sum\nolimits_{i=1}^{n}{\mathcal{D}}_{B_{n}}(v,\mathsf{S}_{i})>t\Big{\\}}$
$\displaystyle\leq$ $\displaystyle\mathbb{P}\Big{\\{}\exists
v\in{\mathcal{F}}_{n}\text{ and }\exists 1\leq j\leq
d:\mathbb{E}_{\mathbb{D}_{n}^{\prime}}({\mathcal{D}}_{B_{n}})_{j}(v,\mathsf{S}_{i}^{\prime})-\frac{2}{n}\sum\nolimits_{i=1}^{n}({\mathcal{D}}_{B_{n}})_{j}(v,\mathsf{S}_{i})>\frac{t}{d}\Big{\\}}.$

Note that $|({\mathcal{C}}_{B_{n}}(\mathsf{Y}))_{j}|\leq B_{n}$,
$\|(v_{B_{n}})_{j}\|_{\infty}\leq B_{n}$, and $B_{n}\geq\mathtt{B}\geq 1$. By
Theorem 11.4 of Györfi et al. (2002) and letting
$\epsilon=1/2,\alpha=\beta=t/(2d)$ in Györfi et al. (2002, Theorem 11.4), it
yields that for each $n\geq 1$,

$\displaystyle\mathbb{P}\Big{\\{}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(\hat{v}_{n},\mathsf{S}_{i})>t\Big{\\}}$
$\displaystyle\leq$ $\displaystyle\mathbb{P}\Big{\\{}\exists
v\in{\mathcal{F}}_{n}\text{ and }\exists 1\leq j\leq
d:\mathbb{E}_{\mathbb{D}_{n}^{\prime}}({\mathcal{D}}_{B_{n}})_{j}(v,\mathsf{S}_{i}^{\prime})-\frac{2}{n}\sum\nolimits_{i=1}^{n}({\mathcal{D}}_{B_{n}})_{j}(v,\mathsf{S}_{i})>\frac{t}{d}\Big{\\}}$
$\displaystyle\leq$ $\displaystyle
14\mathcal{N}_{\infty}(t/(80dB_{n}),{\mathcal{F}}_{n},n)\exp\left(-\frac{tn}{5136dB_{n}^{4}}\right).$
(C.9)

Then we use the tail probability bound (C.9) to bound the stochastic error.
For any $\alpha_{n}>0$,

$\displaystyle~{}\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(\hat{v}_{n},\mathsf{S}_{i})\Big{]}$
$\displaystyle\leq$
$\displaystyle\alpha_{n}+\int_{\alpha_{n}}^{\infty}\mathbb{P}\Big{\\{}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(\hat{v}_{n},\mathsf{S}_{i})>t\Big{\\}}\mathrm{d}t$
$\displaystyle\leq$
$\displaystyle\alpha_{n}+\int_{\alpha_{n}}^{\infty}14\mathcal{N}_{\infty}(t/(80dB_{n}),{\mathcal{F}}_{n},n)\exp\left(-\frac{tn}{5136dB_{n}^{4}}\right)\mathrm{d}t$
$\displaystyle\leq$
$\displaystyle\alpha_{n}+\int_{\alpha_{n}}^{\infty}14\mathcal{N}_{\infty}(\alpha_{n}/(80dB_{n}),{\mathcal{F}}_{n},n)\exp\left(-\frac{tn}{5136dB_{n}^{4}}\right)\mathrm{d}t$
$\displaystyle\leq$
$\displaystyle\alpha_{n}+14\mathcal{N}_{\infty}(\alpha_{n}/(80dB_{n}),{\mathcal{F}}_{n},n)\exp\left(-\frac{\alpha_{n}n}{5136dB_{n}^{4}}\right)\frac{5136dB_{n}^{4}}{n}.$

By choosing
$\alpha_{n}=\log(14\mathcal{N}_{\infty}(1/n,{\mathcal{F}}_{n},n))\cdot
5136dB_{n}^{4}/n$ and noticing that $\alpha_{n}/(80dB_{n})\geq 1/n$ and
$\mathcal{N}_{\infty}(1/n,{\mathcal{F}}_{n},n)\geq\mathcal{N}_{\infty}(\alpha_{n}/(80dB_{n}),{\mathcal{F}}_{n},n)$,
we obtain that

$\mathbb{E}_{\mathbb{D}_{n}}\Big{[}\frac{1}{n}\sum\nolimits_{i=1}^{n}{\mathcal{G}}_{B_{n}}(\hat{v}_{n},\mathsf{S}_{i})\Big{]}\leq\frac{5136dB_{n}^{4}(\log(14\mathcal{N}_{\infty}(1/n,{\mathcal{F}}_{n},n))+1)}{n}.$
(C.10)

Setting $B_{n}\asymp\mathtt{B}\log n$ and combining (C.8) and (C.10), the
stochastic error is upper bounded by the covering number of the hypothesis
class
${\mathcal{F}}_{n}\subseteq\mathcal{NN}(\mathtt{S},\mathtt{W},\mathtt{D},\mathtt{B},d+1,d)$
as

$\displaystyle\mathcal{E}_{\mathrm{stoc}}\lesssim\frac{d}{n}(\log
n)^{4}\mathtt{B}^{4}\log\mathcal{N}_{\infty}(1/n,{\mathcal{F}}_{n},n)$ (C.11)

where we omit a constant not depending on $n$ or $\mathtt{B}$. By the
relationship between the uniform covering number and the pseudo-dimension of
the deep ReLU network class ${\mathcal{F}}_{n}$ (Anthony and Bartlett, 1999,
Theorem 12.2), it yields that for $n\geq\mathrm{Pdim}({\mathcal{F}}_{n})$,

$\displaystyle\mathcal{N}_{\infty}(1/n,{\mathcal{F}}_{n},n)\leq\left(\frac{2e\mathtt{B}n^{2}}{\mathrm{Pdim}({\mathcal{F}}_{n})}\right)^{\mathrm{Pdim}({\mathcal{F}}_{n})},$
(C.12)

where $\mathrm{Pdim}({\mathcal{F}}_{n})$ denotes the pseudo-dimension of
${\mathcal{F}}_{n}$. By Theorems 3 and 7 of Bartlett et al. (2019), the
pseudo-dimension of the deep ReLU network class
${\mathcal{F}}_{n}\subseteq\mathcal{NN}(\mathtt{S},\mathtt{W},\mathtt{D},\mathtt{B},d+1,d)$
satisfies

$\displaystyle\mathtt{S}\mathtt{D}\log(\mathtt{S}/\mathtt{D})\lesssim\mathrm{Pdim}({\mathcal{F}}_{n})\lesssim\mathtt{S}\mathtt{D}\log(\mathtt{S}).$
(C.13)

Combining (C.11), (C.12), (C.13), and $\mathtt{B}\lesssim A$, we complete the
proof by showing

$\displaystyle\mathcal{E}_{\mathrm{stoc}}\lesssim\frac{d}{n}(\log
n)^{4}A^{4}\mathtt{S}\mathtt{D}\log(\mathtt{S})\log(An^{2}).$

∎

###  C.4 Balancing errors

We present the proof of Theorem 5.12 for balancing the approximation error and
the stochastic error of flow matching.

###### Proof of Theorem 5.12.

According to Corollary 5.9 and Corollary 5.11, it holds that

$\displaystyle\mathcal{E}_{\mathrm{stoc}}\lesssim\frac{1}{n}A^{4}\underline{t}^{-2}(NL)^{2+2/d},\quad\mathcal{E}_{\mathrm{appr}}\lesssim
A^{2}(NL)^{-4/d}+A^{2}\exp(-C_{3}A^{2}/C_{\textrm{LSI}})$

by omitting a polylogarithmic prefactor in $N,L,A,n$, a prefactor in
$\log(1/\underline{t})$, and a prefactor in $d,\kappa,\beta,\sigma,D,R$. Let
$NL\asymp(n\underline{t}^{2})^{d/(2d+6)}$ and $A\asymp\log(\log n)$. Then by
Lemma 5.2, it holds that

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|\hat{v}_{n}(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}\leq\mathcal{E}_{\mathrm{stoc}}+2\mathcal{E}_{\mathrm{appr}}\lesssim(n\underline{t}^{2})^{-2/(d+3)}$

by omitting a polylogarithmic prefactor in $n$, a prefactor in
$\log(1/\underline{t})$, and a prefactor in $d,\kappa,\beta,\sigma,D,R$. ∎

##  Appendix D Distribution estimation errors

In the appendix, we provide proofs for bounding distribution estimation
errors. The discretization error is bounded in Lemma 4.1.

###### Proof of Lemma 4.1.

By the definition of Wasserstein-2 distance, it holds

$\mathcal{W}_{2}^{2}(\hat{p}_{t},\tilde{p}_{t})\leq\int_{{\mathbb{R}}^{d}}\|\hat{X}_{t}(x)-\tilde{X}_{t}(x)\|_{2}^{2}p_{0}(x)\mathrm{d}x=:E_{t}.$

It suffices to consider the propagation of error $E_{t}$ in time
$t\in[0,1-\underline{t}]$. Recall that
$(\hat{X}_{t})_{t\in[0,1-\underline{t}]}$ is the linear interpolation of
$(\hat{X}_{t_{k}})_{0\leq k\leq K}$, thus it is piecewise linear over
$[0,1-\underline{t}]$. To ease the arguments, we consider the dynamics of
$E_{t}$ over each time subinterval $[t_{k-1},t_{k}]$ for $1\leq k\leq K$. For
$t\in[t_{k-1},t_{k}]$, it holds that

$\displaystyle\frac{\mathrm{d}E_{t}}{\mathrm{d}t}=$
$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t_{k-1},\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\tilde{X}_{t}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t_{k-1},\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\hat{X}_{t_{k-1}}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ (D.1)
$\displaystyle+\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\hat{X}_{t}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ (D.2)
$\displaystyle+\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\hat{X}_{t}(x))-\hat{v}_{n}(t,\tilde{X}_{t}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ (D.3)

For term (D.1), the basic inequality $2\langle
a,b\rangle\leq\|a\|_{2}^{2}+\|b\|_{2}^{2}$ and the fact that $\hat{v}_{n}$ is
$\mathtt{L}_{t}$-Lipschitz continuous in $t$ imply that

$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t_{k-1},\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\hat{X}_{t_{k-1}}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ $\displaystyle\leq$
$\displaystyle\int_{{\mathbb{R}}^{d}}\|\hat{v}_{n}(t_{k-1},\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\hat{X}_{t_{k-1}}(x))\|_{2}^{2}p_{0}(x)\mathrm{d}x$
$\displaystyle+\int_{{\mathbb{R}}^{d}}\|\hat{X}_{t}(x)-\tilde{X}_{t}(x)\|_{2}^{2}p_{0}(x)\mathrm{d}x$
$\displaystyle\leq$ $\displaystyle\ d\mathtt{L}_{t}^{2}(t-t_{k-1})^{2}+E_{t}.$
(D.4)

Note that
$\hat{X}_{t}(x)=\hat{X}_{t_{k-1}}(x)+(t-t_{k-1})\hat{v}_{n}(t_{k-1},\hat{X}_{t_{k-1}}(x))$.
For term (D.2), we use $2\langle a,b\rangle\leq\|a\|_{2}^{2}+\|b\|_{2}^{2}$
and the fact that $\hat{v}_{n}$ is $\mathtt{L}_{x}$-Lipschitz continuous in
$x$ to deduce that

$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\hat{X}_{t}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ $\displaystyle\leq$
$\displaystyle\int_{{\mathbb{R}}^{d}}\|\hat{v}_{n}(t,\hat{X}_{t_{k-1}}(x))-\hat{v}_{n}(t,\hat{X}_{t}(x))\|_{2}^{2}p_{0}(x)\mathrm{d}x$
$\displaystyle+\int_{{\mathbb{R}}^{d}}\|\hat{X}_{t}(x)-\tilde{X}_{t}(x)\|_{2}^{2}p_{0}(x)\mathrm{d}x$
$\displaystyle\leq$ $\displaystyle\
d\mathtt{L}_{x}^{2}(t-t_{k-1})^{2}\|\hat{v}_{n}\|_{L^{\infty}([0,1-\underline{t}]\times{\mathbb{R}}^{d})}^{2}+E_{t}$
$\displaystyle\leq$ $\displaystyle\
d\mathtt{L}_{x}^{2}\mathtt{B}^{2}(t-t_{k-1})^{2}+E_{t}.$ (D.5)

For term (D.3), by the Cauchy-Schwartz inequality and the fact that
$\hat{v}_{n}$ is $\mathtt{L}_{x}$-Lipschitz continuous in $x$, we obtain

$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\hat{X}_{t}(x))-\hat{v}_{n}(t,\tilde{X}_{t}(x)),\hat{X}_{t}(x)-\tilde{X}_{t}(x)\rangle
p_{0}(x)\mathrm{d}x\leq 2\mathtt{L}_{x}E_{t}.$ (D.6)

Combining (D.4), (D.5), and (D.6), we obtain

$\frac{\mathrm{d}E_{t}}{\mathrm{d}t}\leq
2(\mathtt{L}_{x}+1)E_{t}+d(\mathtt{L}_{x}^{2}\mathtt{B}^{2}+\mathtt{L}_{t}^{2})(t-t_{k-1})^{2}\quad\text{
for $t\in[t_{k-1},t_{k}]$}.$

By Grönwall’s inequality, it further yields

$e^{-2(\mathtt{L}_{x}+1)t_{k}}E_{t_{k}}-e^{-2(\mathtt{L}_{x}+1)t_{k-1}}E_{t_{k-1}}\leq\frac{1}{3}d(\mathtt{L}_{x}^{2}\mathtt{B}^{2}+\mathtt{L}_{t}^{2})(t_{k}-t_{k-1})^{3}.$

Taking sum over $k=1,2,\cdots,K$ and letting $t_{K}=1-\underline{t}$, we
obtain

$E_{1-\underline{t}}\leq\frac{1}{3}de^{2(\mathtt{L}_{x}+1)(1-\underline{t})}(\mathtt{L}_{x}^{2}\mathtt{B}^{2}+\mathtt{L}_{t}^{2})\sum\nolimits_{k=1}^{K}(t_{k}-t_{k-1})^{3}.$

Let $\Upsilon\equiv t_{k}-t_{k-1}$ for $k=1,2,\cdots,K$. It implies that

$\mathcal{W}_{2}(\hat{p}_{1-\underline{t}},\tilde{p}_{1-\underline{t}})=\mathcal{O}\left(\sqrt{d}e^{\mathtt{L}_{x}}(\mathtt{L}_{x}\mathtt{B}+\mathtt{L}_{t})\Upsilon\right).$

This completes the proof. ∎

The error due to velocity estimation is bounded in Lemma 4.2.

###### Proof of Lemma 4.2.

The proof idea is similar to that of Albergo and Vanden-Eijnden (2023,
Proposition 3). By the definition of the Wasserstein-$2$ distance, it holds

$\mathcal{W}_{2}^{2}(\tilde{p}_{t},p_{t})\leq\int_{{\mathbb{R}}^{d}}\|\tilde{X}_{t}(x)-X_{t}(x)\|_{2}^{2}p_{0}(x)\mathrm{d}x=:R_{t},$

for any $t\in[0,1-\underline{t}]$. By (3.6) and (3.13), it follows that

$\displaystyle\frac{\mathrm{d}R_{t}}{\mathrm{d}t}=$
$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\tilde{X}_{t}(x))-v^{*}(t,X_{t}(x)),\tilde{X}_{t}(x)-X_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ $\displaystyle=$
$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\tilde{X}_{t}(x))-\hat{v}_{n}(t,X_{t}(x)),\tilde{X}_{t}(x)-X_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ (D.7)
$\displaystyle+\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,X_{t}(x))-v^{*}(t,X_{t}(x)),\tilde{X}_{t}(x)-X_{t}(x)\rangle
p_{0}(x)\mathrm{d}x.$ (D.8)

For term (D.7), the fact that $\hat{v}_{n}$ is $\mathtt{L}_{x}$-Lipschitz
continuous in $x$ imply that

$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,\tilde{X}_{t}(x))-\hat{v}_{n}(t,X_{t}(x)),\tilde{X}_{t}(x)-X_{t}(x)\rangle
p_{0}(x)\mathrm{d}x\leq 2\mathtt{L}_{x}R_{t}.$

For term (D.8), the basic inequality $2\langle
a,b\rangle\leq\|a\|_{2}^{2}+\|b\|_{2}^{2}$ imply that

$\displaystyle\int_{{\mathbb{R}}^{d}}2\langle\hat{v}_{n}(t,X_{t}(x))-v^{*}(t,X_{t}(x)),\tilde{X}_{t}(x)-X_{t}(x)\rangle
p_{0}(x)\mathrm{d}x$ $\displaystyle\leq R_{t}+\mathbb{E}_{\mathsf{X}_{t}\sim
p_{t}}\|\hat{v}_{n}(t,\mathsf{X}_{t})-v^{*}(t,\mathsf{X}_{t})\|_{2}^{2}.$

Therefore, we have

$\displaystyle\frac{\mathrm{d}R_{t}}{\mathrm{d}t}\leq(2\mathtt{L}_{x}+1)R_{t}+\mathbb{E}_{\mathsf{X}_{t}\sim
p_{t}}\|\hat{v}_{n}(t,\mathsf{X}_{t})-v^{*}(t,\mathsf{X}_{t})\|_{2}^{2}.$

By Grönwall’s inequality, it further yields

$\displaystyle
R_{1-\underline{t}}\leq\exp(2\mathtt{L}_{x}+1)\mathbb{E}_{(\mathsf{t},\mathsf{X}_{\mathsf{t}})}\|\hat{v}_{n}(\mathsf{t},\mathsf{X}_{\mathsf{t}})-v^{*}(\mathsf{t},\mathsf{X}_{\mathsf{t}})\|_{2}^{2}.$

We complete the proof by noting that
$\mathcal{W}_{2}^{2}(\tilde{p}_{1-\underline{t}},p_{1-\underline{t}})\leq
R_{1-\underline{t}}$. ∎

The early stopping error is bounded in Lemma 4.3.

###### Proof of Lemma 4.3.

The proof is a basic calculation.

$\displaystyle\mathcal{W}_{2}^{2}(p_{1-\underline{t}},p_{1})$
$\displaystyle\leq\mathbb{E}[\|\mathsf{X}_{1-\underline{t}}-\mathsf{X}_{1}\|_{2}^{2}]=\mathbb{E}[\|\underline{t}(\mathsf{Z}-\mathsf{X}_{1})\|_{2}^{2}]$
$\displaystyle=\underline{t}^{2}\left(\mathbb{E}[\|\mathsf{Z}\|_{2}^{2}]+\mathbb{E}[\|\mathsf{X}_{1}\|_{2}^{2}]\right)\lesssim\underline{t}^{2},$

where we omit a polynomial prefactor in
$d,\mathbb{E}[\|\mathsf{X}_{1}\|_{2}^{2}]$. We complete the proof by taking
square roots of both sides. ∎

###### Proof of Theorem 4.4.

Combining Eq. (4.1), Lemmas 4.1, 4.2, 4.3, and Theorem 5.12, it yields

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}\mathcal{W}_{2}(\hat{p}_{1-\underline{t}},p_{1})\lesssim(n\underline{t}^{2})^{-1/(d+3)}+e^{\mathtt{L}_{x}}(\mathtt{L}_{x}\mathtt{B}+\mathtt{L}_{t})\Upsilon+\underline{t}.$

Let $\underline{t}\asymp n^{-1/(d+5)}$, $A\asymp\log(\log n)$, and
$\Upsilon=\mathcal{O}(n^{-3/(d+5)})$. Then it implies

$\displaystyle\mathbb{E}_{\mathbb{D}_{n}}\mathcal{W}_{2}(\hat{p}_{1-\underline{t}},p_{1})\lesssim\underline{t}\vee(n\underline{t}^{2})^{-1/(d+3)}\lesssim
n^{-1/(d+5)},$

where we omit a prefactor scaling polynomially in $\log n$ and a prefactor
with dependence on parameters in $\mathcal{S}_{2}$. This completes the proof.
∎

##  Appendix E Supporting definitions and lemmas

Sobolev spaces are widely studied in the context of functional analysis and
partial differential equations. For ease of reference, we collect several
definitions and existing results on Sobolev spaces that assist our proof. For
a thorough treatment of Sobolev spaces, the interested reader is referred to
Adams and Fournier (2003); Evans (2010). Moreover, we present some results on
polynomial approximation theory in Sobolev spaces that are developed in the
classical monograph on the finite element methods (Brenner and Scott, 2008).
In the sequel, let $d\in\mathbb{N}$, $\Omega\subset{\mathbb{R}}^{d}$ denote an
open subset of $\mathbb{R}^{d}$. We denote by $L^{\infty}(\Omega)$ the
standard Lebesgue space on $\Omega$ with $L^{\infty}$ norm.

###  E.1 Sobolev spaces

We list some definitions for defining Sobolev spaces.

######  Definition E.1 (Sobolev space).

Let $n\in\mathbb{N}_{0}$. Then the Sobolev space $W^{n,\infty}(\Omega)$ is
defined by

$\displaystyle W^{n,\infty}(\Omega):=\\{f\in L^{\infty}(\Omega):D^{\alpha}f\in
L^{\infty}(\Omega)\text{ for all $\alpha\in\mathbb{N}_{0}^{d}$ with
$\|\alpha\|_{1}\leq n$ }\\}.$

Moreover, for any $f\in W^{n,\infty}(\Omega)$, we define the Sobolev norm
$\|\cdot\|_{W^{n,\infty}(\Omega)}$ by

$\displaystyle\|f\|_{W^{n,\infty}(\Omega)}:=\max_{0\leq\|\alpha\|_{1}\leq
n}\|D^{\alpha}f\|_{L^{\infty}(\Omega)}.$

######  Definition E.2 (Sobolev semi-norm).

Let $n,k\in\mathbb{N}_{0}$ with $k\leq n$. For any $f\in
W^{n,\infty}(\Omega)$, we define the Sobolev semi-norm
$|\cdot|_{W^{k,\infty}(\Omega)}$ by

$\displaystyle|f|_{W^{k,\infty}(\Omega)}:=\max_{\|\alpha\|_{1}=k}\|D^{\alpha}f\|_{L^{\infty}(\Omega)}.$

######  Definition E.3 (Vector-valued Sobolev space).

Let $n,k\in\mathbb{N}_{0}$ with $k\leq n$, and $m\in\mathbb{N}$. Then the
vector-valued Sobolev space $W^{n,\infty}(\Omega;{\mathbb{R}}^{m})$ is defined
by

$\displaystyle
W^{n,\infty}(\Omega;{\mathbb{R}}^{m}):=\\{(f_{1},f_{2},\cdots,f_{m}):f_{i}\in
W^{n,\infty}(\Omega),1\leq i\leq m\\}$

Moreover, the Sobolev norm $\|\cdot\|_{W^{n,\infty}(\Omega;{\mathbb{R}}^{m})}$
is defined by

$\displaystyle\|f\|_{W^{n,\infty}(\Omega;{\mathbb{R}}^{m})}:=\max_{1\leq i\leq
m}\|f_{i}\|_{W^{n,\infty}(\Omega)},$

and the Sobolev semi-norm $|\cdot|_{W^{n,\infty}(\Omega;{\mathbb{R}}^{m})}$ is
defined by

$\displaystyle|f|_{W^{k,\infty}(\Omega;{\mathbb{R}}^{m})}:=\max_{1\leq i\leq
m}|f_{i}|_{W^{k,\infty}(\Omega)}.$

###  E.2 Averaged Taylor polynomials

The following definitions and lemmas on averaged Taylor polynomials are
collected from Chapter 4 of Brenner and Scott (2008).

######  Definition E.4 (Averaged Taylor polynomials).

Let $\Omega\subset{\mathbb{R}}^{d}$ be a bounded, open subset and $f\in
W^{m-1,\infty}(\Omega)$ for some $m\in\mathbb{N}$, and let
$x_{0}\in\Omega,r>0,B:=\mathbb{B}^{d}(x_{0},r,\|\cdot\|_{2})$ with its closure
$\bar{B}$ compact in $\Omega$. The Taylor polynomial of order $m$ of $f$
averaged over $B$ is defined as

$\displaystyle Q^{m}f(x):=\int_{B}T_{y}^{m}f(x)\phi(y)\mathrm{d}y,$

where

$\displaystyle
T_{y}^{m}f(x):=\sum_{\|\alpha\|_{1}<m}\frac{1}{\alpha!}D^{\alpha}f(y)(x-y)^{\alpha},$

and $\phi$ is an arbitrary cut-off function supported in $\bar{B}$ being
infinitely differentiable, that is, $\phi\in C^{\infty}({\mathbb{R}}^{d})$
with $\mathrm{supp}(\phi)=\bar{B}$ and
$\int_{{\mathbb{R}}^{d}}\phi(x)\mathrm{d}x=1$.

######  Example 1.

Let $\psi(x)$ be defined by

$\displaystyle\psi(x):=\begin{cases}\exp\\{-1/(1-(\|x-x_{0}\|_{2}/r)^{2})\\},&\text{if
}\|x-x_{0}\|_{2}<r,\\\ 0,&\text{if }\|x-x_{0}\|_{2}\geq r,\end{cases}$

and let $c=\int_{{\mathbb{R}}^{d}}\psi(x)\mathrm{d}x$ with $c>0$, then
$\phi(x)=\psi(x)/c$ is an example of the cut-off function on the ball
$B:=\mathbb{B}^{d}(x_{0},r,\|\cdot\|_{2})$. Moreover, it holds that
$\|\phi\|_{L^{\infty}(B)}\leq C(d)r^{-d}$ where $C(d)>0$ is a constant in $d$.

######  Example 2.

Let $\phi(x)$ be defined by

$\displaystyle\phi(x)=\begin{cases}\pi^{-d/2}\Gamma(d/2+1)r^{-d},&\text{if
}\|x-x_{0}\|_{2}<r,\\\ 0,&\text{if }\|x-x_{0}\|_{2}\geq r,\end{cases}$

then $\phi(x)$ is another example of the cut-off function where $\phi$ puts
constant weight over the ball $B:=\mathbb{B}^{d}(x_{0},r,\|\cdot\|_{2})$.

######  Lemma E.5 (Lemma B.9 in Gühring, Kutyniok and Petersen (2020)).

Let $\Omega\subset{\mathbb{R}}^{d}$ be a bounded, open subset and $f\in
W^{m-1,\infty}(\Omega)$ for some $m\in\mathbb{N}$, and let
$x_{0}\in\Omega,r>0,B:=\mathbb{B}^{d}(x_{0},r,\|\cdot\|_{2})$ with its closure
$\bar{B}$ compact in $\Omega$. The Taylor polynomial of order $m$ of $f$
averaged over $B$ denoted by $Q^{m}f(x)$ is a polynomial of degree less than
$m$ in $x$.

######  Definition E.6 (Star-shaped set).

Let $\Omega,B\subset{\mathbb{R}}^{d}$. We say $\Omega$ is star-shaped with
respect to $B$ if for all $x\in\Omega$, the closed convex hull of $\\{x\\}\cup
B$ is a subset of $\Omega$.

######  Definition E.7 (Chunkiness parameter).

Suppose that $\Omega\subset{\mathbb{R}}^{d}$ has diameter $d_{\Omega}$ and is
star-shaped with respect to a ball $B$. Let

$\displaystyle r_{\max}:=\sup\\{r>0:\Omega\text{ is star-shaped with respect
to a ball of radius }r\\}.$

Then the chunkiness parameter of $\Omega$ is defined by
$\gamma:=d_{\Omega}/r_{\max}$.

######  Lemma E.8 (Bramble-Hilbert, Lemma 4.3.8 in Brenner and Scott (2008)).

Let $B$ be a ball in $\Omega\subset{\mathbb{R}}^{d}$ such that $\Omega$ is
star-shaped with respect to $B$ and such that its radius $r>r_{\max}/2$, where
$r_{\max}$ is defined in Definition E.7. Moreover, let $d_{\Omega}$ be the
diameter of $\Omega$, $\gamma$ be the chunkiness parameter of $\Omega$, and
$Q^{m}f$ be the Taylor polynomial of order $m$ of $f$ averaged over $B$ for
any $f\in W^{m,\infty}(\Omega)$. Then there exists a constant
$C(d,m,\gamma)>0$ such that

$\displaystyle|f-Q^{m}f|_{W^{k,\infty}(\Omega)}\leq
C(d,m,\gamma)d_{\Omega}^{m-k}|f|_{W^{m,\infty}(\Omega)},\quad k=0,1,\cdots,m.$

##  Appendix F Additional lemmas on approximation

######  Lemma F.1 (Corollary B.5 in Gühring, Kutyniok and Petersen (2020)).

Let $d,m\in\mathbb{N}$ and
$\Omega_{1}\subset{\mathbb{R}}^{d},\Omega_{2}\subset{\mathbb{R}}^{m}$ both be
open, bounded, and convex. If $f\in W^{1,\infty}(\Omega_{1};{\mathbb{R}}^{m})$
and $g\in W^{1,\infty}(\Omega_{2})$ with $\mathrm{rad}(f)\subset\Omega_{2}$,
then for the composition $g\circ f$, it holds that $g\circ f\in
W^{1,\infty}(\Omega_{1})$ and we have

$\displaystyle|g\circ
f|_{W^{1,\infty}(\Omega_{1})}\leq\sqrt{d}m|g|_{W^{1,\infty}(\Omega_{2})}|f|_{W^{1,\infty}(\Omega_{1};{\mathbb{R}}^{m})}$

and

$\displaystyle\|g\circ
f\|_{W^{1,\infty}(\Omega_{1})}\leq\sqrt{d}m\max\\{\|g\|_{L^{\infty}(\Omega_{2})},|g|_{W^{1,\infty}(\Omega_{2})}|f|_{W^{1,\infty}(\Omega_{1};{\mathbb{R}}^{m})}\\}.$

######  Lemma F.2 (Corollary B.6 in Gühring, Kutyniok and Petersen (2020)).

Let $f\in W^{1,\infty}(\Omega)$ and $g\in W^{1,\infty}(\Omega)$. Then $fg\in
W^{1,\infty}(\Omega)$ and we have

$\displaystyle|fg|_{W^{1,\infty}(\Omega)}\leq|f|_{W^{1,\infty}(\Omega)}\|g\|_{L^{\infty}(\Omega)}+\|f\|_{L^{\infty}(\Omega)}|g|_{W^{1,\infty}(\Omega)}$

and

$\displaystyle\|fg\|_{W^{1,\infty}(\Omega)}\leq\|f\|_{W^{1,\infty}(\Omega)}\|g\|_{L^{\infty}(\Omega)}+\|f\|_{L^{\infty}(\Omega)}\|g\|_{W^{1,\infty}(\Omega)}.$

######  Lemma F.3 (Proposition 4 in Yang, Yang and Xiang (2023)).

For any $N,L\in\mathbb{N}$ and $a>0$, there exists a deep ReLU network
$\phi_{\times,a}$ with width $15N$ and depth $2L$ such that
$\|\phi_{\times,a}\|_{W^{1,\infty}((-a,a)^{2})}\leq 12a^{2}$ and

$\displaystyle\|\phi_{\times,a}(x,y)-xy\|_{W^{1,\infty}((-a,a)^{2})}\leq
6a^{2}N^{-L}.$

Furthermore, it holds that

$\displaystyle\phi_{\times,a}(0,y)=\frac{\partial\phi_{\times,a}(0,y)}{\partial
y}=0\text{ for $y\in(-a,a)$}.$

######  Lemma F.4.

For any $N,L\in\mathbb{N}$ and $a,b>0$, there exists a deep ReLU network
$\phi_{\times,(a,b)}$ with width $15N$ and depth $2L$ such that
$\|\phi_{\times,(a,b)}\|_{W^{1,\infty}((-a,a)\times(-b,b))}\leq 12ab$ and

$\displaystyle\|\phi_{\times,(a,b)}(x,y)-xy\|_{W^{1,\infty}((-a,a)\times(-b,b))}\leq
6abN^{-L}.$

Furthermore, it holds that

$\displaystyle\phi_{\times,(a,b)}(x,0)=\frac{\partial\phi_{\times,(a,b)}(x,0)}{\partial
x}=0\text{ for $x\in(-a,a)$}.$

###### Proof.

The proof idea is similar to that of Proposition 4 in Yang, Yang and Xiang
(2023). ∎

######  Lemma F.5 (Proposition 5 in Yang, Yang and Xiang (2023)).

For any $N,L,s\in\mathbb{N}$ with $s\geq 2$, there exists a deep ReLU network
$\phi_{\times}$ with width $\mathcal{O}(s\vee N)$ and depth
$\mathcal{O}(s^{2}L)$ such that

$\displaystyle\|\phi_{\times}(x)-x_{1}x_{2}\cdots
x_{s}\|_{W^{1,\infty}((0,1)^{s})}\lesssim s(N+1)^{-7sL}.$

Furthermore, for any $i=1,2,\cdots,s$, if $x_{i}=0$, then we have

$\displaystyle\phi_{\times}(x_{1},x_{2},\cdots,x_{i-1},0,x_{i+1},\cdots,x_{s})=\frac{\partial\phi_{\times}(x_{1},x_{2},\cdots,x_{i-1},0,x_{i+1},\cdots,x_{s})}{\partial
x_{j}}=0,\ i\neq j.$

######  Lemma F.6 (Lemma 6 in Yang, Yang and Xiang (2023)).

Let $\\{g_{m}\\}_{m\in\\{1,2\\}^{d}}$ be the partition of unity given in
Definition B.4. Then it satisfies:

  * (1)

$\sum_{m\in\\{1,2\\}^{d}}g_{m}(x)=1$ for every $x\in[0,1]^{d}$;

  * (2)

$\mathrm{supp}(g_{m})\cap[0,1]^{d}\subset\Omega_{m}$ where $\Omega_{m}$ is
given in Definition B.3;

  * (3)

For any $m=[m_{1},m_{2},\cdots,m_{d}]^{\top}\in\\{1,2\\}^{d}$ and
$x=[x_{1},x_{2},\cdots,x_{d}]^{\top}\in[0,1]^{d}\setminus\Omega_{m}$, there
exists an index $j\in\\{1,2,\cdots,d\\}$ such that $g_{m_{j}}=0$ and
$\frac{\mathrm{d}g_{m_{j}}(x_{j})}{\mathrm{d}x_{j}}=0$.

######  Lemma F.7 (Lemma 7 in Yang, Yang and Xiang (2023)).

For any $\chi(x)\in W^{1,\infty}((0,1)^{d})$, let
$B_{5}:=\max\\{\|\chi\|_{W^{1,\infty}((0,1)^{d})},\|\phi_{m}\|_{W^{1,\infty}((0,1)^{d})}\\}$.
Then for any $m\in\\{1,2\\}^{d}$, it holds that

$\displaystyle\|\phi_{m}(x)\cdot\chi(x)\|_{W^{1,\infty}((0,1)^{d})}=\|\phi_{m}(x)\cdot\chi(x)\|_{W^{1,\infty}(\Omega_{m})},$
$\displaystyle\|\phi_{m}(x)\cdot\chi(x)-\phi_{\times,B_{5}}(\phi_{m}(x),\chi(x))\|_{W^{1,\infty}((0,1)^{d})}$
$\displaystyle=\|\phi_{m}(x)\cdot\chi(x)-\phi_{\times,B_{5}}(\phi_{m}(x),\chi(x))\|_{W^{1,\infty}(\Omega_{m})}.$

######  Lemma F.8.

For any $\chi(x)\in L^{\infty}((0,1)^{d})$, let
$B_{6}:=\max\\{\|\chi\|_{L^{\infty}((0,1)^{d})},\|\phi_{m}\|_{L^{\infty}((0,1)^{d})}\\}$.
Then for any $m\in\\{1,2\\}^{d}$, it holds that

$\displaystyle\|\phi_{m}(x)\cdot\chi(x)\|_{L^{\infty}((0,1)^{d})}=\|\phi_{m}(x)\cdot\chi(x)\|_{L^{\infty}(\Omega_{m})},$
$\displaystyle\|\phi_{m}(x)\cdot\chi(x)-\phi_{\times,B_{6}}(\phi_{m}(x),\chi(x))\|_{L^{\infty}((0,1)^{d})}$
$\displaystyle=\|\phi_{m}(x)\cdot\chi(x)-\phi_{\times,B_{6}}(\phi_{m}(x),\chi(x))\|_{L^{\infty}(\Omega_{m})}.$

###### Proof.

The proof is similar to that of (Yang, Yang and Xiang, 2023, Lemma 7). To
prove the equalities, we need to show that

$\displaystyle\|\phi_{m}(x)\cdot\chi(x)\|_{L^{\infty}((0,1)^{d}\setminus\Omega_{m})}=0\quad\text{and}\quad\|\phi_{\times,B_{6}}(\phi_{m}(x),\chi(x))\|_{L^{\infty}((0,1)^{d}\setminus\Omega_{m})}=0.$

In Lemma B.5, it is shown that

$\displaystyle\phi_{m}(x):=\phi_{\times}(g_{m_{1}}(x_{1}),g_{m_{2}}(x_{2}),\cdots,g_{m_{d}}(x_{d}))=0$

where
$g_{m}(x)=[g_{m_{1}}(x_{1}),g_{m_{2}}(x_{2}),\cdots,g_{m_{d}}(x_{d})]^{\top}$
is defined in Definition B.4. Then by Lemma F.6, for any
$x=[x_{1},x_{2},\cdots,x_{d}]^{\top}\in(0,1)^{d}\setminus\Omega_{m}$, there
exists $m_{j}$ such that $g_{m_{j}}(x_{j})=0$. By the definition of
$\phi_{m}(x)$ and Lemma F.5, it yields that

$\displaystyle\phi_{m}(x)=0,\quad\forall x\in(0,1)^{d}\setminus\Omega_{m}.$

Therefore, for any $x\in(0,1)^{d}\setminus\Omega_{m}$, it holds that
$|\phi_{m}(x)\cdot\chi(x)|=0$.

Similarly, for any $x\in(0,1)^{d}\setminus\Omega_{m}$, it holds that

$\displaystyle\phi_{\times,B_{6}}(\phi_{m}(x),\chi(x))=\phi_{\times,B_{6}}(0,\chi(x))=0.$

This completes the proof. ∎

######  Lemma F.9 (Proposition 4.3 in Lu et al. (2021)).

Given any $N,L\in\mathbb{N}$ and $\delta\in(0,1/(3K)]$ for $K=\lfloor
N^{1/d}\rfloor^{2}\lfloor L^{2/d}\rfloor$, there exists a deep ReLU network
$\phi$ with width $4N+5$ and depth $4L+4$ such that

$\displaystyle\phi(x)=k,\quad
x\in\left[\frac{k}{K},\frac{k+1}{K}-\delta\cdot\mathds{1}_{\\{k<K-1\\}}\right],\quad
k=0,1,\cdots,K-1.$

######  Lemma F.10 (Proposition 4.4 in Lu et al. (2021)).

Given any $N,L,s\in\mathbb{N}$ and $\xi_{i}\in[0,1]$ for
$i=0,1,\cdots,N^{2}L^{2}-1$, there exists a deep ReLU network $\phi$ with
width $16s(N+1)\log_{2}(8N)$ and depth $(5L+2)\log_{2}(4L)$ such that

$\displaystyle|\phi(i)-\xi_{i}|\leq(NL)^{-2s}\text{ for
}i=0,1,\cdots,N^{2}L^{2}-1$

and that

$\displaystyle\phi(x)\in[0,1],\ x\in{\mathbb{R}}.$

##  Appendix G Auxiliary lemmas

In this appendix, we exhibit Brascamp-Lieb inequality (Brascamp and Lieb,
1976, Theorem 4.1), Tweedie’s formula (Robbins, 1956; Efron, 2011), and
Hatsell-Nolte identity (Hatsell and Nolte, 1971; Dytso, Poor and Shamai Shitz,
2023).

######  Lemma G.1 (Brascamp-Lieb inequality).

Let $\mu(\mathrm{d}x)=\exp(-U(x))\mathrm{d}x$ be a probability distribution on
a convex set $\Omega\subseteq{\mathbb{R}}^{d}$ whose potential function
$U:\Omega\to{\mathbb{R}}$ is of class $C^{2}$ and strictly convex. Then for
every locally Lipschitz function $f\in L^{2}(\Omega,\mu)$,

$\mathrm{Var}_{\mathsf{X}\sim\mu}(f)\leq\mathbb{E}_{\mathsf{X}\sim\mu}\left[\langle\nabla_{x}f(\mathsf{X}),(\nabla^{2}_{x}U(\mathsf{X}))^{-1}\nabla_{x}f(\mathsf{X})\rangle\right].$
(G.1)

######  Lemma G.2 (Tweedie’s formula).

Suppose that $\mathsf{X}\sim\mu$ and $\epsilon\sim\gamma_{d,\sigma^{2}}$. Let
$\mathsf{Y}=\mathsf{X}+\epsilon$ and $p(y)$ be the marginal density of
$\mathsf{Y}$. Then it holds that
$\mathbb{E}[\mathsf{X}|\mathsf{Y}=y]=y+\sigma^{2}\nabla_{y}\log p(y)$.

######  Lemma G.3 (Hatsell-Nolte identity).

Suppose that $\mathsf{X}\sim\mu$ and $\epsilon\sim\gamma_{d,\sigma^{2}}$. Let
$\mathsf{Y}=\mathsf{X}+\epsilon$ and $p(y)$ be the marginal density of
$\mathsf{Y}$. Then it holds that

$\displaystyle\mathrm{Cov}(\mathsf{X}|\mathsf{Y}=y)=\sigma^{2}\nabla_{y}\mathbb{E}[\mathsf{X}|\mathsf{Y}=y]=\sigma^{2}\mathrm{I}_{d}+\sigma^{4}\nabla^{2}_{y}\log
p(y).$

## References

  * Adams and Fournier (2003) [author] Adams, Robert A.R. A. and Fournier, John J. F.J. J. F. (2003). Sobolev Spaces, second ed. Pure and Applied Mathematics 140\. Academic Press.
  * Albergo, Boffi and Vanden-Eijnden (2023) [author] Albergo, Michael S.M. S., Boffi, Nicholas MN. M. and Vanden-Eijnden, EricE. (2023). Stochastic interpolants: A unifying framework for flows and diffusions. arXiv preprint arXiv:2303.08797.
  * Albergo and Vanden-Eijnden (2023) [author] Albergo, Michael S.M. S. and Vanden-Eijnden, EricE. (2023). Building normalizing flows with stochastic interpolants. In The Eleventh International Conference on Learning Representations.
  * Anthony and Bartlett (1999) [author] Anthony, MartinM. and Bartlett, Peter L.P. L. (1999). Neural Network Learning: Theoretical Foundations 9\. Cambridge University Press.
  * Bartlett, Maiorov and Meir (1998) [author] Bartlett, PeterP., Maiorov, VitalyV. and Meir, RonR. (1998). Almost linear VC dimension bounds for piecewise polynomial networks. In Advances in Neural Information Processing Systems 11\.
  * Bartlett et al. (2019) [author] Bartlett, Peter L.P. L., Harvey, NickN., Liaw, ChristopherC. and Mehrabian, AbbasA. (2019). Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks. The Journal of Machine Learning Research 20 2285–2301.
  * Bauer and Kohler (2019) [author] Bauer, BenediktB. and Kohler, MichaelM. (2019). On deep learning as a remedy for the curse of dimensionality in nonparametric regression. The Annals of Statistics 47 2261 – 2285.
  * Benton, Deligiannidis and Doucet (2024) [author] Benton, JoeJ., Deligiannidis, GeorgeG. and Doucet, ArnaudA. (2024). Error bounds for flow matching methods. Transactions on Machine Learning Research.
  * Benton et al. (2024) [author] Benton, JoeJ., Bortoli, Valentin DeV. D., Doucet, ArnaudA. and Deligiannidis, GeorgeG. (2024). Linear convergence bounds for diffusion models via stochastic localization. In The Twelfth International Conference on Learning Representations.
  * Bortoli (2022) [author] Bortoli, Valentin DeV. D. (2022). Convergence of denoising diffusion models under the manifold hypothesis. Transactions on Machine Learning Research.
  * Brascamp and Lieb (1976) [author] Brascamp, Herm J.H. J. and Lieb, Elliott H.E. H. (1976). On extensions of the Brunn-Minkowski and Prékopa-Leindler theorems, including inequalities for log concave functions, and with an application to the diffusion equation. Journal of Functional Analysis 22 366–389.
  * Brenner and Scott (2008) [author] Brenner, Susanne C.S. C. and Scott, L. RidgwayL. R. (2008). The Mathematical Theory of Finite Element Methods, third ed. Texts in Applied Mathematics 15 4, 93–127. Springer New York, New York, NY.
  * Bruno et al. (2023) [author] Bruno, StefanoS., Zhang, YingY., Lim, Dong-YoungD.-Y., Akyildiz, Ömer DenizÖ. D. and Sabanis, SotiriosS. (2023). On diffusion-based generative models and their error bounds: The log-concave case with full convergence estimates. arXiv preprint arXiv:2311.13584.
  * Cattiaux and Guillin (2014) [author] Cattiaux, PatrickP. and Guillin, ArnaudA. (2014). Semi log-concave Markov diffusions. In Séminaire de probabilités XLVI (CatherineC. Donati-Martin, AntoineA. Lejay and AlainA. Rouault, eds.) 231–292. Springer International Publishing, Cham.
  * Chang et al. (2024) [author] Chang, JinyuanJ., Ding, ZhaoZ., Jiao, YulingY., Li, RuoxuanR. and Yang, Jerry ZhijianJ. Z. (2024). Deep conditional generative learning: model and error analysis. arXiv preprint arXiv:2402.01460.
  * Chen, Daras and Dimakis (2023) [author] Chen, SitanS., Daras, GiannisG. and Dimakis, AlexA. (2023). Restoration-degradation beyond linear diffusions: A non-asymptotic analysis for DDIM-type samplers. In Proceedings of the 40th International Conference on Machine Learning 202 4462–4484. PMLR.
  * Chen, Lee and Lu (2023) [author] Chen, HongruiH., Lee, HoldenH. and Lu, JianfengJ. (2023). Improved analysis of score-based generative modeling: User-friendly bounds under minimal smoothness assumptions. In Proceedings of the 40th International Conference on Machine Learning 202 4735–4763. PMLR.
  * Chen et al. (2018) [author] Chen, Ricky T. Q.R. T. Q., Rubanova, YuliaY., Bettencourt, JesseJ. and Duvenaud, David KD. K. (2018). Neural ordinary differential equations. In Advances in Neural Information Processing Systems 31\. Curran Associates, Inc.
  * Chen et al. (2020) [author] Chen, MinshuoM., Liao, WenjingW., Zha, HongyuanH. and Zhao, TuoT. (2020). Distribution approximation and statistical estimation guarantees of generative sdversarial networks. arXiv preprint arXiv:2002.03938.
  * Chen et al. (2022) [author] Chen, MinshuoM., Jiang, HaomingH., Liao, WenjingW. and Zhao, TuoT. (2022). Nonparametric regression on low-dimensional manifolds using deep ReLU networks: Function approximation and statistical recovery. Information and Inference: A Journal of the IMA 11 1203-1253.
  * Chen et al. (2023a) [author] Chen, SitanS., Chewi, SinhoS., Li, JerryJ., Li, YuanzhiY., Salim, AdilA. and Zhang, AnruA. (2023a). Sampling is as easy as learning the score: Theory for diffusion models with minimal data assumptions. In The Eleventh International Conference on Learning Representations.
  * Chen et al. (2023b) [author] Chen, SitanS., Chewi, SinhoS., Lee, HoldenH., Li, YuanzhiY., Lu, JianfengJ. and Salim, AdilA. (2023b). The probability flow ODE is provably fast. In Thirty-seventh Conference on Neural Information Processing Systems.
  * Chen et al. (2023c) [author] Chen, MinshuoM., Huang, KaixuanK., Zhao, TuoT. and Wang, MengdiM. (2023c). Score approximation, estimation and distribution recovery of diffusion models on low-dimensional data. In Proceedings of the 40th International Conference on Machine Learning. Proceedings of Machine Learning Research 202 4672–4712. PMLR.
  * Cheng et al. (2023) [author] Cheng, XiuyuanX., Lu, JianfengJ., Tan, YixinY. and Xie, YaoY. (2023). Convergence of flow-based generative models via proximal gradient descent in Wasserstein space. arXiv preprint arXiv:2310.17582.
  * Cole and Lu (2024) [author] Cole, FrankF. and Lu, YulongY. (2024). Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian distributions. In The Twelfth International Conference on Learning Representations.
  * Cui et al. (2023) [author] Cui, HugoH., Krzakala, FlorentF., Vanden-Eijnden, EricE. and Zdeborová, LenkaL. (2023). Analysis of learning a flow-based generative model from limited sample complexity. arXiv preprint arXiv:2310.03575.
  * Dai et al. (2023) [author] Dai, YinY., Gao, YuanY., Huang, JianJ., Jiao, YulingY., Kang, LicanL. and Liu, JinJ. (2023). Lipschitz transport maps via the Föllmer flow. arXiv preprint arXiv:2309.03490.
  * Dalalyan (2017) [author] Dalalyan, Arnak S.A. S. (2017). Theoretical guarantees for approximate sampling from smooth and log-concave densities. Journal of the Royal Statistical Society Series B: Statistical Methodology 79 651-676.
  * Daubechies et al. (2022) [author] Daubechies, IngridI., DeVore, RonaldR., Foucart, SimonS., Hanin, BorisB. and Petrova, GuerganaG. (2022). Nonlinear approximation and (deep) ReLU networks. Constructive Approximation 55 127–172.
  * DeVore, Hanin and Petrova (2021) [author] DeVore, RonaldR., Hanin, BorisB. and Petrova, GuerganaG. (2021). Neural network approximation. Acta Numerica 30 327–444.
  * DeVore, Howard and Micchelli (1989) [author] DeVore, Ronald A.R. A., Howard, RalphR. and Micchelli, CharlesC. (1989). Optimal nonlinear approximation. Manuscripta mathematica 63 469–478.
  * Durmus and Moulines (2017) [author] Durmus, AlainA. and Moulines, ÉricÉ. (2017). Nonasymptotic convergence analysis for the unadjusted Langevin algorithm. The Annals of Applied Probability 27 1551 – 1587.
  * Dwivedi et al. (2019) [author] Dwivedi, RaazR., Chen, YuansiY., Wainwright, Martin JM. J. and Yu, BinB. (2019). Log-concave sampling: Metropolis-Hastings algorithms are fast. Journal of Machine Learning Research 20 1–42.
  * Dytso, Poor and Shamai Shitz (2023) [author] Dytso, AlexA., Poor, H. VincentH. V. and Shamai Shitz, ShlomoS. (2023). Conditional mean estimation in Gaussian noise: A meta derivative identity with applications. IEEE Transactions on Information Theory 69 1883-1898.
  * Efron (2011) [author] Efron, BradleyB. (2011). Tweedie’s formula and selection bias. Journal of the American Statistical Association 106 1602–1614.
  * Eldan and Lee (2018) [author] Eldan, RonenR. and Lee, James R.J. R. (2018). Regularization under diffusion and anticoncentration of the information content. Duke Mathematical Journal 167 969–993.
  * Esser et al. (2024) [author] Esser, PatrickP., Kulal, SumithS., Blattmann, AndreasA., Entezari, RahimR., Müller, JonasJ., Saini, HarryH., Levi, YamY., Lorenz, DominikD., Sauer, AxelA., Boesel, FredericF., Podell, DustinD., Dockhorn, TimT., English, ZionZ., Lacey, KyleK., Goodwin, AlexA., Marek, YannikY. and Rombach, RobinR. (2024). Scaling rectified flow transformers for high-resolution image synthesis. arXiv preprint arXiv:2403.03206.
  * Evans (2010) [author] Evans, Lawrence C.L. C. (2010). Partial Differential Equations, second ed. Graduate Studies in Mathematics 19\. American Mathematical Society.
  * Fan and Gu (2023) [author] Fan, JianqingJ. and Gu, YihongY. (2023). Factor augmented sparse throughput deep ReLU neural networks for high dimensional regression. Journal of the American Statistical Association just-accepted 1–28.
  * Farrell, Liang and Misra (2021) [author] Farrell, Max H.M. H., Liang, TengyuanT. and Misra, SanjogS. (2021). Deep neural networks for estimation and inference. Econometrica 89 181-213.
  * Gao, Huang and Jiao (2023) [author] Gao, YuanY., Huang, JianJ. and Jiao, YulingY. (2023). Gaussian interpolation flows. arXiv preprint arXiv:2311.11475.
  * Gao, Nguyen and Zhu (2023) [author] Gao, XuefengX., Nguyen, Hoang MH. M. and Zhu, LingjiongL. (2023). Wasserstein convergence guarantees for a general class of score-based generative models. arXiv preprint arXiv:2311.11003.
  * Gao and Zhu (2024) [author] Gao, XuefengX. and Zhu, LingjiongL. (2024). Convergence analysis for general probability flow ODEs of diffusion models in Wasserstein distances. arXiv preprint arXiv:2401.17958.
  * Goodfellow et al. (2014) [author] Goodfellow, IanI., Pouget-Abadie, JeanJ., Mirza, MehdiM., Xu, BingB., Warde-Farley, DavidD., Ozair, SherjilS., Courville, AaronA. and Bengio, YoshuaY. (2014). Generative adversarial nets. In Advances in Neural Information Processing Systems 27 2672–2680.
  * Gühring, Kutyniok and Petersen (2020) [author] Gühring, IngoI., Kutyniok, GittaG. and Petersen, PhilippP. (2020). Error bounds for approximations with deep ReLU neural networks in $\mathit{W}^{s,p}$ norms. Analysis and Applications 18 803-859.
  * Györfi et al. (2002) [author] Györfi, LászlóL., Kohler, MichaelM., Krzyzak, AdamA. and Walk, HarroH. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer.
  * Hartman (2002) [author] Hartman, PhilipP. (2002). Existence. In Ordinary Differential Equations II, 8-23. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA.
  * Hatsell and Nolte (1971) [author] Hatsell, Charles P.C. P. and Nolte, Loren W.L. W. (1971). Some geometric properties of the likelihood ratio (Corresp.). IEEE Transactions on Information Theory 17 616-618.
  * Ho, Jain and Abbeel (2020) [author] Ho, JonathanJ., Jain, AjayA. and Abbeel, PieterP. (2020). Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33 6840–6851.
  * Hon and Yang (2022) [author] Hon, SeanS. and Yang, HaizhaoH. (2022). Simultaneous neural network approximation for smooth functions. Neural Networks 154 152–164.
  * Huang et al. (2022) [author] Huang, JianJ., Jiao, YulingY., Li, ZhenZ., Liu, ShiaoS., Wang, YangY. and Yang, YunfeiY. (2022). An error analysis of generative adversarial networks for learning distributions. Journal of Machine Learning Research 23 1–43.
  * Huang et al. (2023) [author] Huang, DingD., Huang, JianJ., Li, TingT. and Shen, GuohaoG. (2023). Conditional stochastic interpolation for generative learning. arXiv preprint arXiv:2312.05579.
  * Jiao, Wang and Yang (2023) [author] Jiao, YulingY., Wang, YangY. and Yang, YunfeiY. (2023). Approximation bounds for norm constrained neural networks with applications to regression and GANs. Applied and Computational Harmonic Analysis 65 249-278.
  * Jiao et al. (2023) [author] Jiao, YulingY., Shen, GuohaoG., Lin, YuanyuanY. and Huang, JianJ. (2023). Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors. The Annals of Statistics 51 691–716.
  * Jing, Berger and Jaakkola (2023) [author] Jing, BowenB., Berger, BonnieB. and Jaakkola, TommiT. (2023). AlphaFold meets flow matching for generating protein ensembles. In NeurIPS 2023 Generative AI and Biology (GenBio) Workshop.
  * Kingma and Welling (2014) [author] Kingma, Diederik P.D. P. and Welling, MaxM. (2014). Auto-encoding variational bayes. In International Conference on Learning Representations.
  * Kohler and Langer (2021) [author] Kohler, MichaelM. and Langer, SophieS. (2021). On the rate of convergence of fully connected deep neural network regression estimates. The Annals of Statistics 49 2231 – 2249.
  * LeCun, Bengio and Hinton (2015) [author] LeCun, YannY., Bengio, YoshuaY. and Hinton, GeoffreyG. (2015). Deep learning. Nature 521 436–444.
  * Ledoux (2001) [author] Ledoux, MichelM. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89\. American Mathematical Society.
  * Lee, Lu and Tan (2022) [author] Lee, HoldenH., Lu, JianfengJ. and Tan, YixinY. (2022). Convergence for score-based generative modeling with polynomial complexity. In Advances in Neural Information Processing Systems 35 22870–22882. Curran Associates, Inc.
  * Lee, Lu and Tan (2023) [author] Lee, HoldenH., Lu, JianfengJ. and Tan, YixinY. (2023). Convergence of score-based generative modeling for general data distributions. In Proceedings of The 34th International Conference on Algorithmic Learning Theory 201 946–985. PMLR.
  * Li et al. (2024) [author] Li, GenG., Wei, YutingY., Chen, YuxinY. and Chi, YuejieY. (2024). Towards non-asymptotic convergence for diffusion-based generative models. In The Twelfth International Conference on Learning Representations.
  * Lipman et al. (2023) [author] Lipman, YaronY., Chen, Ricky T. Q.R. T. Q., Ben-Hamu, HeliH., Nickel, MaximilianM. and Le, MatthewM. (2023). Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations.
  * Liu, Gong and Liu (2023) [author] Liu, XingchaoX., Gong, ChengyueC. and Liu, QiangQ. (2023). Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations.
  * Lu et al. (2021) [author] Lu, JianfengJ., Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2021). Deep network approximation for smooth functions. SIAM Journal on Mathematical Analysis 53 5465-5506.
  * Ma et al. (2024) [author] Ma, NanyeN., Goldstein, MarkM., Albergo, Michael S.M. S., Boffi, Nicholas M.N. M., Vanden-Eijnden, EricE. and Xie, SainingS. (2024). SiT: Exploring flow and diffusion-based generative models with scalable interpolant transformers. arXiv preprint arXiv:2401.08740.
  * Marzouk et al. (2023) [author] Marzouk, YoussefY., Ren, ZhiZ., Wang, SvenS. and Zech, JakobJ. (2023). Distribution learning via neural differential equations: A nonparametric statistical perspective. arXiv preprint arXiv:2309.01043.
  * McCann (1997) [author] McCann, Robert J.R. J. (1997). A convexity principle for interacting gases. Advances in Mathematics 128 153–179.
  * Mikulincer and Shenfeld (2021) [author] Mikulincer, DanD. and Shenfeld, YairY. (2021). The Brownian transport map. arXiv preprint arXiv:2111.11521.
  * Nakada and Imaizumi (2020) [author] Nakada, RyumeiR. and Imaizumi, MasaakiM. (2020). Adaptive approximation and generalization of deep neural network with intrinsic dimensionality. Journal of Machine Learning Research 21 1–38.
  * Neklyudov et al. (2023) [author] Neklyudov, KirillK., Brekelmans, RobR., Severo, DanielD. and Makhzani, AlirezaA. (2023). Action matching: Learning stochastic dynamics from samples. In Proceedings of the 40th International Conference on Machine Learning 202 25858–25889. PMLR.
  * Oko, Akiyama and Suzuki (2023) [author] Oko, KazusatoK., Akiyama, ShuntaS. and Suzuki, TaijiT. (2023). Diffusion models are minimax optimal distribution estimators. In Proceedings of the 40th International Conference on Machine Learning 202 26517–26582. PMLR.
  * Padilla, Tansey and Chen (2022) [author] Padilla, Oscar Hernan MadridO. H. M., Tansey, WesleyW. and Chen, YanzhenY. (2022). Quantile regression with ReLU Networks: Estimators and minimax rates. The Journal of Machine Learning Research 23 11251–11292.
  * Pedrotti, Maas and Mondelli (2023) [author] Pedrotti, FrancescoF., Maas, JanJ. and Mondelli, MarcoM. (2023). Improved convergence of score-based diffusion models via prediction-correction. arXiv preprint arXiv:2305.14164.
  * Petersen and Voigtlaender (2018) [author] Petersen, PhilippP. and Voigtlaender, FelixF. (2018). Optimal approximation of piecewise smooth functions using deep ReLU neural networks. Neural Networks 108 296–330.
  * Rezende and Mohamed (2015) [author] Rezende, DaniloD. and Mohamed, ShakirS. (2015). Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning 37 1530–1538. PMLR.
  * Robbins (1956) [author] Robbins, Herbert E.H. E. (1956). An empirical Bayes approach to statistics. In Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955 I 157–163. University of California Press, Berkeley and Los Angeles.
  * Salakhutdinov (2015) [author] Salakhutdinov, RuslanR. (2015). Learning deep generative models. Annual Review of Statistics and Its Application 2 361–385.
  * Schmidt-Hieber (2020) [author] Schmidt-Hieber, JohannesJ. (2020). Nonparametric regression using deep neural networks with ReLU activation function. The Annals of Statistics 48 1875 – 1897.
  * Shah, Chen and Klivans (2023) [author] Shah, KulinK., Chen, SitanS. and Klivans, AdamA. (2023). Learning mixtures of gaussians using the DDPM objective. In Advances in Neural Information Processing Systems 36\.
  * Shen, Yang and Zhang (2020) [author] Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2020). Deep network approximation characterized by number of neurons. Communications in Computational Physics 28 1768–1811.
  * Shen, Yang and Zhang (2022) [author] Shen, ZuoweiZ., Yang, HaizhaoH. and Zhang, ShijunS. (2022). Optimal approximation rate of ReLU networks in terms of width and depth. Journal de Mathématiques Pures et Appliquées 157 101-135.
  * Shen et al. (2022) [author] Shen, GuohaoG., Jiao, YulingY., Lin, YuanyuanY., Horowitz, Joel L.J. L. and Huang, JianJ. (2022). Estimation of non-crossing quantile regression process with deep ReQU neural networks. arXiv preprint arXiv:2207.10442. Accepted by JMLR.
  * Siegel (2023) [author] Siegel, Jonathan W.J. W. (2023). Optimal approximation rates for deep ReLU neural networks on Sobolev and Besov spaces. Journal of Machine Learning Research 24 1–52.
  * Sohl-Dickstein et al. (2015) [author] Sohl-Dickstein, JaschaJ., Weiss, EricE., Maheswaranathan, NiruN. and Ganguli, SuryaS. (2015). Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the 32nd International Conference on Machine Learning 37 2256–2265. PMLR.
  * Song and Ermon (2019) [author] Song, YangY. and Ermon, StefanoS. (2019). Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems 32 11895–11907. Curran Associates, Inc.
  * Song, Meng and Ermon (2021) [author] Song, JiamingJ., Meng, ChenlinC. and Ermon, StefanoS. (2021). Denoising diffusion implicit models. In International Conference on Learning Representations.
  * Song et al. (2021) [author] Song, YangY., Sohl-Dickstein, JaschaJ., Kingma, Diederik P.D. P., Kumar, AbhishekA., Ermon, StefanoS. and Poole, BenB. (2021). Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations.
  * Song et al. (2023) [author] Song, YuxuanY., Gong, JingjingJ., Xu, MinkaiM., Cao, ZiyaoZ., Lan, YanyanY., Ermon, StefanoS., Zhou, HaoH. and Ma, Wei-YingW.-Y. (2023). Equivariant flow matching with hybrid probability transport for 3D molecule generation. In Advances in Neural Information Processing Systems 36\.
  * Stone (1982) [author] Stone, Charles JC. J. (1982). Optimal global rates of convergence for nonparametric regression. The Annals of Statistics 10 1040–1053.
  * Suzuki (2019) [author] Suzuki, TaijiT. (2019). Adaptivity of deep ReLU network for learning in Besov and mixed smooth Besov spaces: optimal rate and curse of dimensionality. In International Conference on Learning Representations.
  * Suzuki and Nitanda (2021) [author] Suzuki, TaijiT. and Nitanda, AtsushiA. (2021). Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic Besov space. In Advances in Neural Information Processing Systems 34 3609–3621. Curran Associates, Inc.
  * Tabak and Turner (2013) [author] Tabak, Esteban G.E. G. and Turner, Cristina V.C. V. (2013). A family of nonparametric density estimation algorithms. Communications on Pure and Applied Mathematics 66 145–164.
  * Vempala and Wibisono (2019) [author] Vempala, SantoshS. and Wibisono, AndreA. (2019). Rapid convergence of the unadjusted Langevin algorithm: Isoperimetry suffices. In Advances in Neural Information Processing Systems 32\. Curran Associates, Inc.
  * Vershynin (2018) [author] Vershynin, RomanR. (2018). High-Dimensional Probability: An Introduction with Applications in Data Science 47\. Cambridge University Press.
  * Villani (2009) [author] Villani, CédricC. (2009). Displacement interpolation. In Optimal Transport: Old and New 113–162. Springer Berlin Heidelberg, Berlin, Heidelberg.
  * Wibisono and Yang (2022) [author] Wibisono, AndreA. and Yang, Kaylee YingxiK. Y. (2022). Convergence in KL divergence of the inexact Langevin algorithm with application to score-based generative models. arXiv preprint arXiv:2211.01512.
  * Yang, Yang and Xiang (2023) [author] Yang, YahongY., Yang, HaizhaoH. and Xiang, YangY. (2023). Nearly optimal VC-dimension and pseudo-dimension bounds for deep neural network derivatives. arXiv preprint arXiv:2305.08466.
  * Yarotsky (2017) [author] Yarotsky, DmitryD. (2017). Error bounds for approximations with deep ReLU networks. Neural Networks 94 103–114.
  * Yarotsky (2018) [author] Yarotsky, DmitryD. (2018). Optimal approximation of continuous functions by very deep ReLU networks. In Conference on Learning Theory 639–649. PMLR.