#  CoNFiLD: Conditional Neural Field Latent Diffusion Model Generating
Spatiotemporal Turbulence

Pan Du  Meet Hemant Parikh  Xiantao Fan  Xin-Yang Liu  Jian-Xun Wang
Department of Aerospace and Mechanical Engineering, University of Notre Dame,
Notre Dame, IN, USA  Lucy Family Institute for Data & Society, University of
Notre Dame, Notre Dame, IN, USA  Center for Sustainable Energy (ND Energy),
University of Notre Dame, Notre Dame, IN, USA<EMAIL_ADDRESS>
###### Abstract

This study introduces the Conditional Neural Field Latent Diffusion (CoNFiLD)
model, a novel generative learning framework designed for rapid simulation of
intricate spatiotemporal dynamics in chaotic and turbulent systems within
three-dimensional irregular domains. Traditional eddy-resolved numerical
simulations, despite offering detailed flow predictions, encounter significant
limitations due to their extensive computational demands, restricting their
applications in broader engineering contexts. In contrast, deep learning-based
surrogate models promise efficient, data-driven solutions. However, their
effectiveness is often compromised by a reliance on deterministic frameworks,
which fall short in accurately capturing the chaotic and stochastic nature of
turbulence. The CoNFiLD model addresses these challenges by synergistically
integrating conditional neural field encoding with latent diffusion processes,
enabling the memory-efficient and robust probabilistic generation of
spatiotemporal turbulence under varied conditions. Leveraging Bayesian
conditional sampling, the model can seamlessly adapt to a diverse range of
turbulence generation scenarios without the necessity for retraining, covering
applications from zero-shot full-field flow reconstruction using sparse sensor
measurements to super-resolution generation and spatiotemporal flow data
restoration. Comprehensive numerical experiments across a variety of
inhomogeneous, anisotropic turbulent flows with irregular geometries have been
conducted to evaluate the model’s versatility and efficacy, showcasing its
transformative potential in the domain of turbulence generation and the
broader modeling of spatiotemporal dynamics. ††Videos of all numerical
experiments can be found at https://sites.nd.edu/jianxun-wang/animations

###### keywords:

Generative AI , Surrogate Modeling , Chaotic Systems , Inverse Problems ,
Bayesian Learning , Data Assimilation

††journal: Elsevier

##  1 Introduction

Turbulent flows, characterized by their inherent chaotic and multiscale
nature, are a central subject in the study of fluid dynamics, essential for
understanding phenomena in diverse areas such as aerospace, oceanography, and
combustion. Traditionally, simulating these complex spatiotemporal behaviors
has relied on first-principle eddy-resolving methods like Direct Numerical
Simulation (DNS) and Large Eddy Simulation (LES), which require numerically
solving the governing partial differential equations (PDEs) for fluid flows.
While these methods offer detailed insights, their application is largely
limited by significant computational demands. The fine-scale spatiotemporal
resolution required by DNS and LES to accurately capture the wide range of
space and time scales in turbulence structure results in substantial
computational loads, making them impractical for most engineering
applications.

The rapid advancements in machine/deep learning (ML/DL) have profoundly
influenced computational fluid dynamics (CFD) [1, 2], bringing a fresh and
innovative dimension to the field, marked by recent developments such as
advanced DL-based discretization [3, 4], data-driven closure modeling [5, 6,
7], accelerated CFD solving processes [8], and the differentiable hybrid
neural modeling, a framework that unifies conventional CFD and DL through
differentiable programming [9, 10]. Moreover, DL has become instrumental in
developing rapid surrogate or reduced-order models, offering efficient
alternatives to computationally-intensive numerical solvers for emulating
complex spatiotemporal dynamics. These models, often built on autoregressive
learning architectures, are adept at predicting future flow states based on
previous conditions, relying on temporal correlations learned from training
data. An important aspect of these models is the integration of dimensionality
reduction techniques, such as Proper Orthogonal Decomposition (POD) and
Convolutional Neural Network (CNN) autoencoders, with sequence neural
networks, e.g., Long-short Term Memory (LSTM) and transformers. Notable
examples include the convolutional autoencoder-based autoregressive learning
models by Fukami and co-workers for inflow turbulence synthesis and super-
resolution [11, 12, 13], and the work of Yousif et al. [14], who combined CNN
autoencoders with LSTM networks, further advancing these models with
adversarial training and attention mechanisms [15]. To effectively handle
unstructured flow data within irregular domains, Graph Neural Networks
(GNN)-based autoencoder coupled with temporal attention models has been
proposed and shown the effectiveness [16]. Despite the promise, challenges
remain, particularly in the turbulence regime. The deterministic nature of
these ML-based surrogate models often inadequately captures the stochastic
behavior inherent in turbulent flows. These models, largely relying on
autoregressive architectures, are able to learn the complex distribution of
turbulence, limiting their capacity to produce stochastic flow realizations.
This can result in substantial deviations in long-term predictions, as the
chaotic nature of turbulence magnifies the impact of even minor inaccuracies
or perturbations. Furthermore, there is a risk of cumulative error propagation
in these models, potentially undermining the robustness and reliability of
their long-term forecasting capabilities.

Generative AI, rooted in probabilistic learning and statistical inference,
offers a promising direction to overcome these limitations. These models are
capable of learning the complex probabilistic distributions within datasets,
allowing for the generation of new data samples that statistically resemble
the training sets. In the context of turbulence simulation, generative models
are particularly valuable as they can capture the multi-scale and stochastic
characteristics of turbulence, thereby enabling the synthesis of instantaneous
flow field realizations that align with the statistical characteristics
observed in real-world turbulent data. The recent surge in deep generative
models for turbulence, primarily driven by Generative Adversarial Networks
(GANs), underscores their potential and promise. GANs operate through a
dynamic interplay between a generator, which creates synthetic turbulent data,
and a discriminator, which distinguishes between synthetic and real data. This
iterative adversarial process refines the generator’s output, aiming for
convergence to the actual data distribution. Variants like Wasserstein GAN
(WGAN), conditional GAN (cGAN), deep convolutional GAN (DCGAN), super-
resolution GAN (SRGAN), and cycle-consistent GAN (CycGAN) have been adapted
for specific tasks, such as turbulence generation [17, 18], super-resolution
[19, 20, 21, 22], and data inpainting [23]. However, the primary limitation of
these models is their focus on single-snapshot generation, as they are trained
on isolated flow snapshots without temporal coherence, which restricts their
ability to synthesize spatiotemporal turbulence. Attempts to integrate GANs
with sequential networks have been made, but these often result in GANs acting
as deterministic encoders, not fully exploiting their stochastic generation
capabilities [15]. Only a few studies, such as TempoGAN by Xie et al. [24] and
WGAN-RNN model by Kim and Lee [25], have leveraged GANs for stochastic
turbulence generation. While GANs have shown potential for turbulence
synthesis, they often face significant challenges: their training is
notoriously challenging due to the oscillatory behavior between the generator
and discriminator components [26]. Additionally, they are susceptible to “mode
collapse”, a limitation that results in a reduced diversity of output in the
generated simulations [27]. These factors critically impede their efficacy in
accurately modeling complex turbulent dynamics. In addition to GANs,
normalizing flows (NFlows) have also been explored for turbulence generation.
These models stand out for their ability to directly model complex data
distributions through a series of invertible, differentiable transformations,
which is particularly useful in emulating intricate dynamics like turbulence.
Geneva and Zabaras [28] have utilized NFlows for super-resolving Very Large
Eddy Simulation (VLES) data, and Sun et al. [29] developed a sequential NFlows
model integrating GNN-autoencoding and attention mechanism to synthesize
instantaneous backward-facing-step flows. However, NFlows have the known
scalability issue due the complexity of computing Jacobians in
transformations, making them infeasible to handle real-world 3D turbulence
data.

Diffusion models have recently advanced the field of generative modeling,
outperforming GANs and NFlows in a variety of computer vision tasks [30, 31,
32]. These models are uniquely characterized by their progressive approach of
transforming data from a simple distribution into a complex one. This is
achieved by initially introducing noise into the dataset and then
systematically denoising it through deep neural networks (DNNs). There are two
primary categories of diffusion models: denoising diffusion probabilistic
models (DDPMs) [33] and score-based diffusion models [34], both of which can
be unified within the stochastic differential Equation (SDE)-based framework
[35]. The advantages of diffusion models are manifold, including the ease of
training, the capability of capturing multi-scale features, and their
proficiency in conditional generation, particularly within a Bayesian
framework. While diffusion models have recently shown significant success in
fields like image generation and super-resolution [30, 36], their application
in turbulence simulation represents an emerging and largely uncharted domain.
Recent studies have explored the use of DDPMs in super-resolution or
inpainting of turbulence data [37, 38, 39]. However, these initial works
mainly focused on single-snapshot generations, typically in 2D Komogrov flows,
homogenous and isotropic in nature. Most recently, Gao et al. [40] have taken
a leap forward with the development of a Bayesian conditional diffusion model
for spatiotemporal turbulence generation. This model has showcased its
capability to stochastically generate the temporal evolution of complex, wall-
bounded turbulence in a variety of conditions, including URANS super-fidelity,
auto-regressive generation, and super-resolution generation. However, the
foundational architecture of this model, VideoDiffusion [41], utilizes 3D
convolution in physical spatiotemporal space, encountering scalability and
efficiency challenges. This limitation confines its application to small-scale
2D spatial fields with a limited temporal extent of the generated segments.
Furthermore, the model’s backbone architecture, a CNN-based 3D U-Net,
inherently requires regular domains with uniform grids, posing a limitation in
handling complex, irregular geometries with unstructured grids, which are
prevalent in CFD, thereby restricting its adaptability to a broader range of
real-world turbulence simulation scenarios.

In this work, we proposed a conditional neural field latent diffusion
(CoNFiLD) model, innovatively designed for efficiently generating complex
spatiotemporal dynamics of chaotic and turbulent systems across diverse
conditions, addressing both regular and irregular geometrical configurations.
Distinct from the majority of existing literature that focuses on single-
snapshot (image) generation, CoNFiLD emphasizes capturing the probabilistic
distribution of time-evolving turbulent flow sequences, operating as a
sophisticated stochastic spatiotemporal process, allowing for an effective
generation of new instantaneous flow realizations through random sampling
under a variety of conditions. The proposed model synergistically integrates
conditional neural field (CNF) techniques with a latent probabilistic
diffusion model, enabling forward and reverse diffusion process being operated
in the CNF-encoded latent space. This innovative architecture leverages the
advantage and effectiveness of CNF for meshless nonlinear dimension reduction,
which have been demonstrated in recent literature [42, 43], ensuring robust
performance in diverse geometrical configurations and scalable applications.
By significantly improving the scalability and efficiency of both offline
training and online generation, this work overcomes the limitations of the
previous model: scalability constraints and uniform grid requirements [40].
Moreover, the proposed CoNFiLD is also featured for its zero-shot conditional
generation by leveraging the Bayesian formulation and differentiable
programming, thereby eliminating the need for retraining when adapting to new
flow conditions. The generative learning capability of our CoNFiLD model has
been showcased through its application to a wide range of real-world 3D
turbulent flow cases, including scenarios with wall-bounded turbulence, flow
separations, and intricate 3D geometries. Remarkably versatile, the
unconditionally trained CoNFiLD model can be directly applied for conditional
generation tasks without the need of re-training. These applications span from
reconstructing full-field spatiotemporal flows from sparse sensor data to
generating super-resolution spatiotemporal flows and restoring corrupted flow
data. This work represents a significant contribution to the field of
spatiotemporal generative modeling and turbulence simulation, offering a
comprehensive and efficient solution for generating realistic, complex
instantaneous turbulent flows in various scenarios. To the best of the
authors’ knowledge, this study represents the first development of a neural
field encoded latent diffusion model for the 4D generation of spatiotemporal
dynamics in chaotic and turbulent systems with complex, irregular domains.

The remainder of this paper is structured as follows: Section 2 elaborates on
the proposed CoNFiLD framework and its mathematical details. Section 3
provides a comprehensive set of numerical experiments to evaluate and
demonstrate CoNFiLD’s generative capabilities across various wall-bounded
turbulence scenarios. The computational efficiency, memory usage, and
scalability comparisons of CoNFiLD with other methodologies are discussed in
Section 4. The paper is summarized in Section 5, outlining the main findings
and contributions.

##  2 Methodology

###  2.1 Overview of CoNFiLD generative learning framework

Turbulent flows, inherently chaotic and stochastic across various spatial and
temporal scales, fundamentally represent stochastic spatiotemporal processes.
The work aims to construct a data-driven model capable of generating unsteady
instantaneous turbulent flows. This is achieved through generative AI
techniques designed to learn the underlying probability distribution
$p\big{(}\bm{\Phi}(\bm{x},t)\big{)}$ of the spatiotemporal turbulent flow
fields $\bm{\Phi}(\bm{x},t)$ from instantaneous flow datasets
$\bm{\Phi}(\bm{x},t)\in\mathcal{A}_{train}$. To this end, we present the
Conditional Neural Field Latent Diffusion (CoNFiLD) model, an innovative
generative learning framework that leverages neural implicit representations
to facilitate efficient and scalable diffusion-based generation within a
compact latent space. As depicted in Fig. 1, our CoNFiLD features a unique
combination of Conditional Neural Field (CNF) and Latent Diffusion Models
(LDM), distinguishing it from prior work that applied diffusion processes
directly in the high-dimensional physical domain, which encounters significant
computational hurdles and memory constraints [40]. Specifically, the CoNFiLD
model is constructed in three stages.

Figure 1: Overview of the proposed Conditional Neural Field Latent Diffusion
(CoNFiLD) model. (a) Architectures: a FiLM-based CNF for encoding dynamic flow
sequences into latent space, where the underlying distribution of the latent
vectors is implicitly captured by learning reverse diffusion (denoising)
processes. (b) Zero-shot generation: synthesizing new spatiotemporal flow
fields with arbitrary length, either unconditionally or based on specific
conditions (e.g., sparse sensor data), without the need for retraining.

First, a CNF, $\mathscr{E}_{\zeta,\gamma}(\mathbf{x},\bm{L})$, is designed to
encode a time sequence of instantaneous flow fields, discretized as
$\Phi(\bm{x},t)\in\mathbb{R}^{N_{m}\times N_{t}}$, into a time sequence of
latents $\bm{z}_{0}\in\mathbb{R}^{N_{l}\times N_{t}}$, where $\zeta,\gamma$
are trainable parameters of the CNF encoder, and $N_{m},N_{l},N_{t}$ represent
the dimensions of spatial space, latent space, and time length, respectively.
Once trained, the CNF forms a neural implicit representation of the
spatiotemporal flow field conditioned on the latent vector
$\bm{L}=\bm{z}_{0}$, i.e.,
$\bm{\Phi}(\bm{x},t)\approx\mathscr{E}_{\zeta^{*},\gamma^{*}}(\mathbf{x},\bm{L})$.
Unlike conventional encoders, the CNF encoder here is formulated in an auto-
decoding fashion [44], where the latents $\bm{L}$ are optimized by minimizing
the mismatch between the field of interest values and corresponding CNF
outputs,

$\displaystyle\bm{z}_{0}({:},t_{i})$
$\displaystyle=\mathrm{Encode}\Big{(}\bm{\Phi}({:},t_{i}),\mathscr{E}_{\zeta^{*},\gamma^{*}}\Big{)}$
(1)
$\displaystyle=\arg\min_{\bm{L}}\sum_{j}^{N_{m}}\Big{\|}\bm{\Phi}(t_{i},\mathbf{x}_{j})-\mathscr{E}\left(\bm{L}(i),\mathbf{x}_{j};\zeta,\gamma\right)\Big{\|}_{L_{2}},\;\
\forall i\in[1,2,\dots,N_{t}]$

In practice, all the snapshots
$\\{\bm{\Phi}({:},t_{i})\\},i\in[1,\dots,N_{t}]$ are encoded into latent
vectors $\\{\bm{z}_{0}({:},t_{i})\\}_{1}^{N_{t}}$ simultaneously, forming a
latent-time snapshot $\bm{z}_{0}$ as a 2D “image”.

Following the encoding phase, a probabilistic diffusion module is introduced
to implicitly learn the underlying probability distribution $p(\bm{z}_{0})$ of
the latent dynamics $\bm{z}_{0}$ through bi-direction diffusion processes.
Initially, the latent samples undergo a forward Markovian diffusion process,
characterized by a series of carefully designed white noise additions that
incrementally nudge the latent representations towards the fully perturbed
state with an isotropic Gaussian distribution. Subsequently, by learning the
reverse diffusion process (i.e., denoising process) through neural networks,
the model is capable of generating new latent samples $\bm{z}_{0}$ from
randomly sampled white noises using the learned denoising scheme.

Finally, the newly generated latents $\bm{z}_{0}\in\mathcal{L}_{test}$ is fed
into the trained CNF to decode them back to the physical space for obtaining
the synthesized spatiotemporal flow fields
$\bm{\Phi}(\bm{x},t)\in\mathcal{A}_{test}$ (see Fig. 1),

$\displaystyle\bm{\Phi}(t_{i},{:})$
$\displaystyle=\mathrm{Decode}\Big{(}\bm{z}_{0}({:},t_{i}),\mathscr{E}_{\zeta^{*},\gamma^{*}}\Big{)}$
(2)
$\displaystyle=\mathscr{E}\Big{(}\bm{x},\bm{z}_{0}({:},t_{i}),;\zeta^{*},\gamma^{*}\Big{)},\;\;\;\;\;\forall
i\in[1,2,\dots,N_{t}]$

The much higher data compression ratios of the CNF-based encoder, compared to
other encoding methods, allow the generative model to operate in a
significantly reduced-dimensional latent space, which addresses the
computational challenges in synthesizing large-scale, high-dimensional
spatiotemporal turbulence data. Moreover, the CNF’s ability to process
arbitrary point queries significantly enhances the model’s versatility in
managing irregular domains and adaptive meshes. The training of the CoNFiLD
model unfolds in a decoupled two-step strategy: firstly, the CNF encoder is
trained to transform spatiotemporal flow fields into latent representations;
this is followed by the diffusion model being trained on these
representations. This dual-phase training strategy facilitates efficient
utilization of the latent space and enables robust model optimization and
inference.

Upon completing its training, the CoNFiLD model can rapidly generate new 4D
spatiotemporal flow samples, $\bm{\Phi}\in\mathbb{R}^{N_{m}\times N_{T}}$.
Notably, the length $N_{T}$ of generated time sequences can significantly
exceed the length $N_{t}$ of those used during training ($N_{T}>N_{t}$). This
novel feature stems from the shift-invariance of convolution kernels learned
in the latent space, where the latent diffusion model is capable of
synthesizing an arbitrarily extended sequence of latent vectors, which are
subsequently decoded into high-resolution 4D flow fields via the trained CNF
decoder. Another distinctive feature of CoNFiLD is its _zero-shot_ conditioned
generation capability, which enables the creation of 4D flow realizations
under specific conditions (e.g., sparse sensor measurements, low-resolution
data) _without_ the need for retraining the model. Unlike traditional
conditional generative methods, which require conditionally paired training
data and necessitate retraining for new conditions, CoNFiLD’s diffusion
process is trained unconditionally only once and can then generate samples
under a variety of conditions during inference. As shown in Fig. 1, this novel
feature holds significant practical value, finding applications in a range of
inverse problems, such as spatiotemporal super-resolution of flow data (super-
resolution), full-field reconstruction of instantaneous flow fields from
sparse sensor measurements (flow reconstruction), and restoring missing
information in damaged flow data (data restoration). The major highlights of
our method include: (1) CNF-based encoding with a high compression ratio,
facilitating efficient diffusion processes within the latent space; (2) the
ability of CNF to process arbitrary pointwise queries, enhancing adaptability
to irregular domains and enabling support for unstructured data and adaptive
meshes; (3) a Bayesian conditioning sampling mechanism that allows for
versatile conditional generation without the necessity for retraining; (4)
significant reduction of memory usage in subsampling-based conditional
generation scenarios.

###  2.2 Conditional neural field encoding

Neural fields (NF) have emerged as state-of-the-art in learning implicit
representations of coordinate-based functional fields, demonstrating
exceptional performance in modeling and compressing complex signals such as
images [45, 46], videos [47, 48], 3D scenes [49], and 3D shapes [44, 50].
Despite their tremendous success in computer vision, the exploration of NFs in
dimension reduction of large-scale spatiotemporal data of physical systems
(e.g., turbulence data) remains sparse. A recent effort in this direction by
Pan et al. [42] employs a neural field fully conditioned by hyper-network for
spatiotemporal dimension reduction, where the parameters of the NF are fully
determined by the hyper-network. This architecture, while effective,
necessitates explicit conditions (e.g., sensor signals, time stamps) and
considerable memory overhead for the hyper-network, often surpassing that of
the NF itself. In contrast, the CNF of our CoNFiLD utilizes a more flexible
and robust conditioning mechanism, the Feature-wise Linear Modulation (FiLM)
[51], and the encoder is formulated in an auto-decoding manner. The primary NF
leverages the SIREN network architecture [45], renowned for its capacity to
capture domains with rich periodic features through sinusoidal activation
functions. The mathematical representation is as follows,

$\begin{gathered}\mathrm{SIREN}(\bm{x})=\bm{W}_{p}\left(\eta_{p-1}\circ\eta_{p-2}\circ\dots\circ\eta_{1}\right)\left(\omega_{0}\bm{W}_{0}\bm{x}+\bm{B}_{0}\right)+\bm{B}_{p},\\\
\eta_{i}(\bm{o}_{i-1})=\sin(\bm{W}_{i}\bm{o}_{i-1}+\bm{B}_{i}),\;\;\;i\in[1,2,\dots,p],\end{gathered}$
(3)

where $\\{\bm{W}_{i}\\}^{p}_{0}$ and $\\{\bm{B}_{i}\\}^{p}_{0}$ are trainable
parameters of NF (collectively denoted as $\zeta$), $\bm{o}_{i-1}$ is the
$i-1^{\mathrm{th}}$ layer output, and $\omega_{0}$ is a hyperparameter that
modulates the initial input signal frequency. SIREN requires a unique
initialization, $w_{i}\sim\mathcal{U}(-r/\sqrt{n},r/\sqrt{n}),r\in\mathbb{R}$,
ensure outputs across layers follow a standard normal distribution, where
$w_{i}$ is individual entry of weight matrices and $\mathcal{U}(\cdot)$
represents a uniform distribution with lower/upper bound hyperparameters $r$.
Typically, $\omega_{0}=30$ and $r=\sqrt{6}$ are chosen for robust performance.

The SIREN is modulated by FiLM conditioning, where latent vectors $\bm{L}$ are
passed into multiple linear transformation layers. Thus, the CNF, denoted by
$\mathscr{E}$, is described as follows,

$\begin{gathered}\mathscr{E}(\bm{x},\bm{L})=\bm{W}_{p}\left(\eta^{\prime}_{p-1}\circ\eta^{\prime}_{p-2}\circ\dots\circ\eta^{\prime}_{1}\right)\left(\omega_{0}\bm{W}_{0}\bm{x}+\bm{B}_{0}+\bm{c}_{0}\right)+\bm{B}_{p},\\\
\eta^{\prime}_{i}(\bm{o}_{i-1},\bm{c}_{i})=\sin(\bm{W}_{i}\bm{o}_{i-1}+\mathbf{B}_{i}+\bm{c}_{i}),\;\;\;i\in[1,2,\dots,p],\\\
\bm{c}_{i}(\bm{L})=\bm{W}^{cond}_{i}\bm{L}+\mathbf{B}^{cond}_{i},\end{gathered}$
(4)

where $\\{\bm{W}^{cond}_{i}\\}^{p}_{1}$ and $\\{\bm{B}^{cond}_{i}\\}^{p}_{1}$
(collectively denoted as $\gamma$) are trainable parameters for the FiLM
conditioning layers, which introduce a bias adjustment $\bm{c}_{i}$ on each
SIREN layer before the activation function is applied and $\eta^{\prime}_{i}$
represents the $i^{\mathrm{th}}$ FiLM-modified intermediate layer. This
implicit model represents a continuous function across the spatial domain
$\Omega$, usually being trained on discretized datasets. Within our CoNFiLD
framework, the CNF functions dually as an encoder and decoder, enabling
seamless data transform between physical and latent space. The encoding
strategy was meticulously designed to ensure that the SIREN exclusively models
the spatial field, whereas the conditional information is encapsulated within
a latent vector corresponding to each spatial field frame at discrete time
steps $t_{i}$. Consequently, the latent encoding of a spatiotemporal field
segment is structured as a 2D “image”, effectively preserving the original
time dimension. This configuration enables unrestricted kernel convolution
across images of any temporal length, introducing an innovative approach for
time forecasting in the generation process, further detailed in Section 2.3.

The CNF is trained on the dataset $\bm{\Phi}\in\mathcal{A}_{train}$ by solving
the following optimization,

$\displaystyle\bm{L}^{*},\zeta^{*},\gamma^{*}=\arg\min_{L,\zeta,\gamma}\sum_{i}^{N_{t}}\sum_{j}^{N_{m}}\Big{\|}\bm{\Phi}(\bm{x}_{j},t_{i})-\mathscr{E}\Big{(}\bm{L}(i),\bm{x}_{j};\zeta,\gamma\Big{)}\Big{\|}_{L_{2}},$
(5)

where the optimized latent vectors $\bm{L}^{*}$, representing the latents
$\bm{z}_{0}$ of the spatiotemporal flow field $\bm{\Phi}$, are obtained. This
optimization employs an alternating-direction strategy, which involves
updating the CNF parameters ($\zeta,\gamma$) and the latent vectors ($\bm{L}$)
in turns. Specifically, the latent vectors are updated per batch with the CNF
parameters frozen; subsequently, $\zeta$ and $\gamma$ are updated while the
latent vectors remain temporarily fixed. This alternating-direction
optimization approach has been empirically shown to foster a stable and robust
convergence during the training process [52]. After training, encoding a new
spatiotemporal field necessitates an optimization, as described by Eq. 1,
whereas the decoding phase simply involves feeding the latent vectors and
spatial coordinates into the trained CNF to retrieve the corresponding Field
of Interest (FOI) values. The efficiency of the CNF decoder becomes
particularly evident during the latent diffusion-based generation phase.

###  2.3 Latent probabilistic diffusion modeling

Given a CNF-encoded latent state $\bm{z}_{0}$ whose underlying distribution is
$p(\bm{z}_{0})$, i.e., $\bm{z}_{0}\sim p(\bm{z}_{0})$, a _forward diffusion
process_ is defined by progressively perturbing $\bm{z}_{0}$ with Gaussian
noise of variance $\beta_{\tau}$, through the transition kernel,

$p(\bm{z}_{\tau}|\bm{z}_{\tau-1})=\mathcal{N}(\bm{z}_{\tau};\sqrt{1-\beta_{\tau}}\bm{z}_{\tau-1},\beta_{\tau}\bm{I}),$
(6)

where $\tau=1,\cdots N_{\tau}$ denotes the diffusion step index with
$N_{\tau}$ as the total number of steps, and
$\bm{I}\in\mathbb{R}^{(N_{l}\times N_{t})^{2}}$ is the identity matrix. This
forward diffusion process yields a sequence of incrementally noised latent
states $\bm{z}_{1},\cdots,\bm{z}_{N_{\tau}}$ with the joint probability
density $p(\bm{z}_{1},\cdots,\bm{z}_{N_{\tau}}|\bm{z}_{0})$,

$p(\bm{z}_{1},\cdots,\bm{z}_{N_{\tau}}|\bm{z}_{0})=\prod_{\tau=1}^{N_{\tau}}q(\bm{z}_{\tau}|\bm{z}_{\tau-1}).$
(7)

With sufficient perturbation steps, the marginalized distribution
$p(\bm{z}_{N_{\tau}}|\bm{z}_{0})$ asymptotically converges to an isotropic
Gaussian distribution, denoted by
$p(\bm{z}_{\tau})=\mathcal{N}(\bm{0},\sigma^{2}_{N_{\tau}|0}\bm{I})$,
facilitating straightforward sampling. Using the re-parameterization trick
[33], the conditional distribution of each noised latent state given
$\bm{z}_{0}$ is also Gaussian, explicitly defined as,

$p(\bm{z}_{\tau}|\bm{z}_{0})=\mathcal{N}(\bm{z}_{\tau},\sqrt{\bar{\alpha}_{\tau}}\bm{x}_{0},(1-\bar{\alpha}_{\tau})\bm{I})),$
(8)

where $\alpha_{\tau}=1-\beta_{\tau}$ and
$\bar{\alpha}_{\tau}=\prod_{s=1}^{\tau}\alpha_{s}$. The forward diffusion is
characterized through a predetermined series of variance parameters
$\beta_{\tau}$, known as variance schedule, which can adopt various forms,
such as linear, quadratic, or cosine schedules [32].

Upon establishing the forward diffusion process, its reversal (i.e., _reverse
diffusion process_) becomes particularly compelling since it enables the
synthesis of new latent samples of $\bm{z}_{0}$ from white noise vectors
sampled from the isotropic Gaussian distribution $p(\bm{z}_{N_{\tau}})$. This
process relies on the assumption that each step’s perturbation is sufficiently
small, ensuring that the conditional probability
$p(\bm{z}_{\tau-1}|\bm{z}_{\tau})$, or the reverse transition kernel, remains
Gaussian. However, directly computing this reverse transition kernel is
infeasible, as it requires the knowledge of the distribution of the entire
latent space, which is unknown _a priori_. To overcome this challenge, all
existing diffusion-based generation methods rely on neural networks to learn
either the reverse transition kernel or the score function with trainable
parameters $\bm{\theta}$, both of which can be unified in the same framework
[34, 33, 35]. Training the neural network parameterization can be formulated
as a likelihood maximization problem,

$\min_{\bm{\theta}}\sum_{\mathscr{E}_{\zeta^{*},\gamma^{*}}(\bm{z}_{0})\in\mathcal{A}_{train}}{-\log
p_{\bm{\theta}}(\bm{z}_{0})}.$ (9)

However, as $\log p_{\bm{\theta}}(\bm{z}_{0})$ is not tractable, we can
instead minimize the variational bound $L_{\mathrm{VB}}$ on the negative log
likelihood (derivation can be found in [53]),

$\displaystyle\mathbb{E}\big{[}-\log p_{\bm{\theta}}(\bm{z}_{0})\big{]}\leq
L_{\mathrm{VB}}=$
$\displaystyle\mathbb{E}_{p}\bigg{[}D_{KL}\Big{(}p(\bm{z}_{N_{\tau}}|\bm{z}_{0})\big{\|}p(\bm{z}_{N_{\tau}})\Big{)}$
(10) $\displaystyle+$
$\displaystyle\sum_{\tau=2}^{N_{\tau}}D_{KL}\Big{(}p(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})\big{\|}p_{\bm{\theta}}(\bm{z}_{\tau-1}|\bm{z}_{\tau})\Big{)}-\log
p_{\bm{\theta}}(\bm{z}_{0}|\bm{z}_{1})\bigg{]},$

where $D_{KL}(\cdot\|\cdot)$ represents Kullback-Leibler (KL) divergence
operator. Since the first term of $L_{\mathrm{VB}}$ contains no trainable
parameters $\bm{\theta}$ and remains a constant during training, it can be
dropped. Therefore, the diffusion model can be trained by minimizing
$\tilde{L}_{\mathrm{VB}}$,

$\tilde{L}_{\mathrm{VB}}=\mathbb{E}_{p}\bigg{[}\sum_{\tau=2}^{N_{\tau}}D_{KL}\Big{(}p(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})\big{\|}p_{\bm{\theta}}(\bm{z}_{\tau-1}|\bm{z}_{\tau})\Big{)}-\log
p_{\bm{\theta}}(\bm{z}_{0}|\bm{z}_{1})\bigg{]}.$ (11)

The reverse transition kernel is Gaussian,
$p_{\bm{\theta}}(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})=\mathcal{N}\Big{(}\bm{z}_{\tau-1};\bm{\mu}_{\bm{\theta}}(\bm{z}_{\tau},t),\bm{\Sigma}_{\bm{\theta}}(\bm{z}_{\tau},t)\Big{)}$,
where the mean and covariance functions are parameterized by neural networks.
Specifically, the parameterization is designed based on the reverse
conditional probability $p(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})$, which
is analytically tractable when conditioned on $\bm{z}_{0}$,

$p(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})=\mathcal{N}\bigg{(}\bm{z}_{\tau-1};\frac{1}{\sqrt{\alpha_{\tau}}}\left(\bm{x}_{\tau}-\frac{1-\alpha_{\tau}}{\sqrt{1-\bar{\alpha}_{\tau}}}\bm{\epsilon}_{\tau}\right),\frac{1-\bar{\alpha}_{\tau-1}}{1-\bar{\alpha}_{\tau}}\beta_{\tau}\bm{I}\bigg{)}$
(12)

where the $\bm{\epsilon}_{\tau}$ is the noise added at step $\tau$. Therefore,
the mean function $\bm{\mu}_{\bm{\theta}}(\bm{z}_{\tau},\tau)$ of reverse
kernel is parameterized as follows,

$\bm{\mu}_{\bm{\theta}}(\bm{z}_{\tau},\tau)=\frac{1}{\sqrt{\alpha_{\tau}}}\left(\bm{x}_{\tau}-\frac{1-\alpha_{\tau}}{\sqrt{1-\bar{\alpha}_{\tau}}}\bm{\epsilon}_{\bm{\theta}}(\bm{z}_{\tau},\tau;\bm{\theta})\right)$
(13)

where the noise function
$\bm{\epsilon}_{\bm{\theta}}(\bm{z}_{\tau},\tau;\bm{\theta})$ is approximated
by a U-Net variant with residual blocks, self-attention blocks, and group
normalization [54, 30]. The variance function $\bm{\Sigma}_{\theta}$ remains
fixed as
$\bm{\Sigma}_{\theta}=(1-\bar{\alpha}_{\tau-1})\beta_{\tau}/(1-\bar{\alpha}_{\tau})\bm{I}$
based on Eq. 12. Since the KL divergence between two Gaussian distributions
has a closed form, $\tilde{L}_{\mathrm{VB}}$ in Eq. 11 can be expressed as,

$\tilde{L}_{\mathrm{VB}}=\mathbb{E}_{\bm{z}_{0},\bm{\epsilon}}\bigg{[}\frac{(1-\alpha_{\tau})^{2}}{2\alpha_{\tau}(1-\bar{\alpha}_{\tau})\|\bm{\Sigma}_{\bm{\theta}}\|^{2}_{L_{2}}}\Big{\|}\bm{\epsilon}_{\tau}-\bm{\epsilon}_{\bm{\theta}}\big{(}\sqrt{\bar{\alpha}_{t}}\bm{z}_{0}+\sqrt{1-\bar{\alpha}_{t}}\bm{\epsilon}_{\tau},\tau\big{)}\Big{\|}^{2}_{L_{2}}\bigg{]}.$
(14)

Ho et al. [33] further simplified the VB loss by ignoring the weighting term,

$\tilde{L}_{\mathrm{simple}}=\mathbb{E}_{\bm{z}_{0},\bm{\epsilon}}\bigg{[}\Big{\|}\bm{\epsilon}_{\tau}-\bm{\epsilon}_{\bm{\theta}}\big{(}\sqrt{\bar{\alpha}_{t}}\bm{z}_{0}+\sqrt{1-\bar{\alpha}_{t}}\bm{\epsilon}_{\tau},\tau\big{)}\Big{\|}^{2}_{L_{2}}\bigg{]}.$
(15)

In this work, the latent diffusion model training adopts a hybrid form
suggested by Nichol and Dhariwal [54], which uses both
$\tilde{L}_{\mathrm{simple}}$ and $\tilde{L}_{\mathrm{vb}}$ with a weight
parameter $\lambda$, leading to the following optimization,

$\bm{\theta}^{*}=\arg\min_{\bm{\theta}}\sum_{\mathscr{E}_{\zeta^{*},\gamma^{*}}(\bm{z}_{0})\in\mathcal{A}_{train}}\Big{[}\tilde{L}_{\mathrm{simple}}+\lambda\tilde{L}_{\mathrm{VB}}\Big{]},$
(16)

where noise vector $\bm{\epsilon}$ is randomly sampled from a standard
Gaussian distribution, i.e., $\bm{\epsilon}\sim\mathcal{N}(\bm{0},\bm{I})$,
and $\tau$ is randomly sampled from a discrete uniform distribution
$\mathcal{U}(1,N_{\tau})$.

###  2.4 Bayesian conditional generation of spatiotemporal fields

Upon completing the training phase, the CoNFiLD model can rapidly generate new
4D spatiotemporal flow fields $\bm{\Phi}\in\mathbb{R}^{N_{m}\times N_{T}}$ by
sampling from the latent diffusion model and performing the CNF decoding.
Remarkably, the model enables the generation of time sequences of flow fields
that far exceed the temporal scope of the training data, i.e., $N_{T}>N_{t}$.
This extended generative capacity allows the CoNFiLD to not only synthesize
new spatiotemporal flow data possessing turbulence statistics consistent with
the training data, but also to extrapolate temporally well beyond the time
sequence length of the training set. More importantly, a particularly notable
feature of the CoNFiLD model is its adeptness at _zero-shot_ conditional
generation within a Bayesian sampling framework. This capability allows for
the spatiotemporal generation under various conditions, such as specific
initial states, sparse observations, or low-fidelity simulations, all without
necessitating retraining for each unique scenario. This aspect highlights the
CoNFiLD’s versatility and adaptability, offering tailored turbulence
predictions based on available data or desired outcomes.

####  2.4.1 Unconditional generation

The trained CoNFiLD model can be utilized to generate new spatiotemporal
turbulent flow sequences by sampling the learned distribution via the reverse
diffusion process. Starting from samples of a multivariate isotropic Gaussian
distribution, $\bm{z}_{N_{\tau}}\sim\mathcal{N}(\bm{0},\bm{I})$, the noises
are progressively removed using the learned reverse transition kernel
$p_{\bm{\theta}^{*}}(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})$,

$p_{\bm{\theta}^{*}}(\bm{z}_{\tau-1}|\bm{z}_{\tau},\bm{z}_{0})=\mathcal{N}\bigg{(}\bm{z}_{\tau-1};\frac{1}{\sqrt{\alpha_{\tau}}}\left(\bm{z}_{\tau}-\frac{1-\alpha_{\tau}}{\sqrt{1-\bar{\alpha}_{\tau}}}\bm{\epsilon}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})\right),\frac{1-\bar{\alpha}_{\tau-1}}{1-\bar{\alpha}_{\tau}}\beta_{\tau}\bm{I}\bigg{)},$
(17)

allowing for the sampling of $\bm{z}_{\tau-1}$ based on the sample
$\bm{z}_{\tau}$ at step $\tau$. Namely, the following reverse sampling is
conducted,

$\bm{z}_{\tau-1}=\frac{1}{\sqrt{\alpha_{\tau}}}\left(\bm{z}_{\tau}-\frac{1-\alpha_{\tau}}{\sqrt{1-\bar{\alpha}_{\tau}}}\bm{\epsilon}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})\right)+\sigma_{\tau}\bm{\epsilon},\
\ \bm{\epsilon}\sim\mathcal{N}(\bm{0},\bm{I}).$ (18)

By iteratively applying Eq. 18, a new latent $\bm{z}_{0}$ starting from white
noise $\bm{z}_{N_{\tau}}$ can be obtained, which is then decoded back to a
spatiotemporal turbulent flow field
$\bm{\Phi}=\mathcal{E}_{\zeta^{*},\gamma^{*}}(\bm{x},\bm{z}_{0})$ via the CNF
decoder. This process aligns with the score-based generative modeling
framework [34], where Eq. 18 can be expressed in terms of the score function,

$\bm{z}_{\tau-1}=\frac{1}{\sqrt{\alpha_{\tau}}}\Big{(}\bm{z}_{\tau}+(1-\alpha_{\tau})\bm{s}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})\Big{)}+\sigma_{\tau}\bm{\epsilon},\
\ \bm{\epsilon}\sim\mathcal{N}(\bm{0},\bm{I}).$ (19)

where the Stein score function
$\bm{s}(\bm{z}_{\tau},\tau)=\nabla_{\bm{z}_{t}}\log p(\bm{z}_{t})$ is
approximated by neural network parameterization
$\bm{s}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})$ with optimized
parameters $\bm{\theta}^{*}$.

####  2.4.2 Conditional generation

Mathematically, conditions can be systematically represented by a vector
$\bm{\Psi}\in\mathbb{R}^{N_{\bm{\Psi}}}$, hereafter referred to as the
condition vector. This vector $\bm{\Psi}$ may represent low-fidelity
solutions, instantaneous flow measurements from sparse sensor array, low-
resolution observation data, or other related information regarding to the
spatiotemporal flow field of interest $\bm{\Phi}$. From Bayesian perspective,
conditional generation involves the sampling of the conditional probability
$p(\bm{\Phi}|\bm{\Psi})$, which can be bridged to the unconditioned density
$p(\bm{\Phi})$ via Bayes’ rule,

$p(\bm{\Phi}|\bm{\Psi})\propto p(\bm{\Psi}|\bm{\Phi})p(\bm{\Phi}),$ (20)

where $p(\bm{\Phi})$ is learned by the CoNFiLD after the unconditional
training,

$p(\bm{\Phi})\approx
p^{\mathrm{CoNFiLD}}(\bm{\Phi};\bm{\theta}^{*},\zeta^{*},\gamma^{*}).$ (21)

The relationship between the condition $\bm{\Psi}$ and corresponding flow
sequences $\bm{\Phi}$ is defined as follows,

$\bm{\Psi}=\mathcal{F}(\bm{\Phi})+\bm{\epsilon}_{c},$ (22)

where $\mathcal{F}:\mathbb{R}^{N_{m}\times
N_{T}}\to\mathbb{R}^{N_{\bm{\Psi}}}$ is a nonlinear mapping from the
spatiotemporal turbulent field $\bm{\Phi}$ to its associated condition vector
$\bm{\Psi}$ (e.g., partial observation); $\bm{\epsilon}_{c}$ represents the
uncertainty inherent in the state-to-condition mapping, typically modeled as a
zero-mean Gaussian random variable. Conditional generation can be
conceptualized as a Bayesian inverse problem, which has attracted significant
interest within the computer vision community, notably in the area of image
restoration [55]. In these contexts, the state-to-condition mapping often
exhibits a linear nature. The Diffusion Posterior Sampling (DPS) technique,
which accommodates both linear and nonlinear mappings within the pixel space,
represents a recent advancement in this area, albeit operating within the
pixel space [56]. Expanding upon the DPS concept, we adapt it to the CNF-
encoded latent diffusion process, facilitating _zero-shot_ conditional
generation. In CoNFiLD, Eq. 22 is reformulated as,

$\bm{\Psi}=\mathcal{F}\Big{(}\mathcal{E}_{\zeta^{*},\gamma^{*}}(\bm{z}_{0};\zeta^{*},\gamma^{*})\Big{)}+\bm{\epsilon}_{c}.$
(23)

The conditional relationship between the perturbed latent states and the
conditions $\bm{\Psi}$ is expressed via Bayes’s theorem as
$p(\bm{z}_{\tau}|\bm{\Psi})={p(\bm{\Psi}|\bm{z}_{\tau})p(\bm{z}_{\tau})}/p(\bm{\Psi})$,
with the normalizing constant $p(\bm{\Psi})$ being generally intractable. This
complexity is circumvented by shifting the formulation to score functions,

$\nabla_{\bm{z}_{\tau}}\log
p(\bm{z}_{\tau}|\bm{\Psi})=\nabla_{\bm{z}_{\tau}}\log
p(\bm{\Psi}|\bm{z}_{\tau})+\nabla_{\bm{z}_{\tau}}\log p(\bm{z}_{\tau}),$ (24)

where second term $\nabla_{\bm{z}_{\tau}}\log p(\bm{z}_{\tau})$ is the score
function for unconditional generation, approximated by
$\bm{s}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau)$ post-training. However, the
gradient of log-likelihood term, $\nabla_{\bm{z}_{\tau}}\log
p(\bm{\Psi}|\bm{z}_{\tau})$, remains to be estimated. To this end, the
likelihood density is factorized,

$\displaystyle p(\bm{\Psi|\bm{z}_{\tau}})$ $\displaystyle=\int
p(\bm{\Psi}|\bm{z}_{0},\bm{z}_{\tau})p(\bm{z}_{0}|\bm{z}_{\tau})d\bm{z}_{0}=\int
p(\bm{\Psi}|\bm{z}_{0})p(\bm{z}_{0}|\bm{z}_{\tau})d\bm{z}_{0}$ (25)
$\displaystyle=\mathbb{E}_{\bm{z}_{0}\sim
p(\bm{z}_{0}|\bm{z}_{\tau})}\Big{[}p(\bm{\Psi}|\bm{z}_{0})\Big{]},$

which can be approximated by,

$p(\bm{\Psi}|\bm{z}_{\tau})=\mathbb{E}_{\bm{z}_{0}\sim
p(\bm{z}_{0}|\bm{z}_{\tau})}\big{[}p(\bm{\Psi}|\bm{z}_{0})\big{]}\approx
p\big{(}\bm{\Psi}|\mathbb{E}[\bm{z}_{0}|\bm{z}_{\tau}]\big{)}.$ (26)

The approximation error is theoretically bounded bounded with the Jensen gap
[57]. Accordingly, the gradient of the log likelihood can be approximated as,

$\nabla_{\bm{z}_{\tau}}\log
p(\bm{\Psi}|\bm{z}_{\tau})\approx\nabla_{\bm{z}_{\tau}}\log
p\big{(}\bm{\Psi}|\mathbb{E}[\bm{z}_{0}|\bm{z}_{\tau}]\big{)}$ (27)

where the posterior mean
$\hat{\bm{z}}_{0}=\mathbb{E}[\bm{z}_{0}|\bm{z}_{\tau}]$ can be computed as,

$\hat{\bm{z}}_{0}=\mathbb{E}[\bm{z}_{0}|\bm{z}_{\tau}]=\frac{1}{\sqrt{\bar{\alpha}_{\tau}}}\bigg{(}\bm{z}_{\tau}+(1-\bar{\alpha}_{\tau})\nabla_{\bm{z}_{\tau}}\log
p(\bm{z}_{\tau})\bigg{)},$ (28)

where the Stein score $\nabla_{\bm{z}_{\tau}}\log p(\bm{z}_{\tau})$ has been
learned during the unconditional diffusion modeling training, i.e.,
$\nabla_{\bm{z}_{\tau}}\log
p(\bm{z}_{\tau})\approx\bm{s}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})$.
The we have approximated $\hat{\bm{z}}_{0}$,

$\hat{\bm{z}}_{0}\approx\hat{\bm{z}}_{0}^{*}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})=\frac{1}{\sqrt{\bar{\alpha}_{\tau}}}\bigg{(}\bm{z}_{\tau}+(1-\bar{\alpha}_{\tau})\bm{s}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})\bigg{)},$
(29)

Based on the nested probabilistic nonlinear relationship between the condition
$\bm{\Psi}$ and CNF-encoded flow field latents $\bm{z}_{0}$, the approximated
likelihood is,

$p(\bm{\Psi}|\bm{z}_{0})\approx
p(\bm{\Psi}|\bm{z}_{0}^{*})\sim\mathcal{N}\Bigg{(}\mathcal{F}\bigg{(}\mathcal{E}_{\zeta^{*},\gamma^{*}}\Big{(}\bm{z}_{0}^{*}(\bm{z}_{\tau},\tau;\bm{\theta}^{*});\zeta^{*},\gamma^{*}\Big{)}\bigg{)},\sigma_{c}^{2}\bm{I}\Bigg{)},$
(30)

if the uncertainty term $\epsilon_{c}$ in Eq. 22 has a zero-mean Gaussian
distribution, i.e.,
$\epsilon_{c}\sim\mathcal{N}(\bm{0},\sigma_{c}^{2}\bm{I})$. By differentiating
the approximated log likelihood with respect to $\bm{z}_{\tau}$,

$\nabla_{\bm{z}_{\tau}}\log
p(\bm{z}_{\tau}|\bm{\Psi})\approx-\frac{1}{\sigma_{c}^{2}}\nabla_{\bm{z}_{\tau}}\Bigg{\|}\bm{\Psi}-\mathcal{F}\bigg{(}\mathcal{E}_{\zeta^{*},\gamma^{*}}\Big{(}\bm{z}_{0}^{*}(\bm{z}_{\tau},\tau;\bm{\theta}^{*});\zeta^{*},\gamma^{*}\Big{)}\bigg{)}\Bigg{\|}^{2}_{L2},$
(31)

which can be computed using the chain rule,

$\displaystyle\nabla_{\bm{z}_{\tau}}\log p(\bm{z}_{\tau}|\bm{\Psi})$
$\displaystyle\approx\nabla_{\bm{z}_{\tau}}\log
p_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}(\bm{\Psi}|\bm{z}_{\tau})$ (32)
$\displaystyle=-\frac{2}{\sigma_{c}^{2}}(\bm{\Psi}-\mathcal{F}(\mathcal{E}_{\zeta^{*},\gamma^{*}}))\frac{\partial\mathcal{F}(\mathcal{E}_{\zeta^{*},\gamma^{*}})}{\partial\mathcal{E}_{\zeta^{*},\gamma^{*}}}\frac{\partial\mathcal{E}_{\zeta^{*},\gamma^{*}}(\bm{z}_{0}^{*})}{\partial\bm{z}_{0}^{*}}\frac{\partial\bm{z}_{0}^{*}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})}{\partial\bm{z}_{\tau}},$

by leveraging the automatic differentiation (AD) capability using
differentiable programming for the implementation. Finally, the gradient of
log posterior (i.e., guided score function
$\bm{s}^{\mathrm{guided}}_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}$) as shown by
Eq. 24, can be computed as,

$\displaystyle\nabla_{\bm{z}_{\tau}}\log p(\bm{z}_{\tau}|\bm{\Psi})$
$\displaystyle\approx\nabla_{\bm{z}_{\tau}}\log
p_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}(\bm{\Psi}|\bm{z}_{\tau})+\nabla_{\bm{z}_{\tau}}\log
p_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}(\bm{z}_{\tau})$ (33)
$\displaystyle=p_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}(\bm{\Psi}|\bm{z}_{\tau})+\bm{s}_{\bm{\theta}^{*}}(\bm{z}_{\tau},\tau;\bm{\theta}^{*})$
$\displaystyle=\bm{s}^{\mathrm{guided}}_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}(\bm{\Psi},\bm{z}_{\tau},\tau;\zeta^{*},\gamma^{*},\bm{\theta}^{*}).$

Therefore, conditional sampling can be achieved by modifying the unconditional
score function as above. Without the need of retraining the CoNFiLD, new
spatiotemporal turbulent flow fields $\bm{\Phi}|\bm{\Psi}$ given conditions
$\bm{\Psi}$ can be rapidly generated using the guided score function
$\bm{s}^{\mathrm{guided}}_{\bm{\theta}^{*},\zeta^{*},\gamma^{*}}$ derived in
Eq. 33.

##  3 Numerical Results

In this section, we conduct extensive numerical experiments to assess the
performance of our proposed CoNFiLD method on a variety of stochastic
spatiotemporal flow generation scenarios, including irregular pipe flow with
stochastic forcing, turbulent channel flow, flow over periodic hills, and
wall-bounded turbulence with roughness, highlighting the model’s proficiency
in navigating both regular and irregular geometries and managing scenarios
with varying flow separation. The dynamics of these fluid flows are governed
by the unsteady incompressible Navier-Stokes (NS) equations.

$\begin{split}\frac{\partial\bm{u}}{\partial
t}+(\bm{u}\cdot\nabla)\bm{u}&=-\nabla p+\nu\nabla^{2}\bm{u}+\bm{f},\\\
\nabla\cdot\bm{u}&=0,\end{split}$ (34)

where $\bm{u}(\bm{x},t)$ denotes the velocity vector, $p(\bm{x},t)$ the
pressure, $\nu$ the viscosity, and $\bm{f}(\bm{x},t)$ the forcing term. We
will first present the model’s capability to synthesize new 4D instantaneous
flow fields across these scenarios, with a comparison against DNS references.
Additionally, the trained CoNFiLD will be used for zero-shot conditional
generation for various data assimilation and inverse problem applications
without the need for retraining. These applications range from the full-field
reconstruction of flow sequences from sparse sensor measurements to super-
resolved spatiotemporal generation and turbulence data restoration.

###  3.1 Unconditional generation of spatiotemporal flow fields

####  3.1.1 Two dimensional irregular pipe flow with stochastic forcing term

We begin with a 2D flow within an irregular pipe subject to stochastic forcing
to demonstrate CoNFiLD’s capability of handling unstructured flow data with
irregular geometries. This system can be described by Eq. 34 with a stochastic
forcing term $\bm{f}=[f_{x}(\bm{x},t),f_{y}(\bm{x},t)]^{T}$, which is governed
by a stochastic diffusion equation,

$\frac{\partial\bm{f}}{\partial t}=\nu_{f}\nabla^{2}\bm{f}+\bm{\delta},$ (35)

where $\bm{\delta}=[\delta_{x},\delta_{y}]^{T}$ represents a stochastic source
term, with each component sampled from a standard normal distribution
$\delta_{x},\delta_{y}\sim\mathcal{N}(0,1)$, and $\nu_{f}=2$ is the diffusion
coefficient for spreading the stochastic forcing. To generate training data,
DNS is conducted by solving these stochastic incompressible NS equations on a
2D irregular domain with unstructured grids (see Fig. 2(d)). A long-span
spatiotemporal flow sequence $\bm{\Phi}^{dns}$ consisting of $16,000$
instantaneous flow fields of $u,v,p$ is obtained from the DNS, subsequently
partitioned into $15,873$ shorter sub-sequences $\tilde{\bm{\Phi}}_{i}$, each
consisting of $N_{t}=128$ snapshots, to assemble a dataset, of which 80$\%$ is
used for training
($\mathcal{A}_{train}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{12,698}$) and
remaining is reserved for testing purpose. The CoNFiLD is trained on
$\mathcal{A}_{train}$ unconditionally.

Figure 2: Unconditional generation of flow in a 2D irregular pipe with
stochastic forcing. (a) A trajectory of velocity magnitude ($\|\bm{u}\|$)
fields of the DNS data $\bm{\Phi}^{dns}$ (ground truth). (b) Three randomly
generated flow sequence samples by CoNFiLD (velocity magnitude fields at
selected time steps). (c) Comparison of the PDF of the velocity magnitude
(left panel) and pressure $p$ (right panel) between the CoNFiLD generated
samples and the DNS labels (ground truth). (d) The irregular computational
domain with unstructured grids. (e-f) The comparison of the time-averaged mean
(e) and standard deviation over time (f) between the generated samples (left)
and label data (middle), with the absolute discrepancy (right).

The results generated by the CoNFiLD model are compared with DNS references in
Fig. 2. Panel (a) depicts a sequence of velocity magnitude snapshots from DNS
at the $0^{th}$, $320^{th}$, $640^{th}$, $960^{th}$, and $1280^{th}$ numerical
time steps, showcasing the stochastic spatiotemporal dynamics through
irregular vortex movement patterns over time. For comparison, three randomly
generated flow sequence samples by CoNFiLD are presented in panel (b), which
exhibit similar stochastic behaviors, maintaining visual and physical
consistency with coherent temporal evolution and clearly defined boundary
layers. Despite their similar stochastic spatiotemporal behavior, the
instantaneous flow patterns differ across different generated trajectory
samples and the DNS reference, highlighting CoNFiLD’s ability to capture the
underlying distribution of the training dataset instead of merely replicating
label data. This is further substantiated in Fig. 2(c), through a comparison
of the probability density function (PDF) of velocity magnitude and pressure
between the 25 CoNFiLD generated flow sequences and the DNS datasets. The PDFs
of velocity magnitude (left panel) and pressure (right panel) for both
CoNFiLD-generated samples (red curves) and DNS data (blue curves) show close
alignment, with only minor discrepancies observed at certain peaks of the
velocity magnitude PDF. In Fig. 2(e), the time-averaged velocity magnitude,
$\displaystyle M(x,y)=\frac{1}{N_{t}}\sum_{t=1}^{N_{t}}||\bm{u}_{t}(x,y)||$,
derived from the CoNFiLD-generated samples is almost identical to that of the
reference DNS data, with an average discrepancy value of merely $0.041$,
representing approximately $4\%$ difference from the reference mean. Figure
2(f) presents the standard deviation of velocity magnitude over time,
$S(x,y)=\sqrt{\frac{1}{N_{t}}\sum_{i=1}^{N_{t}}\left(||\bm{u}_{t}(x,y)||-M(x,y)\right)^{2}}$,
for generated samples against reference DNS data. The minimal discrepancy in
standard deviation, with an absolute mean spatial discrepancy of
$0.0294$—approximately $8.4\%$ of the reference, demonstrates CoNFiLD’s
capability to not only generate accurate spatiotemporal samples but also
effectively capture the underlying distributions.

####  3.1.2 Generating equilibrium inflow turbulence of 3D channel flows

In this subsection, we demonstrate the CoNFiLD model on synthesizing sequences
of instantaneous inlet velocity fields for 3D turbulent channel flows,
highlighting its utility in generating accurate inflow turbulence boundary
conditions, critical for eddy-resolving simulations. Focused on a fully-
developed turbulent channel flow, governed by the incompressible NS equations
with a forcing term $\bm{f}$ that simulates constant pressure gradients
driving the flow, this setup ensures homogeneity in the streamwise and
spanwise directions, while turbulence statistics exhibit variations only in
the wall-normal direction [58]. Our objective here is to generate time-
coherent, three-dimensional instantaneous velocity fields at the channel’s
$z-y$ cross-section ($\bm{u}(y,z,t)=[u(y,z,t),v(y,z,t),w(y,z,t)]^{T}$:
$\partial\Omega\times\mathbb{R}^{+}\to\mathbb{R}^{3}$). The training data,
obtained from fully-resolved DNS of a 3D turbulent channel flow at a friction
Reynolds number of $Re_{\tau}=180$, is sampled over a duration of four flow-
through-time ($T_{\mathrm{flow}}$) with a learning step size of $\Delta
t^{+}_{\mathrm{train}}=0.4$ that is $100\times$ numerical time step size
$\delta t=0.004$, exhibiting temporal correlation. Only instantaneous velocity
flow fields on one cross section $\bm{\Phi}^{dns}$ of $1,200$ learning time
steps are collected to create our dataset. The same DNS resolution is
maintained, i.e., $N_{z}\times N_{y}=100\times 400$. The DNS flow sequence is
divided into $945$ shorter sub-sequences $\tilde{\bm{\Phi}}_{i}$, with each
comprising $N_{t}=256$ snapshots roughly corresponding to one
$T_{\mathrm{flow}}$. This forms a database, of which $80\%$ is used as the
training set
$\mathcal{A}_{\mathrm{train}}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{756}$ and the
remaining $20\%$ is reserved as the test set
$\mathcal{A}_{\mathrm{test}}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{189}$ in the
conditional generation.

Figure 3: Unconditional generation of equilibrium inflow turbulence. (a)
Instantaneous streamwise velocity $u$ obtained by DNS. (b) Three distinct
realizations of $u$ generated by CoNFiLD. (c) Analysis of turbulence
statistics, highlighting mean streamwise velocity (left), root-mean-square
(RMS) of velocity fluctuations (middle), and two-point correlations of each
velocity component at $y^{+}=150$ (right). $\overline{\Box}$ indicates time-
averaged quantities, while $\langle\Box\rangle$ denotes ensemble average
across all samples. Spatial coordinates are normalized by the wall unit
$y^{+}=\frac{yu_{\tau}}{\nu}$, where $y$ is the wall normal distance,
$u_{\tau}$ is the friction velocity, and $\nu$ is the kinematic viscosity.
Velocity statistics are scaled by $u_{\tau}$ for normalization.

The unconditional inflow turbulence generation results of CoNFiLD are compared
with DNS reference in Fig. 3, illustrating both the fidelity and diversity of
the CoNFiLD-generated spatiotemporal velocity field samples. For this
assessment, an ensemble of 50 flow sequences, each with 256 snapshots
(equivalent to 25,600 numerical steps), was synthesized to ensure statistical
convergence. Out of these, three exemplary flow sequences generated by CoNFiLD
are showcased in Fig. 3(b), where the stochastic behavior and vortex patterns
all visually resemble those of the DNS reference in Fig. 3(a), affirming the
model’s fidelity in capturing the essence of turbulent flows. Notably, the
individual instantiations of the generated flow fields exhibit substantial
variability, showcasing a departure from the deterministic nature of neural
solvers like ConvLSTM or Transformer architectures [16], which are
conventionally engineered to output a single deterministic realization. This
comparison underscores CoNFiLD’s ability to not only capture the complex
dynamics of turbulent flows but also to introduce a rich diversity in the
synthesized spatiotemporal velocity field samples, a critical aspect for the
realistic representation of turbulence phenomena. To further quantitatively
evaluate the performance of the CoNFiLD model, we conducted a detailed
analysis of the turbulence statistics across all generated flow sequence
samples. As shown in Fig. 3(c), the turbulence statistics obtained from our
model are in good agreement with those obtained by DNS. In particular, the
mean streamwise velocity profile generated by CoNFiLD accurately matches with
the DNS, reflecting the expected behavior across the linear viscous sublayer,
buffer layer, and logarithmic law region. Similarly, the root-mean-square
(RMS) of velocity fluctuations generated by CoNFiLD is in good agreement with
the DNS results. Additionally, the two-point correlation exhibits an initial
decline to negative values before asymptotically approaching zero, aligning
with DNS observations. This analysis demonstrates that CoNFiLD-generated flow
captures the entire range of turbulence scales and structures, resembling
those identified in DNS with remarkable accuracy. Notably, we didn’t find any
discernible bumps and wiggling in the two point correlations of generated flow
as reported in Gao et al. [40], showing CoNFiLD’s superior performance
compared to state-of-the-art generative methods such as the video diffusion
model.

####  3.1.3 Generating non-equilibrium turbulence of periodic hill

In addition to the previous scenario, we further demonstrate CoNFiLD’s
capability in generating spatiotemporal non-equilibrium turbulence flows
through a classical periodic hill benchmark case, featuring a broad spectrum
of complex flow behaviors including separation, recirculation, and
reattachment. These complex turbulence phenomena are prevalent in a wide range
of engineering applications, from aerospace propulsion to chemical processing,
and pose significant challenges for both traditional numerical models and
data-driven surrogates [59]. For the periodic hill case, the turbulence is
statistically two-dimensional – in streamwise ($x$-) and wall-normal ($y$-)
directions. Therefore, the CoNFiLD here is trained to generate time-coherent,
three-dimensional instantaneous velocity fields at the $x-y$ plane,
$\bm{u}(x,y,t)=[u(x,y,t),v(x,y,t),w(x,y,t)]^{T}$:
$\mathbb{R}^{2}\times\mathbb{R}^{+}\to\mathbb{R}^{3}$. Similar to the previous
example, the training data is a subset of fully-resolved 3D DNS simulation
results with $Re_{h}=2800$, defined by the height $h$ of the hill.
Specifically, to manage computational costs, we first downsample the 3D DNS
data over a duration of $10T_{\mathrm{flow}}$ using a learning time step size
of $\Delta t^{+}_{\mathrm{train}}=1.9$, which consists of $300$ numerical
timesteps, retaining the temporal coherence. We then select three spanwise
cross sections along the $z$ axis from the downsampled data, spaced apart by a
distance of $\Delta z^{+}_{\mathrm{slice}}=112$, thereby reducing spatial
correlation. Fourier Fast Transform (FFT) filter is applied to eliminate high
frequencies beyond a certain threshold and reduce the spatial resolution to
$N_{x}\times N_{y}=88\times 133$, with the threshold set by the highest
frequency the downsampled mesh can accurately represent, according to the
Nyquist-Shannon sampling theorem.

Figure 4: Unconditional generation of non-equilibrium turbulence over periodic
hills. (a) Instantaneous velocity magnitude of the DNS flow data. (b)
Instantaneous velocity magnitudes of three randomly generated realizations by
CoNFiLD. (c) Turbulence statistics at selected locations, including the mean
streamwise velocity $\bar{u}$ (upper left), mean vertical velocity $\bar{v}$
(upper right), Reynolds shear stress $\overline{u^{\prime}v^{\prime}}$ (lower
left), and the total turbulence kinetic energy ($\bar{k}$) (lower right). The
spatial coordinates are normalized by the hill height $h$, and the statistical
quantities are normalized by the bulk velocity $U_{b}$.

This extensive flow sequence $\bm{\Phi}^{dns}$ is partitioned into $2,115$
shorter sub-sequences $\tilde{\bm{\Phi}}_{i}$, each containing $N_{t}=256$
snapshots. This forms a dataset $\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{2115}$,
where $80\%$ is used for training
$\mathcal{A}_{\mathrm{train}}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{1692}$ and
$20\%$ is reserved for testing
$\mathcal{A}_{\mathrm{test}}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{423}$.

The comparison of instantaneous flows unconditionally generated by CoNFiLD
against the ground truth, derived from DNS data, is shown in Fig. 4, where
velocity contours and turbulence statistics are analyzed. Three of the 150
CoNFiLD-generated spatiotemporal trajectories are randomly selected and
presented in panel (b), each comprising a total of $N_{T}=1024$ snapshots
(equivalent to $307,200$ numerical steps), against the DNS ground truth in
panel (a). The comparison shows that all the CoNFiLD-generated flow samples
vividly recreate similar vortex structures and flow characteristics of this
non-equilibrium turbulent flow as the reference, showcasing CoNFiLD’s
exceptional ability to synthesize realistic and physically accurate non-
equilibrium turbulent behaviors. Similar to the prior example, each generated
sample retains uniqueness while closely mimicking the physical behavior of
ground truth. The physical validity of generated flows is further
quantitatively evidenced by the statistical analysis presented in Fig. 4(c),
where the time-averaged mean flow profiles of the generated flow sequences
align closely with the labeled data in both streamwise and wall-normal
directions. Detailed examination of the velocity profiles identifies a
consistent pattern of flow separation immediately downstream of the hill (at
$x/h\leq 5$) across all generated samples, mirroring the DNS results.
Additionally, both the generated and DNS data exhibit a clear recirculation
zone between $x/h=2$ and $x/h=4$ with reattachment occurring around $x/h=5\sim
5.5$, where no negative mean velocity is observed. More remarkably, the
Reynolds shear stress $\langle\overline{u^{\prime}v^{\prime}}\rangle$ and
turbulence kinetic energy (TKE) $\bar{k}$
(=$\displaystyle\frac{1}{2}\left(\langle\overline{u^{\prime}u^{\prime}}\rangle+\langle\overline{v^{\prime}v^{\prime}}\rangle\right)$
of the generated samples closely match those of the ground truth, with peaks
observed in the free shear layer (shown in Fig. 4 (c)). These results affirm
that the turbulence synthesized by CoNFiLD faithfully replicates the
statistical characteristics of the label data. Notably, conventional RANS and
LES methods tend to underpredict some of these statistical metrics, especially
in complex flow regimes with separations and recirculations. In contrast,
CoNFiLD can accurately capture the flow statistics yet with substantially less
computational cost, as further discussed in Sec. 4.

####  3.1.4 Generating 3D wall-bounded turbulence with wall roughness

After showcasing CoNFiLD’s effectiveness in synthesizing cross-sectional
spatiotemporal turbulence, we extend its application to a more challenging
scenario: the spatiotemporal generation of sophisticated instantaneous wall-
bounded turbulent flows within 3D domains featuring regular wall-roughness
elements. Turbulent flows over a rough surface are ubiquitous in various naval
systems due to manufacturing processes or service-induced erosion and
biofouling [60]. Different roughness conditions significantly affect near-wall
turbulence structures and the transfer of scalar, momentum, and energy,
impacting the safety, performance, and efficiency of marine systems. However,
accurately modeling and predicting rough-wall turbulence with eddy-resolving
simulations demand prohibitive computational resources, positioning CoNFiLD as
a valuable alternative for fast surrogate modeling. In response, CoNFiLD is
applied in this case to learn from high-fidelity DNS data, enabling the
efficient generation of realistic turbulent flows over rough surfaces with
significant speedup. Specifically, our goal here is to generate time-coherent,
four-dimensional realistic instantaneous velocity fields
($\bm{u}(x,y,z,t)=[u(x,y,z,t),v(x,y,z,t),w(x,y,z,t)]^{T}$:
$\Omega\times\mathbb{R}^{+}\to\mathbb{R}^{3}$). The training data originates
from a fully resolved 3D transient DNS of wall-bounded turbulence over cubic
roughness elements, at a Reynolds number of $Re_{h}=3200$, which is based on
the cube height $h$. We subsample exclusively during the fully developed phase
of flow using a time step of $\Delta t^{+}_{\mathrm{train}}=0.8$, which is
$100\times$ the numerical timestep, to preserve temporal correlation. Due to
the GPU memory limitations in our lab, the training and turbulence generation
for this 3D domain focus to a sub-region. We apply the same filtering and
downsampling methods as detailed in Sec. 3.1.3 for the 3D sub-region,
resulting in a spatiotemporal flow sequences $\bm{\Phi}^{dns}$ consisting of
$1200$ snapshots with a resolution of $N_{x}\times N_{y}\times N_{z}=32\times
34\times 62$. This long-span sequence is partitioned into $817$ shorter sub-
sequences $\tilde{\bm{\Phi}}_{i}$, each consisting of $N_{t}=384$ snapshots,
to assemble the dataset $\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{817}$. During
training, $80\%$ of the database is used as the training set
$\mathcal{A}_{\mathrm{train}}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{653}$ and the
remaining $20\%$ is reserved as the test set
$\mathcal{A}_{\mathrm{test}}=\\{\tilde{\bm{\Phi}}_{i}\\}_{i=1}^{164}$ for
conditional generation validation.

Figure 5: Unconditional generation of 3D wall-bounded turbulence. The
instantaneous velocity magnitude (top) and iso-surfaces of Q-criterion
(bottom) of (a) DNS and (b) two randomly generated realizations by CoNFiLD.
(c) The turbulence statistics, including the time-averaged streamwise velocity
$\bar{u}$ (left), and turbulence intensity $I_{u}$ (right), both above and
between the roughness elements. Spatial coordinates are normalized by the
height of the roughness elements, and the statistics are normalized by bulk
velocity $U_{b}$.

Figure 5 (b) showcases two instances of flow generated unconditionally by
CoNFiLD, encompassing 1536 learning steps, equivalent to 153,600 numerical
timesteps, alongside the labeled flow trajectory depicted in Fig. 5 (a). These
flow trajectory samples are visualized through the velocity magnitude contours
and the isosurfaces of the Q criterion, providing a detailed view of the
three-dimensional turbulence characteristics within the domain. Notably, our
CoNFiLD accurately reproduces the large-scale vortices associated with the
roughness, closely mirroring the dynamics observed in DNS. Meanwhile,
noticeable differences in the small-scale vortices among the generated samples
and DNS highlight CoNFiLD’s capability to capture the inherent probabilistic
nature of wall-bounded turbulence. Additionally, the first- and second-order
turbulence statistics of flows generated by CoNFiLD and those obtained from
DNS are compared in Fig. 5 (c), featuring both time-averaged velocity and
turbulence intensity at three representative locations. The agreement of flow
statistics between CoNFiLD and DNS demonstrates the model’s efficacy in
vividly reproducing the instantaneous unsteady flow patterns, which preserve
the accurate mean flow characteristics, indicating a successful replication of
the primary flow mechanism. The results underscore the model’s proficiency in
generating varied instances of wall-bounded turbulence over extended duration
beyond the training scope, providing statistical and physical fidelity
superior to traditional RANS or unsteady RANS, which often fails to accurately
predict flow separations and reattachments around roughness elements. [61].

###  3.2 Zero-shot conditional spatiotemporal generation without retraining

In addition to generating diverse flow realizations that adhere to the
underlying distribution learned during its training phase, the trained CoNFiLD
model is also capable of producing specific flow realizations conditioned on
given inputs, without the need for retraining. This feature significantly
highlights our model’s versatility, enabling efficient and tailored flow
predictions for various application scenarios. In this subsection, three
different conditional generation applications – sensor-based flow
reconstruction, flow data restoration, and super-resolved generation – are
showcased and discussed.

####  3.2.1 Flow reconstruction from sparse sensor measurements

We first explore an application of significant practical importance: full-
field spatiotemporal reconstruction of flow from sparse sensor data through
zero-shot conditional generation, underpinned by Bayesian posterior sampling.
This capability is essential across various engineering domains, where
obtaining comprehensive full-field flow information is challenging due to
complex setups, prohibitive computational costs, or the inherent sparsity and
noise in direct measurements. Traditional approaches have primarily adopted
deterministic models, incorporating dimensionality reduction techniques like
POD or DNN-based autoencoders [62, 63, 64]. Although these methods have
demonstrated some success in flow reconstruction, they often struggle with
accuracy, robustness, and scalability, particularly in large-scale, complex
turbulent flow scenarios [65].

Figure 6: Flow reconstruction from limited sensor measurements using zero-shot
conditional generation. (a) Comparison of label trajectory and the
reconstructed on for non-equilibrium turbulence over periodic hill (1st and
2nd rows) and 3D wall-bounded turbulence (3rd and 4th rows), where right panel
shows the contours and left panel shows sensor locations and single-point
time-series signals at one sensor location. (b) Sensitivity study for number
of sensors (1st column v.s. 2nd column) and w/o noise (2nd column v.s. 3rd
column) at probed points (first row) and unprobed location (2nd row) for
periodic hill case.

We demonstrate CoNFiLD’s sensor-based conditional generation capability on the
two non-equilibrium wall-bounded turbulence cases: flow over periodic hills
and wall roughness elements, as presented in Secs 3.1.3 and 3.1.4. The problem
is formulated as following: placing limited number of flow sensors sparsely
within the flow field simulated by DNS to collect velocity signals at
different times ($\bm{\Psi}$). These measurements serve as conditional inputs
for CoNFiLD to generate full-scale spatiotemporal fields of this specific flow
realization that is observed. For the periodic hill case, we randomly selected
a flow sequence $\bm{\Phi}$ from the test dataset
($\bm{\Phi}\in\mathcal{A}_{\mathrm{test}}$) as the ground truth, containing
$N_{T}=256$ snapshots, equivalent to $76,800$ numerical steps. Similarly, for
the 3D wall-roughness case, the ground truth is a randomly selected test flow
sequence of $N_{T}=384$, corresponding to 38,400 numerical steps. For the
periodic hill and wall roughness cases, we randomly placed 10 and 100 sensors,
corresponding to $0.1\%$ and $0.17\%$ of the grid points in each case,
respectively. These sparse sensor measurements are then utilized to
reconstruct the full-field spatiotemporal flows. Performance is assessed by
comparing the reconstructed flows to the ground truth, as shown in Figure
6(a), which displays contour comparisons and single-point time-series signal
analysis at the sensor location. Unlike unconditional generation, the
reconstructed flows, despite being one of many realizations generated by
CoNFiLD, show notable similarities to the ground truth in both contour maps
and sensor signal patterns, owing to the inclusion of conditional information
(i.e., sensor measurements). Note that the conditionally generated samples,
though very similar, are still slightly different from each other. The
scattering of the generated ensemble can be viewed as the uncertainty of the
flow reconstruction, bypassing the necessity for model retraining. This
adaptability and stochasticity of CoNFiLD enable it to not only reconstruct
the specific flow realization observed by the sensors but also provide
uncertainty estimates accordingly. This capability distinctively
differentiates our approach from deterministic regression-based reconstruction
methods, which are restricted to producing a single deterministic flow
sequence. A closer examination of the contours indicates minor discrepancies
in capturing small-scale flow structures, consistent with unconditional
generation. The disparity is slightly more noticeable in the 3D rough-wall
turbulence case, reflecting its higher complexity. Future improvements in
model capacity and computational resources may address these limitations.

We further explored how sensor configuration influences flow reconstruction
performance in the periodic hill case. This involved adjusting the number of
sensors and incorporating noise to better simulate real-world conditions, with
the results presented in Fig. 6(b). The first and second columns compare the
reconstruction performance using 1 and 100 sensors, respectively, by plotting
mean and standard deviation (std) together with the ground truth for both
probed and unprobed locations. To ensure statistical reliability, we generated
and analyzed 50 samples, determining the mean and std. The uncertainty is
visualized by shading an area that spans three stds from the mean, providing
clear insight into the variability of generated realizations. With a single
sensor, the reconstructed uncertainty is considerable; however, the mean,
despite deviating from the ground truth, roughly follows its trend at both
probed and unprobed locations. Increasing the sensor count to 100
significantly enhances the alignment of the mean curve with the actual data
and markedly narrows the uncertainty bounds. This improvement aligns with the
expectations from Bayesian perspective, as more conditional information
sharpens the high-density regions of the likelihood function, resulting in a
more concentrated posterior distribution. Intuitively, our certainty about the
reconstructed flow field increases with more observations. Additionally, we
introduced Gaussian noise ($10\%$ of the original data range) to the signal of
the 100 sensors and plotted the results in the third column. Compared to the
second column, there is no notable performance drop at both probed and
unprobed points even with noisy measurements, indicating the robustness of our
model. These findings underscore the significant potential of our CoNFiLD
model in scaling up to various real-world applications, demonstrating its
flexibility with respect to sensor arrangements and its robustness against
variations in signal quality.

####  3.2.2 Flow restoration from damaged data

The storage of turbulence data presents a substantial challenge within the CFD
community, with data corruption noted as a major concern [66]. Although
physics-based [67, 68] and deep learning strategies [69] have shown success in
recovering fluid dynamics data for canonical flows, such as lid-driven cavity
and flow around a cylinder, their applications in restoring turbulent flow
data is less explored. To tackle this problem, we demonstrate another novel
application of CoNFiLD: high-fidelity restoration of corrupted turbulence
data. We use the damaged data as conditional input ($\bm{\Psi}$) to facilitate
the recovery of lost flow information by conditional generation. In this
study, the data damage is defined as the absence of flow information at a
central subregion of the fluid domain, mathematically described as a
spatiotemporal masking operation. This objective is to precisely restore the
missing flow details by leveraging the information available from the
surrounding regions.

Figure 7: Inpainting of inlet turbulence for channel flow. (a) Instantaneous
velocity magnitude contours of DNS, damaged and recovered data. (b) Comparison
of the PDF of the velocity magnitude between DNS, damaged and recovered data.
(c) Comparison of the velocity magnitude profile between DNS, damaged, and
recovered data at three spanwise locations

Using the turbulence inlet case previously presented in Section 3.1.2, we
illustrate the data restoration capability of the CoNFiLD model. A subset of
the trajectory (${\bm{\Phi}}$) with $N_{T}=32$ frames (equivalent to $3,200$
numerical time steps) from the test dataset, previously unseen by the CoNFiLD
model, is selected as the ground truth. The corrupted data are created by
masking the central subregion of the ground truth across all time steps, as
shown in the second row of Fig. 7(a). These corrupted data then serve as
conditional information for CoNFiLD to infer the flow dynamics within the
masked area. Notably, the square damaged region defined here is illustrative;
in practice, the shape of the damaged region can vary significantly, extending
to the domain’s boundaries without restrictions.

To accurately quantify uncertainty in the restoration process, we generate
$18$ conditioned samples, two of which are presented in Fig. 7(a) alongside
the original and damaged data. The CoNFiLD model consistently restores the
flow within the masked areas, seamlessly integrating with the surrounding data
without noticeable discrepancies/inconsistencies at the interface. However,
each generated sample varies slightly from the others, subtle in the contour
plots but apparently reflected in the depiction of uncertainty regions shown
in Fig. 7(d). A closer look at velocity magnitude $||u||$ profiles at three
cross-sections (Fig. 7(d)), reveals increased uncertainty from the periphery
towards the center of the damaged area. This trend is due to higher spatial
covariance with adjacent known flow information near the edges, leading to
reduced uncertainty compared to the central portion of the masked area.
Nonetheless, the overall uncertainty remains minimal, suggesting that CoNFiLD
effectively utilizes surrounding flow information to draw from the posterior
distribution closely aligned with the ground truth. Further evidence of
CoNFiLD’s proficiency is presented in Fig. 7(c), where it significantly
refines the probability density function (PDF) of the velocity magnitude
$||u||$ of the damaged data, aligning it closely with the PDF of the original
data.

####  3.2.3 Spatiotemporal super-resolution of low-fidelity data

Super-resolution techniques are rapidly being adopted across various
computational and experimental communities to derive significant details from
low-resolution (LR) images and data. Analytical, physics-based, and deep
learning super-resolution techniques have shown promising results, from
improving low-fidelity simulation results to enhancing under-resolved 4D flow
MR imaging data [28, 70, 37, 71, 72, 73, 74]. Motivated by these advancements,
we present another capability of our proposed CoNFiLD model—creating highly
detailed instantaneous flows from LR counterparts, showcasing significant
potential for large-scale super-resolution challenges. Through the turbulence
channel flow case, we demonstrate the zero-shot super-resolution capability of
the trained CoNFiLD model, regardless of the quality of LR data. We select a
sub-trajectory (${\bm{\Phi}}$) comprising $N_{T}=256$ frames (equivalent to
$25600$ numerical time steps) from the test dataset to serve as the ground
truth. Three different levels of LR data are generated by downsampling the
high-resolution (HR) DNS ($400\times 100$) to three different resolutions,
$64\times 16$, $16\times 4$, and $4\times 1$, to cover a spectrum of LR
scenarios typically encountered in practice. 25 samples are generated for each
LR scenario to ensure accurate estimation of the statistical metrics.

Figure 8: Super-resolution for inlet turbulence of 3D channel flow. (a)
Instantaneous velocity magnitude contours of DNS. (b) Super-resolved (SR)
generation results by CoNFiLD, comparing the conditional LR input and SR
contours. (c) Comparison of TKE spectra cross DNS, CoNFiLD SR results, and
bicubic interpolation baselines. Notably, due to its inadequate performance,
bicubic interpolation is excluded for the most extreme downscaling scenario
($4\times 1\to 400\times 100$).

The performance of the CoNFiLD is illustrated in Fig. 8, with panel (a)
showing the ground truth trajectory ($400\times 100$) and (b) displaying pairs
of LR input and its super-resolved (SR) flow contours, across three different
LR settings. Impressively, regardless of the initial quality of the LR data,
all CoNFiLD-reconstructed flows are up-scaled to the original high resolution
of $400\times 100$, achieving a visual fidelity closely akin to the DNS
reference. This can be further substantiated through the TKE spectrum analysis
in Fig. 8(c), comparing the SR flows against both the ground truth and the
baseline SR result using bicubic interpolation. The bicubic SR method
significantly fails to recover high-frequency details starting from such low
resolution input, and its performance deteriorates with decreasing input
quality. In stark contrast, CoNFiLD’s reconstructions accurately replicate the
true spectrum across all scales, even for the input with the lowest resolution
($4\times 1$). Upon closer examination of the instantaneous flow contour
comparisons, the SR reconstructions for the lowest resolution (at the bottom
of Panel (b)) deviate from ground truth data, primarily because the
exceedingly low input resolution provides negligible informative conditions,
rendering the model’s behavior similar to unconditional generation. As input
resolution increases (from bottom to top in panel (b)), the conditionally
generated SR samples increasingly align with the instantaneous flow patterns
of the ground truth, with samples generated from the $64\times 16$ LR input
nearly indistinguishable from the DNS data. CoNFiLD’s Bayesian formulation
enables robust SR across varying input qualities, contrasting with many
existing SR methods highly dependent on input resolution. Notably, CoNFiLD’s
ability to handle super-resolution tasks across different discretized flow
representations—structured or unstructured—without retraining highlights its
versatility. This adaptability clearly surpasses the capabilities of CNNs,
which require retraining for different input resolutions and qualities.
Similarly, although GNNs can manage unstructured data, they struggle with
scale generalization. These attributes emphasize CoNFiLD’s efficacy and
adaptability in super-resolution applications, underlining its potential to
tackle complex engineering challenges beyond flow data enhancement.

##  4 Discussion

To evaluate CoNFiLD’s efficiency improvements over traditional CFD methods and
existing DL-based generative models, we assessed its computational cost
compared to established benchmarks, such as OpenFOAM (a CPU-based CFD solver
in C++) [75], Diff-FlowFSI (an in-house GPU-enabled, fully-vectorized
differentiable CFD solver in JAX) [76], and the video diffusion model for
spatiotemporal turbulence generation [40]. This comparison was made using the
inlet turbulence generation case detailed in Section 3.1.2, reporting the
expected time cost for generating $N_{T}=300$ (60,000 numerical timesteps) of
spatiotemporal turbulence flow sequences in Fig. 9(a). Compared to OpenFOAM,
the GPU-accelerated fully vectorized JAX solver, Diff-FlowFSI achieves a
30-fold increase in speed. The video diffusion model, operating directly on
physical space, further boosts this speedup to 128 times. Remarkably, by
operating a diffusion process in latent space, our CoNFiLD extends this
speedup to an impressive 1737-fold. This exceptional efficiency stems from
multiple factors: Firstly, CoNFiLD runs on GPUs, contrasting with CPU-based
OpenFOAM, providing an initial efficiency boost at the cost of higher memory
demand. Secondly, compared to GPU-accelerated solvers like Diff-FlowFSI facing
timestep constraints by the CFL condition, CoNFiLD can employ significantly
greater timestep size without convergence issues. It leverages pre-trained
knowledge on the probability distribution of all possible flow solutions for
rapid online inference, offering an additional layer of efficiency boost.
Thirdly, existing DL-based video generative modeling techniques such as the
video diffusion model, though a similar probabilistic view, directly operate
in pixel/physical space, which can be easily bottlenecked by memory for
extended sequences. Unlike the model by Gao et al. [40], which requires an
auto-regressive conditional generation for long-span generation, CoNFiLD
generates latent images for large $N_{T}$ with a much smaller memory
footprint, enabling direct generation without the need for auto-regressive
sequential conditioning. The only overhead of the CoNFiLD model is the cost of
the decoding process, which is negligible compared to the latent diffusion
process (approximately ten times less). This distinction adds another layer of
performance boost to our model. In summary, CoNFiLD achieves unparalleled
performance gains among peer methods, showing substantial potential for
scaling up to higher dimension flow data.

Figure 9: Performance evaluation. (a) Inference time cost comparison among
OpenFOAM ran on CPU and Diff-FLowFSI ran on GPU, Video-diffusion model [40],
and CoNFiLD (ours). (b) GPU memory cost comparison for unconditional
generation w/o CNF encoder. (c) GPU memory cost comparison for implicit (ours)
and explicit decoding strategy (e.g., POD, CNN) in subsampling-based
conditional generation.

We further explored the memory usage differences when CoNFiLD performs
diffusion processes either in physical space (without the CNF encoder) or in
latent space (with the CNF encoder), focusing on unconditional generation
scenarios for a fair comparison (shown in Fig. 9(b)). This comparison also
relates to other generative AI-based spatiotemporal flow generators that lack
an encoding mechanism (e.g., video-diffusion [40]). Monitoring CoNFiLD’s
memory demand over a range of inference lengths, from 1 to 8000 learning
steps, both with and without the CNF encoder, we found significant performance
differences. As illustrated in Fig. 9(b), absent the CNF encoder, memory usage
quickly reaches the maximum capacity of current top-tier GPUs with increasing
inference length, maxing out at 16 learning time steps for the Nvidia RTX4090
and 20 steps for the Nvidia A100. In contrast, with the CNF encoder, memory
consumption increases more gradually, enabling significantly longer inference
stretches—up to 3900 and 8000 steps for the Nvidia RTX4090 and A100,
respectively. This results in an extraordinary extension of inference lengths
by factors of 243 and 400 for the two GPUs, underscoring the substantial
benefits of latent space synthesis facilitated by the CNF encoder’s robust
encoding capabilities. Notably, the CNF achieves compression ratios of
$0.017\%$ for periodic hill case and $0.002\%$ for 3D rough wall turbulence
case, an impressive achievement given the complexity of the flows processed.
While convolutional autoencoders (CAE) may reach similar compression ratios,
as suggested by related research [42, 77, 78], the memory constraints of
loading the full-field 3D/4D data become the bottleneck. Moreover, CNF’s
inherent implicit nature to handle unstructured data sets it apart—Unlike CAE,
which explicitly encode data via convolutions and pooling on fixed regular
grids, CNF allows CoNFiLD to train on and generate turbulence on unstructured
grids simply by querying the CNF with desired coordinates and latent vectors.

Surprisingly, the advantages of incorporating CNF extend beyond this, as we
found a distinct benefit for memory efficiency brought by the CNF in the
subsampling-based conditional generation of CoNFiLD. In particular, the
conditional generation process entails a forward evaluation and
backpropagation of Eq.23. Note that this requires resolving the whole field
before performing the forward function $\mathcal{F}$. The procedure remains
the same if the CNF encoder is substituted by an explicit ML-based encoder
like CAE. However, if the function $\mathcal{F}$ involves a subsampling
process $\mathcal{M}$ in time and the spatial dimension, we can apply the
subsampling $\mathcal{M}$ on the query spatiotemporal coordinates before
passing them into the CNF decoder, thereby bypassing the recovery of the whole
flow field and significant reducing the memory usage both in forward
computation and backward gradient estimation. To demonstrate this, we define
the forward function $\mathcal{F}$ simply as a masking function that preserves
$10\%$ spatial points. The memory consumption of CoNFiLD using this pre-
subsampling technique (only available with CNF) versus the original process
(the only option for explicit encoders) for different inference lengths $N$
are plotted in Fig. 9(c), where one can observe that using an explicit encoder
quickly exceeds the top tier GPUs at $N=64$ and $N=128$ for Nvidia RTX4090 and
A100 respectively. In contrast, the presence of CNF controls the memory cost
under the limit of Nvidia RTX4090 in all three occasions. This fully verifies
the memory benefit of CNF during the subsampling-based conditional generation
of the CoNFiLD.

##  5 Conclusion

In this study, we introduce CoNFiLD, an innovative deep generative learning
framework designed for probabilistic generation of complex, three-dimensional
spatiotemporal turbulence. At its core, CoNFiLD uniquely combines a
Conditioned Neural Field (CNF) with a latent diffusion model to enable
scalable, long-span spatiotemporal generation. Specifically, the CNF leverages
its high-efficiency compression capabilities to encode high-dimensional,
intricate scientific data into a compact latent space, and simultaneously, an
unconditional diffusion model operates in the CNF-encoded latent space,
effectively generating new spatiotemporal sequences in a scalable manner. This
unique integration catalyzes the formation of a novel class of latent
diffusion models for space-time generation, marking a significant advancement
in the field of generative modeling.

CoNFiLD has demonstrated its proficiency in generating a broad spectrum of
turbulent flows across complex and irregular domains, successfully capturing
intricate chaotic dynamics and turbulent phenomena. Moreover, CoNFiLD offers
versatile zero-shot conditional generation capabilities, making it highly
applicable to real-time data assimilation or scalable inverse problems in a
variety of scientific and engineering applications, such as sensor-based flow
reconstruction, data restoration, and super-resolution data enhancement, all
without the necessity for model retraining. Additionally, our model exhibits
robust performance and superior computational/memory efficiency compared to
classic numerical simulations and other state-of-the-art generative AI
techniques. While CoNFiLD is primarily showcased on large-scale turbulence
generation in this study, its can be naturally applied to modeling general
spatiotemporal dynamics across various scientific domains.

## Acknowledgment

The authors would like to acknowledge the funds from Office of Naval Research
under award numbers N00014-23-1-2071 and National Science Foundation under
award numbers OAC-2047127. XF would also like to acknowledge the fellowship
provided by the Environmental Change Initiative and Center for Sustainable
Energy at University of Notre Dame.

## Compliance with Ethical Standards

Conflict of Interest: The authors declare that they have no conflict of
interest.

## References

  * [1] S. L. Brunton, B. R. Noack, P. Koumoutsakos, Machine learning for fluid mechanics, Annual review of fluid mechanics 52 (2020) 477–508.
  * [2] R. Vinuesa, S. L. Brunton, Enhancing computational fluid dynamics with machine learning, Nature Computational Science 2 (6) (2022) 358–366.
  * [3] Y. Bar-Sinai, S. Hoyer, J. Hickey, M. P. Brenner, Learning data-driven discretizations for partial differential equations, Proceedings of the National Academy of Sciences 116 (31) (2019) 15344–15349.
  * [4] D. Kochkov, J. A. Smith, A. Alieva, Q. Wang, M. P. Brenner, S. Hoyer, Machine learning–accelerated computational fluid dynamics, Proceedings of the National Academy of Sciences 118 (21) (2021) e2101784118.
  * [5] J.-X. Wang, J.-L. Wu, H. Xiao, Physics-informed machine learning approach for reconstructing reynolds stress modeling discrepancies based on DNS data, Physical Review Fluids 2 (3) (2017) 034603.
  * [6] R. Maulik, O. San, A. Rasheed, P. Vedula, Subgrid modelling for two-dimensional turbulence using neural networks, Journal of Fluid Mechanics 858 (2019) 122–144.
  * [7] V. Shankar, R. Maulik, V. Viswanathan, Differentiable turbulence ii, arXiv preprint arXiv:2307.13533 (2023).
  * [8] J. Tompson, K. Schlachter, P. Sprechmann, K. Perlin, Accelerating eulerian fluid simulation with convolutional networks, in: International Conference on Machine Learning, PMLR, 2017, pp. 3424–3433.
  * [9] X.-Y. Liu, M. Zhu, L. Lu, H. Sun, J.-X. Wang, Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics, Communications Physics 7 (1) (2024) 31.
  * [10] X. Fan, J.-X. Wang, Differentiable hybrid neural modeling for fluid-structure interaction, Journal of Computational Physics 496 (2024) 112584.
  * [11] K. Fukami, Y. Nabae, K. Kawai, K. Fukagata, Synthetic turbulent inflow generator using machine learning, Physical Review Fluids 4 (6) (2019) 064603.
  * [12] K. Fukami, K. Fukagata, K. Taira, Super-resolution reconstruction of turbulent flows with machine learning, Journal of Fluid Mechanics 870 (2019) 106–120.
  * [13] K. Fukami, K. Fukagata, K. Taira, Machine-learning-based spatio-temporal super resolution reconstruction of turbulent flows, Journal of Fluid Mechanics 909 (2021) A9.
  * [14] M. Z. Yousif, L. Yu, H. Lim, Physics-guided deep learning for generating turbulent inflow conditions, Journal of Fluid Mechanics 936 (2022) A21.
  * [15] M. Z. Yousif, M. Zhang, L. Yu, R. Vinuesa, H. Lim, A transformer-based synthetic-inflow generator for spatially developing turbulent boundary layers, Journal of Fluid Mechanics 957 (2023) A6.
  * [16] X. Han, H. Gao, T. Pfaff, J.-X. Wang, L. Liu, Predicting physics in mesh-reduced space with temporal attention, in: International Conference on Learning Representations, 2022.
URL https://openreview.net/forum?id=XctLdNfCmP

  * [17] K. Stengel, A. Glaws, D. Hettinger, R. N. King, Adversarial super-resolution of climatological wind and solar data, Proceedings of the National Academy of Sciences 117 (29) (2020) 16805–16815.
  * [18] C. Drygala, B. Winhart, F. di Mare, H. Gottschalk, Generative modeling of turbulence, Physics of Fluids 34 (3) (2022).
  * [19] Z. Deng, C. He, Y. Liu, K. C. Kim, Super-resolution reconstruction of turbulent velocity fields using a generative adversarial network-based artificial intelligence framework, Physics of Fluids 31 (12) (2019).
  * [20] H. Kim, J. Kim, S. Won, C. Lee, Unsupervised deep learning for super-resolution reconstruction of turbulence, Journal of Fluid Mechanics 910 (2021) A29.
  * [21] A. Güemes, S. Discetti, A. Ianiro, B. Sirmacek, H. Azizpour, R. Vinuesa, From coarse wall measurements to turbulent velocity fields through deep learning, Physics of fluids 33 (7) (2021) 075121.
  * [22] L. Yu, M. Z. Yousif, M. Zhang, S. Hoyas, R. Vinuesa, H.-C. Lim, Three-dimensional esrgan for super-resolution reconstruction of turbulent flows with tricubic interpolation-based transfer learning, Physics of Fluids 34 (12) (2022).
  * [23] M. Buzzicotti, F. Bonaccorso, P. C. Di Leoni, L. Biferale, Reconstruction of turbulent data with deep generative models for semantic inpainting from turb-rot database, Physical Review Fluids 6 (5) (2021) 050503.
  * [24] Y. Xie, E. Franz, M. Chu, N. Thuerey, tempogan: A temporally coherent, volumetric gan for super-resolution fluid flow, ACM Transactions on Graphics (TOG) 37 (4) (2018) 1–15.
  * [25] J. Kim, C. Lee, Deep unsupervised learning of turbulence for inflow generation at various reynolds numbers, Journal of Computational Physics 406 (2020) 109216.
  * [26] J. Gui, Z. Sun, Y. Wen, D. Tao, J. Ye, A review on generative adversarial networks: Algorithms, theory, and applications, IEEE transactions on knowledge and data engineering 35 (4) (2021) 3313–3332.
  * [27] D. Bau, J.-Y. Zhu, J. Wulff, W. Peebles, H. Strobelt, B. Zhou, A. Torralba, Seeing what a gan cannot generate, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 4502–4511.
  * [28] N. Geneva, N. Zabaras, Multi-fidelity generative deep learning turbulent flows, arXiv preprint arXiv:2006.04731 (2020).
  * [29] L. Sun, X. Han, H. Gao, J.-X. Wang, L. Liu, Unifying predictions of deterministic and stochastic physics in mesh-reduced space with sequential flow generative model, in: Thirty-seventh Conference on Neural Information Processing Systems, 2023.
  * [30] P. Dhariwal, A. Nichol, Diffusion models beat gans on image synthesis, Advances in neural information processing systems 34 (2021) 8780–8794.
  * [31] F.-A. Croitoru, V. Hondru, R. T. Ionescu, M. Shah, Diffusion models in vision: A survey, IEEE Transactions on Pattern Analysis and Machine Intelligence (2023).
  * [32] L. Yang, Z. Zhang, Y. Song, S. Hong, R. Xu, Y. Zhao, W. Zhang, B. Cui, M.-H. Yang, Diffusion models: A comprehensive survey of methods and applications, ACM Computing Surveys 56 (4) (2023) 1–39.
  * [33] J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840–6851.
  * [34] Y. Song, S. Ermon, Generative modeling by estimating gradients of the data distribution, Advances in neural information processing systems 32 (2019).
  * [35] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, B. Poole, Score-based generative modeling through stochastic differential equations, arXiv preprint arXiv:2011.13456 (2020).
  * [36] S. Gao, X. Liu, B. Zeng, S. Xu, Y. Li, X. Luo, J. Liu, X. Zhen, B. Zhang, Implicit diffusion models for continuous super-resolution, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 10021–10030.
  * [37] D. Shu, Z. Li, A. B. Farimani, A physics-informed diffusion model for high-fidelity flow field reconstruction, Journal of Computational Physics 478 (2023) 111972.
  * [38] Z. Y. Wan, R. Baptista, Y.-f. Chen, J. Anderson, A. Boral, F. Sha, L. Zepeda-Núñez, Debias coarsely, sample conditionally: Statistical downscaling through optimal transport and probabilistic diffusion models, arXiv preprint arXiv:2305.15618 (2023).
  * [39] T. Li, A. S. Lanotte, M. Buzzicotti, F. Bonaccorso, L. Biferale, Multi-scale reconstruction of turbulent rotating flows with generative diffusion models, Atmosphere 15 (1) (2023) 60.
  * [40] H. Gao, X. Han, X. Fan, L. Sun, L.-P. Liu, L. Duan, J.-X. Wang, Bayesian conditional diffusion models for versatile spatiotemporal turbulence generation, arXiv preprint arXiv:2311.07896 (2023).
  * [41] J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, D. J. Fleet, Video diffusion models, arXiv:2204.03458 (2022).
  * [42] S. Pan, S. L. Brunton, J. N. Kutz, Neural implicit flow: a mesh-agnostic dimensionality reduction paradigm of spatio-temporal data, Journal of Machine Learning Research 24 (41) (2023) 1–60.
  * [43] Y. Yin, M. Kirchmeyer, J.-Y. Franceschi, A. Rakotomamonjy, P. Gallinari, Continuous pde dynamics forecasting with implicit neural representations, in: International Conference on Learning Representations (LCLR), 2023.
URL https://openreview.net/forum?id=B73niNjbPs

  * [44] J. J. Park, P. Florence, J. Straub, R. Newcombe, S. Lovegrove, Deepsdf: Learning continuous signed distance functions for shape representation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 165–174.
  * [45] V. Sitzmann, J. Martel, A. Bergman, D. Lindell, G. Wetzstein, Implicit neural representations with periodic activation functions, Advances in neural information processing systems 33 (2020) 7462–7473.
  * [46] M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Raghavan, U. Singhal, R. Ramamoorthi, J. Barron, R. Ng, Fourier features let networks learn high frequency functions in low dimensional domains, Advances in Neural Information Processing Systems 33 (2020) 7537–7547.
  * [47] T. Li, M. Slavcheva, M. Zollhöfer, S. Green, C. Lassner, C. Kim, T. Schmidt, S. Lovegrove, M. Goesele, R. Newcombe, Z. Lv, Neural 3d video synthesis from multi-view video, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5521–5531.
  * [48] H. Chen, B. He, H. Wang, Y. Ren, S. N. Lim, A. Shrivastava, Nerv: Neural representations for videos, Advances in Neural Information Processing Systems 34 (2021) 21557–21568.
  * [49] B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, R. Ng, Nerf: Representing scenes as neural radiance fields for view synthesis, Communications of the ACM 65 (1) (2021) 99–106.
  * [50] Z. Chen, H. Zhang, Learning implicit fields for generative shape modeling, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 5939–5948.
  * [51] E. Perez, F. Strub, H. De Vries, V. Dumoulin, A. Courville, Film: Visual reasoning with a general conditioning layer, in: Proceedings of the AAAI conference on artificial intelligence, Vol. 32, 2018.
  * [52] L. Serrano, L. Le Boudec, A. Kassaï Koupaï, T. X. Wang, Y. Yin, J.-N. Vittaut, P. Gallinari, Operator learning with neural fields: Tackling pdes on general geometries, Advances in Neural Information Processing Systems 36 (2024).
  * [53] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, in: International conference on machine learning, PMLR, 2015, pp. 2256–2265.
  * [54] A. Q. Nichol, P. Dhariwal, Improved denoising diffusion probabilistic models, in: International Conference on Machine Learning, PMLR, 2021, pp. 8162–8171.
  * [55] X. Li, Y. Ren, X. Jin, C. Lan, X. Wang, W. Zeng, X. Wang, Z. Chen, Diffusion models for image restoration and enhancement–a comprehensive survey, arXiv preprint arXiv:2308.09388 (2023).
  * [56] H. Chung, J. Kim, M. T. Mccann, M. L. Klasky, J. C. Ye, Diffusion posterior sampling for general noisy inverse problems, arXiv preprint arXiv:2209.14687 (2022).
  * [57] X. Gao, M. Sitharam, A. E. Roitberg, Bounds on the jensen gap, and implications for mean-concentrated distributions, arXiv preprint arXiv:1712.05267 (2017).
  * [58] S. B. Pope, S. B. Pope, Turbulent flows, Cambridge university press, 2000.
  * [59] M. Breuer, N. Peller, C. Rapp, M. Manhart, Flow over periodic hills–numerical and experimental study in a wide range of reynolds numbers, Computers & Fluids 38 (2) (2009) 433–457.
  * [60] M. Kadivar, D. Tormey, G. McGranaghan, A review on turbulent flow over rough surfaces: Fundamentals and theories, International Journal of Thermofluids 10 (2021) 100077.  doi:https://doi.org/10.1016/j.ijft.2021.100077.
URL https://www.sciencedirect.com/science/article/pii/S266620272100015X

  * [61] J. L. Santiago, A. Martilli, F. Martín, Cfd simulation of airflow over a regular array of cubes. part i: Three-dimensional simulation of the flow and validation with wind-tunnel measurements, Boundary-layer meteorology 122 (2007) 609–634.
  * [62] P. Dubois, T. Gomez, L. Planckaert, L. Perret, Machine learning for fluid flow reconstruction from limited measurements, Journal of Computational Physics 448 (2022) 110733.
  * [63] J. L. Callaham, K. Maeda, S. L. Brunton, Robust flow reconstruction from limited measurements via sparse representation, Physical Review Fluids 4 (10) (2019) 103907.
  * [64] N. B. Erichson, L. Mathelin, Z. Yao, S. L. Brunton, M. W. Mahoney, J. N. Kutz, Shallow neural networks for fluid flow reconstruction with limited sensors, Proceedings of the Royal Society A 476 (2238) (2020) 20200097.
  * [65] F. Sofos, C. Stavrogiannis, K. Exarchou-Kouveli, D. Akabua, G. Charilas, T. Karakasidis, Current trends in fluid research in the era of artificial intelligence: A review. fluids 7: 116 (2022).
  * [66] D. Brown, P. Messina, D. Keyes, J. Morrison, R. Lucas, J. Shalf, P. Beckman, R. Brightwell, A. Geist, J. Vetter, et al., Scientific grand challenges: Crosscutting technologies for computing at the exascale, US DOE PNNL 20168 (2010) 2–4.
  * [67] S. Lee, I. G. Kevrekidis, G. E. Karniadakis, Resilient algorithms for reconstructing and simulating gappy flow fields in cfd, Fluid Dynamics Research 47 (5) (2015) 051402.  doi:10.1088/0169-5983/47/5/051402.
URL https://dx.doi.org/10.1088/0169-5983/47/5/051402

  * [68] J. Han, L. H. Khieu, E. Johnsen, Reconstruct corrupted data in dg-based cfd simulations, in: AIAA AVIATION 2022 Forum, 2022, p. 4155.
  * [69] M. T. H. de Frahan, R. W. Grout, Data recovery in computational fluid dynamics through deep image priors, arXiv preprint arXiv:1901.11113 (2019).
  * [70] H. Gao, L. Sun, J.-X. Wang, Super-resolution and denoising of fluid flow using physics-informed convolutional neural networks without high-resolution labels, Physics of Fluids 33 (7) (2021) 073603.
  * [71] N. H. de Hoon, A. C. Jalba, E. Eisemann, A. Vilanova, Temporal interpolation of 4d pc-mri blood-flow measurements using bidirectional physics-based fluid simulation., in: VCBM/MedViz, 2016, pp. 59–68.
  * [72] J. Töger, M. J. Zahr, N. Aristokleous, K. Markenroth Bloch, M. Carlsson, P.-O. Persson, Blood flow imaging by optimal matching of computational fluid dynamics to 4d-flow data, Magnetic resonance in medicine 84 (4) (2020) 2231–2245.
  * [73] E. Ferdian, A. Suinesiaputra, D. J. Dubowitz, D. Zhao, A. Wang, B. Cowan, A. A. Young, 4dflownet: super-resolution 4d flow mri using deep learning and computational fluid dynamics, Frontiers in Physics 8 (2020) 138.
  * [74] D. R. Rutkowski, A. Roldán-Alzate, K. M. Johnson, Enhancement of cerebrovascular 4d flow mri velocity fields using machine learning and computational fluid dynamics simulation data, Scientific reports 11 (1) (2021) 10240.
  * [75] H. Jasak, A. Jemcov, Z. Tukovic, et al., Openfoam: A c++ library for complex physics simulations, in: International workshop on coupled methods in numerical dynamics, Vol. 1000, 2007, pp. 1–20.
  * [76] X. Fan, X.-Y. Liu, M. Wang, J.-X. Wang, Diff-flowfsi: A jax-based differentiable solver for turbulent flow and fluid-structure interactions (2024 in. prep).
  * [77] T. Nakamura, K. Fukami, K. Hasegawa, Y. Nabae, K. Fukagata, Convolutional neural network and long short-term memory based reduced order surrogate for minimal turbulent channel flow, Physics of Fluids 33 (2) (2021).
  * [78] K. Hasegawa, K. Fukami, T. Murata, K. Fukagata, Cnn-lstm based reduced order modeling of two-dimensional unsteady flows around a circular cylinder at different reynolds numbers, Fluid Dynamics Research 52 (6) (2020) 065501.