## Generative Learning of the Solution of Parametric Partial Differential Equations Using Guided Diffusion Models and Virtual Observations

Han Gao , Sebastian Kaltenbach , Petros Koumoutsakos a a a, ˚

a School of Engineering and Applied Sciences, Harvard University, 29 Oxford Street, Cambridge, MA 02138, US

## Abstract

We introduce a generative learning framework to model high-dimensional parametric systems using gradient guidance and virtual observations. We consider systems described by Partial Differential Equations (PDEs) discretized with structured or unstructured grids. The framework integrates multi-level information to generate high fidelity time sequences of the system dynamics. We demonstrate the effectiveness and versatility of our framework with two case studies in incompressible, two dimensional, low Reynolds cylinder flow on an unstructured mesh and incompressible turbulent channel flow on a structured mesh, both parameterized by the Reynolds number. Our results illustrate the framework's robustness and ability to generate accurate flow sequences across various parameter settings, significantly reducing computational costs allowing for efficient forecasting and reconstruction of flow dynamics.

Keywords: Partial Differential Equations, Diffusion Models, Parametric Dependence, Gradient Guidance, Virtual Observables, Multiscale Models

## 1. Introduction

High-dimensional Partial Differential Equations (PDEs) provide the foundation for simulating complex phenomena such as climate [2], turbulence [38, 1, 25], material behavior [8] and epidemics [16]. These simulations often depend on a multitude of input parameters, including variations in boundary conditions and intrinsic parameters of the PDE itself. This parametric dependence, coupled with the substantial computational cost of high-fidelity simulations, poses significant challenges for many-query analyses in the context of optimization, uncertainty quantification, and 'what-if' scenarios. [18, 26].

Machine learning approaches using neural networks for approximating the unknown variables [28, 15], have demonstrated promise in solving PDEs formulated as inverse problems. However, they fall short in scenarios involving parameter variations and are less computationally efficient when compared with classical discretisations cast as inverse problems [14, 13]. Neural Operators, such as Fourier Neural Operator (FNO) [20] and DeepONet [23, 36], offer a potent framework for incorporating parametric dependence but struggle with high-dimensional and complex system dynamics. Approaches based on Autoencoders [17] to learn the effective dynamics of high-dimensional PDE systems [12, 33, 6, 7, 24] provide the potential to incorporate parametric dependencies within a suitably constructed latent space. However, the application of these methods to complex dynamical systems remains limited and has not been extensively explored.

Recent diffusion-based models for generative learning [9, 30, 19, 5, 4, 35] have shown potential for generating high-fidelity solutions of high-dimensional PDE systems. Christian et al. [9] extended generative models to physical domains by demonstrating their versatility in surrogate modeling, field reconstruction, and inversion from sparse data; Shu et al. [30] proposed a diffusion model that uses only high-fidelity data for training to reconstruct high-fidelity CFD data from various inputs, including low-fidelity samples and random measurements; Li et al. [19] addressed data augmentation in rotating turbulence, finding diffusion

˚ Corresponding author

Email addresses: hgao1@seas.harvard.edu (Han Gao), skaltenbach@seas.harvard.edu (Sebastian Kaltenbach), petros@seas.harvard.edu (Petros Koumoutsakos)

models promising and enabling probabilistic reconstructions and uncertainty quantification; Gao et al. [5] introduced a generative framework using probabilistic diffusion models for versatile turbulence generation, unifying unconditional and conditional sampling within a Bayesian framework, and demonstrated its capabilities through experiments, including LES synthesis, wall-bounded turbulence generation, and super-resolution of turbulent flows; Wan et al. [35] introduced a two-stage probabilistic framework for statistical downscaling using unpaired data, involving debiasing via optimal transport and upsampling with a probabilistic diffusion model, demonstrating its effectiveness on fluid flow problems and matching physical statistics accurately from low-resolution inputs; Recently, Du et al. [4] introduced the Conditional Neural Field Latent Diffusion (CoNFiLD) model, a generative learning framework for rapid simulation of spatiotemporal dynamics in chaotic and turbulent systems within irregular domains. In this paper the authors integrate conditional neural field encoding with latent diffusion processes for efficient and probabilistic turbulence generation, adaptable to various scenarios without retraining, and demonstrate its effectiveness through numerical experiments.

Despite these promising advancements, there remains a significant gap in the exploration of diffusion models for effectively capturing parametric dependencies in complex high-dimensional systems while preserving the inherent versatility and efficiency of these models.

This paper contributes to addressing these challenges through a novel generative learning framework that leverages gradient guidance and virtual observations. We deploy state-of-the-art diffusion models that have achieved remarkable results in Computer Vision [29] as well as both forward predictions [10, 21, 27, 22] and inverse problems [9, 30]. Moreover, we enforce desired properties or constraints during the predictions phase by employing the concepts of Virtual Observables [11] and gradient guidance [31, 3]. The proposed framework is designed to handle high-dimensional systems and varying parameter inputs without the need for retraining. This capability enables rapid generation of diverse solutions, significantly reducing computational costs and facilitating robust experimentation. We do not attempt to directly predict the full system response due to the resulting large computational costs during both training and predictions of the diffusion model. Instead, we first find a suitable lower-dimensional representation of the system of interest that is able to represent the full high-dimensional solution. In case the high-dimensional PDE solution is given on a structured grid, we deploy a Convolutional Neural Network (CNN) whereas in case of data provided on an unstructured grid we resort to a Graph Neural Network (GNN) formulation. The latter is particularly relevant as many PDE systems are solved on complex geometries using unstructured grids which makes the application of a CNN not possible.

The contributions of this work include:

- 1. a diffusion model augmented with gradient guidance to accurately capture the dynamics of a latent representation of a high-dimensional system. Trained CNNs and GNNs map this latent representation to the full-high-dimensional representation of the system, adapting to diverse structured and unstructured spatial discretizations.
- 2. the integration of multi-level information, ensuring that the generated solutions maintain high fidelity to the underlying physical phenomena across different parameter settings.
- 3. demonstrations of the robustness and versatility of the framework in two case studies: an incompressible flow past a 2D cylinder at low Reynolds numbers on an unstructured mesh and an incompressible 3D turbulent channel flow on a structured mesh, both parameterized by the Reynolds number.

The remainder of this paper is organized as follows: Section 2 introduces the components of the proposed framework, including the CNN/GNN for structured/unstructured meshes for its latent space, the diffusion model, gradient guidance, and virtual observation. Section 3 presents the case studies and results, demonstrating the framework's versatility, while 3.2 highlights how the proposed generative model can handle very complex parametric systems. Finally, Section 4 concludes the paper with a discussion of our findings and potential future directions.

## 2. Methodology

The focus of this paper is on modeling the dynamics of a parameterized, large-scale system of equations that result from the discretization of PDEs. Our approach leverages both micro and macro-level models, structured and unstructured discretizations, and incorporates guidance and virtual observations to enhance the accuracy and efficiency of the model.

## 2.1. Micro system

We consider a parameterized, large-scale system of nonlinear equations, which we refer to as the microlevel model. Given a system parameter vector µ P R N µ , the goal is to approximately model the implicit distribution of the underlying dynamics using generative learning techniques. In more detail, we target the sequence U n len p µ q consists of n len snapshots u P R N u :

$$U _ { n _ { \text{len} } } ( \mu ) = [ \mathfrak { u } _ { 0 } ( \mu ), \mathfrak { u } _ { 1 } ( \mu ), \dots, \mathfrak { u } _ { n _ { \text{len} } - 1 } ( \mu ) ],$$

where each snapshot u i is implicitly parameterized by µ . This sequence is governed by the large-scale discrete dynamical system:

$$\mathbf u _ { 1 } ( \mu ) & = F ( \mathbf u _ { 0 } ( \mu ) ; \mu ),$$

where µ P R N µ also influences the dynamics of the sequence, and F : R N u ˆ R N µ Ñ R N u is a function describing the evolution of the state. The function F determines how the initial state u 0 p µ q progresses to the subsequent state u 1 p µ q , encapsulating the dynamical behavior of the system under the influence of the parameter µ . The spatial domain (Ω) on which the dynamical system of interest is solved is split into N c cells (Ω ) such that: c

$$\Omega = \bigcup _ { c = 1 } ^ { N _ { c } } \Omega _ { c },$$

N

where each cell Ω c is associated with a set of degrees of freedom (DoFs) denoted as v c P R v . Wenote that the micro-level model can be computationally expensive due to its nonlinearity, the large amount of spatial discretization points, and the necessity of small time increments. While the spatial discretization of the solution can be regularly structured for most geometries and PDEs, in general a varying or unstructured discretization is preferable as some areas require a finer discretization than others.

## 2.1.1. Structured Discretization

In a structured discretization, the state u is represented on a structured grid:

»

fi

$$\mathfrak { u } = \left [ \begin{array} { c c c c } v _ { 1 } & v _ { 2 } & \dots & v _ { N _ { w } - 1 } & v _ { N _ { w } } \\ v _ { N _ { w } + 1 } & v _ { N _ { w } + 2 } & \dots & v _ { 2 N _ { w } - 1 } & v _ { 2 N _ { w } } \\ \vdots & \vdots & \ddots & \vdots & \vdots \\ v _ { N _ { w } ( N _ { h } - 1 ) + 1 } & v _ { N _ { w } ( N _ { h } - 1 ) + 2 } & \dots & v _ { N _ { w } N _ { h } - 1 } & v _ { N _ { w } N _ { h } } \end{array} \right ] \in \mathbb { R } ^ { N _ { h } \times N _ { w } \times N _ { w } },$$

where N u can be factorized as N u ' N h ˆ N w ˆ N v . Here, N h and N w represent the number of elements along the height and width of the grid, respectively, while N v indicates the degrees of freedom per element. This structured grid allows for an well ordered representation of the state, facilitating straightforward convolutional operations and data manipulation.

## 2.1.2. Unstructured Discretization

Alternatively, in an unstructured discretization, the state u is represented in an unstructured format:

$$u = \left [ \begin{array} { c } v _ { 1 } \\ v _ { 2 } \\ \vdots \\ v _ { N _ { c } - 1 } \\ v _ { N _ { c } } \end{array} \right ] \in \mathbb { R } ^ { N _ { c } \times N _ { \nu } },$$

where N u can be factorized as N u ' N c ˆ N v . Here, N c denotes the number of unstructured cells, each with N v degrees of freedom. This approach is more flexible than structured discretization, allowing for complex geometries and varying element sizes, which can be advantageous in capturing intricate features of the solution domain.

## 2.2. Macro system

Directly learning the micro-level system can be computationally prohibitive due to the high dimensionality, nonlinearity, and the need for small time increments. To address these challenges, we convert the micro-level system to a macro-level representation. This conversion facilitates more efficient learning and computation by reducing the complexity of the system [17, 34]. Different discretization methods, whether structured or unstructured, can be employed in this conversion process to ensure that the macro-level model accurately captures the essential dynamics of the original micro-level system while significantly reducing computational costs.

## 2.2.1. Macro states of structured data

We assume the micro state u from a structured discretization lies in a low-dimensional subspace

$$\mathcal { V } _ { \theta _ { C N N } } = \left \{ D _ { \theta _ { C N N } } ( z ) \, \Big | \, z \in \mathbb { R } ^ { N _ { h _ { r } } \times N _ { w _ { r } } \times N _ { v _ { r } } } \right \} \, \subset \mathbb { R } ^ { N _ { h } \times N _ { w } \times N _ { v } },$$

where D θ CNN : R N hr ˆ N wr ˆ N v r Ñ R N h ˆ N w ˆ N v is a convolutional decoder that maps the macro state z to its micro state D θ CNN p z q with trainable parameter θ CNN , where N h r ! N ,N h w r ! N , w N v r N v ' O p 1 . q The parameter θ CNN represents the weights and bias parameters of the convolutional filter.

A convolutional layer applies a set of filters W P R F h ˆ F w ˆ F f ˆ F k and a bias vector b P R F k , where F , F h w , F , F f k are the filter height, filter width, the number of features, and the number of filters in the layer, respectively. The output of the convolutional layer given an input X P R H ˆ W ˆ F f , is given by:

Þ

$$X \mapsto Y \in \mathbb { R } ^ { H \times W \times F _ { k } } \colon Y _ { i, j, k } = \sum _ { c = 1 } ^ { F _ { t } } \sum _ { m = 1 } ^ { F _ { h } } \sum _ { n = 1 } ^ { F _ { w } } X _ { i + m - 1, j + n - 1, c } \cdot W _ { m, n, c, k } + b _ { k }.$$

In the decoder, the small height ( N h r ) and width ( N w r ) can be gradually increased to the original height ( N h ) and width ( N w ) by applying up-sampling before applying a standard convolution and non-linear activation functions [6].

To train the model, we let P train ' t µ , µ 1 2 , . . . u Ă R N µ be a collection of decoder training parameters and U train ' t U n len p µ 1 q , U n len p µ 2 q , . . . u be a collection of decoder training sequences. The optimal parameters θ ˚ CNN are obtained by solving the following optimization problem:

$$\theta _ { \text{CNN} } ^ { * } = \underset { \theta _ { \text{CNN} } } { \arg \min } \sum _ { U \in \mathcal { U } _ { \text{train} } } \sum _ { u \in U } | | u - D _ { \theta _ { \text{CNN} } } ( E ( u ) ) | |,$$

where E : R N h ˆ N w ˆ N v Ñ R N hr ˆ N wr ˆ N v r is a parameter-free down-sampler acting as the encoder.

## 2.2.2. Macro states of unstructured data

Next, we consider the micro state u from an unstructured discretization that lies in a low-dimensional subspace

$$\mathcal { V } _ { \theta _ { G N N } } = \left \{ D _ { \theta _ { G N N } } ( z ) \, \Big | \, z \in \mathbb { R } ^ { N _ { h _ { r } } \times N _ { w _ { r } } \times N _ { v _ { r } } } \right \} \subset \mathbb { R } ^ { N _ { c } \times N _ { v } },$$

where D θ GNN : R N hr ˆ N wr ˆ N v r Ñ R N c ˆ N v is a decoder based on a graph neural network with trainable parameter θ GNN .

The general convolutional operation over an unstructured mesh can be expressed as a message-passing process,

Þ

$$\text{process}, \quad & X = \left [ \begin{array} { c } x _ { 1 } \\ x _ { 2 } \\ \dots \\ x _ { N _ { c } - 1 } \\ x _ { N _ { c } } \end{array} \right ] \mapsto Y = \left [ \begin{array} { c } y _ { 1 } \\ y _ { 2 } \\ \dots \\ y _ { N _ { c } - 1 } \\ y _ { N _ { c } } \end{array} \right ] \colon y _ { i } = \gamma _ { \theta _ { \text{NN} } } ( x _ { i }, \oplus _ { j \in \mathcal { N } ( i ) } \phi _ { \text{NN} } ( x _ { i }, x _ { j }, e _ { j, i } ) ),$$

where ' denotes a differentiable, permutation-invariant function, such as sum, mean, or max, e P R N e are edge features from cell j to cell i , and ϕ θ GNN : R N x ˆ R N x ˆ R N e Ñ R N ϕ is a differentiable function that processes the node pair and their edge interaction. The function γ θ GNN : R N x ˆ R N ϕ Ñ R N y updates the node feature.

To optimize the parameters of the GNN, we define θ ˚ GNN as the solution to the following optimization problem:

$$\theta _ { \text{GNN} } ^ { * } = \underset { \theta _ { \text{GNN} } } { \arg \min } \sum _ { U \in \mathcal { U } _ { \text{train} } } \sum _ { u \in U } \| u - D _ { \theta _ { \text{GNN} } } ( E _ { \theta _ { \text{GNN} } } ( u ) ) \|,$$

where the construction of the encoder and decoder is based on the methodology described in [7].

## 2.3. Generative Learning with Diffusion Modeling

Given the encoder and decoder introduced in the previous sections, we can map the micro state sequence U n len p µ q to a low-dimensional latent space representation Z n len p µ q as follows:

$$Z _ { n _ { \text{len} } } ( \mu ) = [ z _ { 0 } ( \mu ), z _ { 1 } ( \mu ), \dots, z _ { n _ { \text{len} } - 1 } ( \mu ) ] \in \mathbb { R } ^ { N _ { h _ { r } } \times N _ { w _ { r } } \times N _ { w _ { r } } \times n _ { \text{len} } },$$

where each z i p µ q ' E θ ˚ p u p µ qq is obtained through the encoder E θ ˚ . By performing diffusion learning on the macro level, we achieve computational efficiency. The forward process, which models the addition of noise on the macro level, is expressed as:

$$\tilde { Z } _ { n _ { i e n } } ^ { ( i ) } ( \mu ) \sim q _ { i } ^ { [ \mu ] } \Big ( \tilde { Z } _ { n _ { i e n } } ^ { ( i ) } ( \mu ) \Big | Z _ { n _ { i e n } } ( \mu ) \Big ) \coloneqq \mathcal { N } \Big ( \Big { Z } _ { n _ { i e n } } ( \mu ), \sigma _ { i } ^ { 2 } \mathcal { I } ), \ \ i = 1, \dots, N _ { n o i s e },$$

where I P R N hr N wr N v r n len ˆ N hr N wr N v r n len is the identity matrix. This step-by-step addition of Gaussian noise ensures that the data distribution becomes more tractable, simplifying the training of the generative model. Conversely, the reverse process involves denoising, leveraging a neural network to iteratively refine the noisy latent representations back to their original form. This bidirectional process not only aids in learning the underlying data distribution but also enhances the robustness of the model, enabling it to generate highquality samples. The reverse process, aimed at denoising and sampling, is defined as:

$$p ^ { [ \mu ] } \Big ( \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big | Z _ { n _ { \text{len} } } ( \mu ), \tilde { Z } _ { n _ { \text{len} } } ^ { ( i + 1 ) } ( \mu ) \Big ) = \mathcal { N } \Big ( \frac { \sigma _ { i + 1 } ^ { 2 } - \sigma _ { i } ^ { 2 } } { \sigma _ { i + 1 } ^ { 2 } } Z _ { n _ { \text{len} } } ( \mu ) + \frac { \sigma _ { i } ^ { 2 } } { \sigma _ { i + 1 } ^ { 2 } } \tilde { Z } _ { n _ { \text{len} } } ^ { ( i + 1 ) } ( \mu ), \frac { ( \sigma _ { i + 1 } ^ { 2 } - \sigma _ { i } ^ { 2 } ) \sigma _ { i } ^ { 2 } } { \sigma _ { i + 1 } ^ { 2 } } \Big ), \quad ( 1 4 )$$

To approximate the reverse process, we use a neural network parameterized by θ diff :

$$p _ { \theta _ { \text{diff} } } ^ { [ \mu ] } \left ( \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big | Z _ { n _ { \text{len} } } ( \mu ), \tilde { Z } _ { n _ { \text{len} } } ^ { ( i + 1 ) } ( \mu ), i \right ) = & \, \mathcal { N } \left ( \frac { \sigma _ { i + 1 } ^ { 2 } - \sigma _ { i } ^ { 2 } } { \sigma _ { i + 1 } ^ { 2 } } \hat { Z } _ { n _ { \text{len} }, \theta _ { \text{diff} } } \left ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i + 1 ) } ( \mu ), i \right ) + \frac { \sigma _ { i } ^ { 2 } } { \sigma _ { i + 1 } ^ { 2 } } \tilde { Z } _ { n _ { \text{len} } } ^ { ( i + 1 ) } ( \mu ), \frac { ( \sigma _ { i + 1 } ^ { 2 } - \sigma _ { i } ^ { 2 } ) \sigma _ { i } ^ { 2 } } { \sigma _ { i + 1 } ^ { 2 } } \right ), \\, \quad \hat { \alpha } \quad \hat { \nu } \quad \hat { \nu } \quad \hat { \nu } \quad \mathcal { N } \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \ \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \mathbb { m } N \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot \quad \cdot$$

where ˆ Z n len , θ diff ` µ, ˜ Z p ` q i 1 n len p µ , i q ˘ : R N µ ˆ R N hr ˆ N wr ˆ N v r ˆ n len ˆ N Ñ R N hr ˆ N wr ˆ N v r ˆ n len is a neural network parametrized by θ diff . The optimal parameters θ ˚ diff are obtained by solving the following optimization problem [29]:

$$\theta _ { \text{diff} } ^ { * } = \arg \min _ { \theta _ { \text{diff} } } \sum _ { \mu \in \mathcal { P } _ { \text{train} } } \mathbb { E } _ { i \sim \text{Uniform} ( 1, N _ { \text{noise} } ) } \Big [ \Big \| \hat { Z } _ { n _ { \text{en} } }, \theta _ { \text{diff} } \left ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i + 1 ) } ( \mu ), i \right ) - Z _ { n _ { \text{len} } } ( \mu ) \Big \| _ { 2 } ^ { 2 } \Big ].$$

After training, the term ˆ Z n len , θ ˚ diff p µ, ˜ Z p q i n len , i q represents the estimated latent representation given ˜ Z p q i n len .

## 2.4. Multi-resolution guidance and virtual guidance

During the reverse process, the score function is modified to incorporate guidance, ensuring that the generated samples adhere to desired properties or constraints.

## 2.4.1. Gradient guidance for diffusion models

The gradient of the log probability density (score) function is approximated as follows:

$$\nabla _ { \tilde { Z } } \log p ^ { [ \mu ] } \Big { ( } \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big { ) } \approx \frac { \hat { Z } _ { n _ { \text{len} } }, \theta _ { \text{diff} } ^ { * } \Big { ( } \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i \Big { ) } - \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) } { \sigma _ { i } ^ { 2 } }.$$

This equation represents the originally-learned score function, where the guidance term will modify the gradient to reflect the desired properties of the generated samples. To guide the generation process, physical information can be incorporated. The residual term which represents the mismatch between the sample and observation is defined as,

$$\hat { R } \left ( \hat { Z } _ { n _ { \text{len} }, \text{o} _ { \text{diff} } ^ { \ast } } \left ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i \right ), \mu \right ) = H \left ( \hat { Z } _ { n _ { \text{len} }, \text{o} _ { \text{diff} } ^ { \ast } } \left ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i \right ) \right ) - P _ { \text{obs} } ( \mu ),$$

where H : R N hr ˆ N wr ˆ N v r ˆ n len Ñ R N obs maps the macro state to a vector that is implicitly governed by the parameter µ via P obs : R N µ Ñ R N obs . To guide the latent representations to satisfy the physical constraints, we then define the virtual likelihood [11] as follows:

$$p ^ { [ \mu ] } \Big ( \hat { R } \Big ( \hat { Z } _ { n _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \i I _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \i i _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \iota _ { \i i _ { \iota _ { \iota _ { \iota _ { \i i _ { \iota _ { \iota _ { \i i _ { \iota _ { \i i _ { \iota _ { \i i _ { \iota _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ \i i _ { \i i _ { \i i _ { \i i _ { \i i _ \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ { \i i _ \i i _ \i i _ { \i i _ { \i i _ { \i i _ { \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i _ \i i } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } { } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } _ } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } \ } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }. } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }, } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } \\ } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ( } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } | } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } & } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }) } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } ; } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } }
 } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } } \$$

Combining the virtual likelihood and the Bayes law, we obtain the original score function with the guidance term for the guided reverse process:

$$& p ^ { [ \mu ] } \Big ( \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big | \hat { R } \Big ( \hat { Z } _ { n _ { \text{len} }, \theta _ { \text{diff} } ^ { * } } ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i ), \mu \Big ) = 0 \Big ) \varnothing \\ & p ^ { [ \mu ] } \Big ( \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big ) \cdot p ^ { [ \mu ] } \Big ( \hat { R } \Big ( \hat { Z } _ { n _ { \text{len} }, \theta _ { \text{diff} } ^ { * } } ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i ), \mu \Big ) = 0 \Big | \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big ).$$

Then, the combined score function with guidance can be expressed as:

$$& \nabla _ { \tilde { Z } } \log p ^ { [ \mu ] } \Big ( \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big | \tilde { R } \Big ( \hat { Z } _ { n _ { \text{len} } }, \theta _ { \text{diff} } ^ { * } ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i ), \mu \Big ) = 0 \Big ) = \\ & \nabla _ { \tilde { Z } } \log p ^ { [ \mu ] } \Big ( \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big ) + \nabla _ { \tilde { Z } } \log p ^ { [ \mu ] } \Big ( \hat { R } \Big ( \tilde { Z } _ { n _ { \text{len} } }, \theta _ { \text{diff} } ^ { * } ( \mu, \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ), i ), \mu \Big ) = 0 \Big | \tilde { Z } _ { n _ { \text{len} } } ^ { ( i ) } ( \mu ) \Big ).$$

We calculate the gradient of the log probability of the guidance term using automatic differentiation,

$$& \nabla _ { \tilde { Z } } \log p ^ { \{ \mu \} } \Big ( \hat { R } \Big ( \tilde { Z } _ { n _ { \ln }, \theta _ { \text{diff} } ^ { * } } ( \mu, \tilde { Z } _ { n _ { \ln } } ^ { ( i ) } ( \mu ), i ), \mu \Big ) = 0 \Big | \tilde { Z } _ { n _ { \ln }, \theta } ^ { ( i ) } ( \mu ) \Big ) = \\ & \quad - \frac { 1 } { \sigma _ { R } ^ { 2 } } \hat { R } \Big ( \hat { Z } _ { n _ { \ln }, \theta _ { \text{diff} } ^ { * } } ( \mu, \tilde { Z } _ { n _ { \ln }, \theta } ^ { ( i ) } ( \mu ), i ), \mu \Big ) \frac { \partial H \Big ( \hat { Z } _ { n _ { \ln }, \theta _ { \text{diff} } ^ { * } } ( \mu, \tilde { Z } _ { n _ { \ln }, \theta } ^ { ( i ) } ( \mu ), i ) \Big ) } { \partial \tilde { Z } _ { n _ { \ln }, \theta } ^ { ( i ) } ( \mu ) }.$$

## 2.4.2. Multi-resolution information

In the multi-resolution guidance framework, we provide guidance at both micro and macro levels to ensure that the generated samples adhere to the desired properties across different scales. The mapping functions for both levels of guidance are defined as follows:

Þ

Þ

$$H \colon \begin{cases} Z \mapsto G _ { \text{readout} } ^ { \text{micro} } ( D _ { \theta ^ { * } } ( \mathbf Z ) ) & \text{micro-level guidance}, \\ Z \mapsto G _ { \text{readout} } ^ { \text{macro} } ( \mathbf Z ) & \text{macro-level guidance}. \end{cases}$$

Here, H represents the mapping function that transforms the latent space representation Z into either a micro-level or macro-level guidance signal. The micro-level guidance is obtained through the decoder D θ ˚ , which maps Z back to the micro-level state before applying the readout function G micro readout . On the other hand, the macro-level guidance directly applies the readout function G macro readout to the latent representation Z .

The observational data P obs is also mapped for both micro and macro levels to provide consistent guid-

ance:

Þ

Þ

$$P _ { \text{obs} } \colon \begin{cases} \mu \mapsto G _ { \text{readout} } ^ { \text{micro} } ( U _ { n _ { \text{len} } } ( \mu ) ) & \text{micro-level guidance,} \\ \mu \mapsto G _ { \text{readout} } ^ { \text{macro} } ( E _ { \text{o} ^ { * } } ( U _ { n _ { \text{len} } } ( \mu ) ) ) & \text{macro-level guidance.} \end{cases}$$

In this context, P obs maps the system parameter µ to the observed data, providing the reference for guidance at both levels. The micro-level observation G micro readout p U n len p µ qq is derived directly from the micro state sequence U n len p µ q . Conversely, the macro-level observation G macro readout p E θ ˚ p U n len p µ qqq is obtained by first encoding the micro state sequence into the latent space and then applying the macro-level readout function.

This multi-resolution approach ensures that the model captures the essential dynamics and constraints at both detailed (micro) and coarse (macro) scales, thus improving the fidelity and robustness of the generative process.

## 2.4.3. Virtual observation

The concept of virtual observation is employed to ensure that the generated samples closely follow the observed data distribution. In many scenarios, direct observations may be sparse or incomplete, making it challenging to train the generative model effectively. To address this, we leverage other models, such as multilayer perceptrons (MLP) or Gaussian processes, to predict these properties as virtual observations. The virtual observation is used to approximate the true observation as

$$\hat { P } _ { \text{obs}, \theta ^ { * } } ( \mu ) \approx G _ { \text{readout} } ^ { \text{micro} } ( U _ { n _ { \text{len} } } ( \mu ) ).$$

Here, ˆ P obs , θ ˚ p µ q represents the estimated observation given the parameter µ , which is trained by applying the micro-level readout function G micro readout to the training micro state sequence U n len p µ q . In cases where direct observational data for a unseen parameter is not available, these predictions can be generated using pre-trained models or Gaussian processes that capture the relevant system properties. By integrating such predictive models, our framework is well-equipped to leverage existing knowledge and models to generate high-fidelity virtual observations. This not only enriches the training data for the generative model but also enhances the robustness and accuracy of the generated samples. The flexibility of using different predictive models allows us to incorporate a wide range of prior knowledge, making the generative process more reliable and efficient.

## 3. Results

In this section, we demonstrate the effectiveness of the proposed generative framework through two case studies: incompressible laminar cylinder flow on an unstructured mesh (Section 3.1) and incompressible turbulent channel flow on a structured mesh (Section 3.2), both parameterized by the Reynolds number. In Section 3.1, we focus on evaluating the robustness and versatility of the framework under various settings to illustrate its capability to directly generate, forecast, and reconstruct flow sequences across different Reynolds-numbers. Additionally, in Section 3.2, we emphasize how the framework leverages multi-level information from virtual observations to guide the generative process for complex dynamical systems.

## 3.1. Laminar cylinder flow

In this test case, we aim to generate laminar flow over a cylinder at Reynolds number between Re P r 100 388 . , s The physical problem involves simulating an incompressible flow around an 2-D cylinder. The rectangular simulation domain ( r´ 4 30 , s ˆ r´ 5 5 ) is , s discretized with an unstructured mesh consisting of 11644 simplex cells, with a cylinder of 0.5 unit radius positioned at the origin and a uniform inlet velocity of 1 unit that remains constant throughout the entire simulation (Figure 1). The training and testing datasets are generated using the DNS simulator [37], employing the numerical methods described in [32]. The training dataset comprises 25 fluid flow simulations, while the testing dataset consists of 24 simulations, each with 150 macro time-steps and a physical time-step size of ∆ t ' 1.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-faddbbb1-f9a3-4cdf-97a0-dc3f4d2b5894.jpg)

x

Figure 1: A general overview of the unstructured mesh (11644 cells) for cylinder flow.

## 3.1.1. Macro state Identification

Applying CNNs to unstructured domains results in inefficient data representation and inflates the required computational resources. CNNs are optimized for regular grids, and their application to irregular structures leads to computational overhead and suboptimal performance [6, 33]. Therefore, we employ the GNN autoencoder as defined in (10), where γ θ ˚ GNN and ϕ θ ˚ GNN in each GNN layer are implemented as simple MLPs with 128 neurons and residual connections. The encoder and decoder each consist of three GNN layers. The nodes at the macro level are non-uniformly distributed to effectively capture and resolve the complex flow patterns around the cylinder. Figure 2 illustrates the encoded graph for the macro states, where the 1024 nodes with 4-dimensional hidden node feature are arranged as N h r ˆ N w r ˆ N v r ' 32 ˆ 32 ˆ 4 in (12). This arrangement allows us to directly utilize the backbone of the diffusion model based on CNN.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-9c8eaf4b-f92e-4259-bfd5-1ef0f9edb08e.jpg)

x

Figure 2: The reduced graph (1024 nodes) from the original mesh in Figure 1 using the GNN auto-encoder.

Although the macro-level state lacks a direct physical interpretation, the diffusion model is trained and generates outputs at this level. To provide insight into the appearance of the macro-state, Figure 3 presents an example of the micro-level state, its corresponding macro-level state, and the output of the decoder which maps the macro-level state back to the micro-level.

## 3.1.2. Generation of Flow Sequences at Different Reynolds-numbers

We evaluate the performance of the generative framework across a range of Reynolds numbers not seen during training. At very low Reynolds numbers, the wake behind the cylinder exhibits longer and more stable tails, indicative of a more laminar flow regime. In contrast, at higher Reynolds numbers, the wake shortens and becomes more complex, with smaller-scale vortices emerging in the flow (Figure 4, A.12). A speed-up factor of over 500 compared to the simulation cost is achieved. These results highlight the model's

Figure 3: An overview of a micro-level state ( u ), its corresponding macro-level state ( z ) and decoded back to its micro-level ( D θ ˚ CNN p z q ): velocity magnitude ( top row, left ), pressure ( top row, right ), macro-level state components 1 to 4 ( middle row, from left to right ), and decoded velocity magnitude ( bottom row, left ), decoded pressure ( bottom row, right ). For visualization purposes, the colorbar is omitted.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-05b1bd21-1643-41c9-8ca4-3ff4240ebacc.jpg)

ability to accurately reproduce the characteristic flow patterns associated with different Reynolds numbers, thereby validating its robustness and effectiveness in generating fluid flow sequences parameterized by the Reynolds-number.

## 3.1.3. Forecasting

The proposed framework is also capable of making short-term predictions. Given a set of initial states (in this study, we utilize 10 states as the initial conditions) and the corresponding parameters, the generative model demonstrates its robustness by eliminating the need for retraining. This is accomplished by evaluating the encoder and defining the macro-level readout function as simply selecting the first several macro states from the generative sequence. The process of enforcing the initial state leverages guided diffusion to ensure that the generated initial velocity and pressure fields align accurately with the provided initial conditions. Figures 5 and A.13 illustrate that the guided initial velocity and pressure fields at all tested Reynolds numbers exhibit a high degree of accuracy, closely matching the expected physical states. Following this initialization, the framework proceeds to forecast the evolution of the flow, including critical quantities such as force coefficients. As depicted in Figure A.14, the model reasonably predicts the force coefficients across various Reynolds numbers, effectively capturing the transition from the laminar developing state to the vortex shedding state. This includes predicting the symmetry-breaking phenomena inherent in these fluid flow regimes. Thus, the framework's ability to maintain fidelity to the physical system while providing substantial computational efficiency is promising.

## 3.1.4. Sparse reconstruction

Next, we perform sparse reconstruction for different test Reynolds numbers. Sparse reconstruction involves using a very limited number of observation points-fewer than the pivotal points used for the encoder (Figure A.15). To tackle this challenge, we employ the same generative model as in the previous scenario, without any additional retraining, and extend the sequence length to 150 steps to capture the flow dynamics over a longer period. The guidance framework remains consistent with the gradient method, but we redefine the micro readout function to select the sparse observation points from the generated micro states. This

Diffusion model at

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-74341072-8565-4166-87f5-f09856378a0e.jpg)

Re

'

106

DNS at

Re

'

106

Figure 4: Velocity magnitude of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

Guided initial state for dynamics at Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-d0e276d7-717c-49fe-9a84-6abcc62758b1.jpg)

´

Guided initial state for dynamics at Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-8bfc7fde-ed01-4b40-9c11-829337bc91b3.jpg)

´

5

Initial state for dynamics at Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-9ab2f950-2efd-4e92-8886-89b3a0bf0442.jpg)

Initial state for dynamics at Re ' 238

Guided initial state for dynamics at Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-8f67bc4b-297a-4a53-be2f-4bcdf38812ee.jpg)

Initial state for dynamics at Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-fc11bb2b-cf59-4a0d-9d33-4ba9f9cec4f6.jpg)

Figure 5: Initial velocity magnitude of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

involves leveraging the decoder to map these points back to the macro level. By doing so, we aim to ensure that the framework can still effectively reconstruct the flow fields, despite the limited observational data.

The proposed framework is capable of stably reconstructing the flow field with reasonable accuracy compared to the synthetic truth obtained from DNS. It effectively manages parametric variations that cause significant differences in the flow field from the initial state to the shedding states. Figures 6, A.16, A.17, and A.18 illustrate the flow reconstruction and corresponding mesh at all considered Reynolds numbers. These visualizations confirm that, at all time steps, the reconstructed flow fields align reasonably well with the DNS results. Notably, the zero-mean force coefficient also closely matches the synthetic truth, accurately reflecting the symmetry-breaking phenomenon, as shown in Figure 7. Overall, the framework's reliably reconstructs the flow fields with high fidelity, despite the sparse observational data. It is particularly beneficial in scenarios where data collection is limited or expensive, enabling accurate flow predictions and analysis. The successful handling of diverse flow conditions further illustrates its potential for other inverse problems in fluid dynamics and related fields.

Figure 6: Velocity magnitude of the viscous flow at three test Reynolds numbers, as reconstructed by the diffusion model and compared with results from DNS.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-cd6ea64e-b4e1-422d-8835-5c8c4bdd043b.jpg)

## 3.2. Guided generation with virtual observation for turbulent channel flow

We examine the performance of the framework in turbulent channel flows. Different Reynolds numbers significantly affect the turbulence structures in the spatiotemporal domain. Accurately modeling and predicting turbulence statistics with eddy-resolving simulations require intractable computational resources, positioning the proposed framework as a promising alternative for fast modeling. To this end, the generative framework is applied to learn from high-fidelity DNS data, enabling the efficient generation of realistic turbulent flows at different unseen Reynolds numbers with significant speedup. Our goal is to generate time-coherent, realistic instantaneous velocity fields at cross-sectional (streamwise-wallnormal) spatiotemporal locations. The training data originates from a fully resolved 3D transient DNS of wall-bounded turbulence over a channel, at Re τ P r 180 605 . , s We subsample exclusively during the fully developed phase

Figure 7: Reconstructed lift coefficients for various Reynolds numbers: comparison between the diffusion model ( ) and DNS results ( ).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-bff0cc99-5879-4003-9b11-6c8548a8b781.jpg)

of flow using a macro time step of ∆ t ' 1. A simple downsampling (non-trainable) method is applied as the encoder, resulting in spatiotemporal flow sequences at the macro level consisting of 50 snapshots ( Z 50 p Re τ q ' r z 0 p Re τ q , ..., z 49 p Re τ qs ) with a resolution of N h r ˆ N w r ˆ N v r ' 32 ˆ 32 ˆ 3 from the micro resolution ( U 50 p Re τ q ' r u 0 p Re τ q , ..., u 49 p Re τ qs ) N h ˆ N w ˆ N v ' 256 ˆ 256 ˆ 3. The focus is on generating sequences of time length equal to 50. During training, 80% of the database at different Re τ is used for training the diffusion model and virtual observations of stress tensor, while the remaining 20% of unseen Re τ is reserved as the test set for conditional generation with virtual observation.

Diffusion model at Re τ ' 226

DNS at Re τ ' 226

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-e7dd397a-0b53-4ccd-bce1-4ca338898af4.jpg)

Figure 8: Streamwise velocity of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

Figures 8, B.19, and B.20 illustrate three instances of flow generated by the proposed framework. These flow trajectory samples are visualized through velocity contours, providing a detailed view of the wallbounded turbulence characteristics within the domain. Notably, our proposed generative framework demonstrates the capability to effectively generate parametric flow features at previously unseen Reynolds numbers, achieving a computational speedup of over 350 while closely approximating the complex flow fields observed in DNS. As the Reynolds number ( Re τ ) increases in turbulent flow, the velocity contours for all three components become more complex and turbulent. Higher Re τ leads to increased turbulence intensity, resulting in more pronounced fluctuations in the velocity field. The flow structures become finer and more intricate, with sharper gradients near the wall due to thinner boundary layers. Additionally, enhanced mixing in the flow results in a more uniform velocity distribution away from the wall. Overall, increasing Re τ produces more detailed and irregular velocity contours, reflecting the heightened turbulence and energy cascade to smaller scales. Our generative model successfully captures these complex, unseen flow fields.

The parametric turbulence generation results of the proposed framework are compared with DNS reference in Figure 9, illustrating both the fidelity and diversity of the diffusion-generated spatiotemporal velocity field samples. For this assessment, an ensemble of 256 flow sequences, each with 50 snapshots, was synthesized to ensure statistical convergence. As shown in the figure, the turbulence statistics obtained from our model

Figure 9: Turbulence statistics: streamwise mean from DNS ( ) and diffusion model ( ); streamwise fluctuation from DNS ( ) and diffusion model ( ); wallnormal fluctuation from DNS ( ) and diffusion model ( ); spanwise fluctuation from DNS ( ) and diffusion model ( ); non-zero cross fluctuation from DNS ( ) and diffusion model ( )

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-27a2f3f0-6ee6-4269-88b4-cda796c3e3ad.jpg)

are in reasonably good agreement with those obtained by DNS. We note that these results were obtained for untrained Reynolds numbers ( Re τ ). The mean streamwise velocity profile generated matches the DNS reasonably well, reflecting the expected behavior across the linear viscous sublayer. As the Reynolds number ( Re τ ) increases, the mean velocity profile versus y ` (the dimensionless wall-normal distance) shows a more pronounced logarithmic region, indicating higher velocities away from the wall due to increased turbulence. The viscous sublayer becomes thinner, and the buffer layer shifts, reflecting stronger velocity gradients near the wall. Velocity fluctuations for wallnormal and spanwise velocity versus y ` also increase with higher Re τ , particularly in the near-wall region, due to intensified turbulent activity. These fluctuations extend further into the log layer, indicating more vigorous mixing and energy transfer across the flow. Overall, higher Re τ results in steeper mean velocity gradients near the wall and increased turbulence intensity throughout the boundary layer, leading to more complex and energetic flow characteristics. The relative good match in the figure shows that the generative framework is able to parametrically capture these statistics accurately.

Figures 10, B.21, and B.22 present the spatial and temporal correlations at different unseen Re τ compared with DNS data. Although not perfect, the generated samples exhibit a correlation decay trend similar to that observed in the DNS data. As the Reynolds number ( Re τ ) increases in channel flow, both temporal and spatial correlations undergo significant changes. Temporally, the correlation time scales become shorter due to the increased turbulence intensity, resulting in more rapid decorrelation of flow structures. This behavior reflects the faster dynamics and more frequent changes in the turbulent flow field. Spatially, the correlation lengths decrease, particularly in the wall-normal and spanwise directions. The increased Reynolds number leads to finer turbulent structures and more complex flow patterns, causing the flow to decorrelate over shorter distances. Additionally, the enhanced mixing and energy transfer at higher Re τ contribute to the more rapid loss of spatial coherence in the velocity field. Overall, the generative framework effectively captures the relationship between higher Re τ values and the resulting shorter temporal and spatial correlations, thereby indicating more dynamic and spatially complex turbulence.

Figures 11 illustrate the energy spectrum for generated sequences and DNS data. Our generative framework successfully captures the changes in the energy spectrum as the Reynolds number ( Re τ ) increases. In the wall-normal and spanwise direction, the energy spectrum shows a shift towards higher frequencies, indicating the presence of smaller-scale turbulent structures. This shift is attributed to the increased turbulence intensity, leading to a broader distribution of energy across various scales. The inertial subrange extends, demonstrating a more efficient energy cascade that transfers energy from larger to smaller scales more rapidly. Overall, higher Re τ results in a more extensive range of turbulent scales and a more detailed and energetic velocity field, as evidenced by the energy spectrum in both the wall-normal and spanwise directions.

Figure 10: Streamwise spatial and temporal correlation at y ` ' 20 from DNS ( ) and diffusion model ( ); at y ` ' 50 from DNS ( ) and diffusion model ( ); at y ` ' 100 from DNS ( ) and diffusion model ( ).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-2be4b34d-51e8-4ec9-9635-a43e64fc86fc.jpg)

Figure 11: Energy spectrum of velocity at y ` ' 20 from DNS ( ) and diffusion model ( ); at y ` ' 50 from DNS ( ) and diffusion model ( ); at y ` ' 100 from DNS ( ) and diffusion model ( ).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-258ab20f-6d04-4460-aa3f-fa5034cd5660.jpg)

## 4. Conclusion

In this work, we present a novel generative learning framework for parametric high-dimensional systems using diffusion models with gradient guidance and virtual observations. Our approach addresses the complexities and computational challenges inherent in simulating large-scale nonlinear systems with high parametric dependencies. The framework employs various encoder-decoder architectures to elevate the generative learning capabilities of diffusion models to the macro level. By integrating both micro and macro-level information, along with virtual observations, our framework achieves versatility and effectiveness in generating complex parametric systems.

We demonstrated the effectiveness of our framework through two case studies: incompressible, twodimensional laminar flow past a cylinder on an unstructured mesh and incompressible turbulent channel flow on a structured mesh, both parameterized by the Reynolds number. The results illustrate the robustness of the framework across different settings, showcasing its capability to generate flow sequences directly across different parameters. By utilizing multi-level information to guide the generative process, the present framework enhances the fidelity and accuracy of the generated flow sequences. In the flow past a cylinder case, it successfully captured the characteristic flow patterns associated with different Reynolds numbers. In the more challenging turbulent channel flow scenario, the framework accurately generated dynamics that matched DNS results, reflecting the complex and energetic characteristics of turbulent flows at varying Reynolds numbers and underscoring the model's capability to manage high-dimensional and dynamic turbulence phenomena.

Overall, the proposed generative framework showed promising results in efficiently and accurately modeling high-dimensional parametric systems. Its ability to integrate gradient guidance and virtual observations allows it to adapt to different scenarios and parameter variations without the need for retraining. This flexibility and efficiency make it a valuable tool for various applications in fluid dynamics and other fields requiring the solution of high-dimensional, parametric PDEs.

## Appendix A. Additional results for cylinder flow

Figure A.12: Pressure of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-0b07ed28-e5a2-4196-a936-8db2949eb487.jpg)

Guided initial state for dynamics at Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-b72eaf87-7a38-477e-96f8-c14ca0f1e24c.jpg)

´

Guided initial state for dynamics at Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-192aa6a7-178c-476e-8c1e-304741a04e74.jpg)

´

Initial state for dynamics at Re ' 106

Initial state for dynamics at Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-80aa34b6-4511-4cb6-b0bb-df9cdbfbcf19.jpg)

Guided initial state for dynamics at Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-9398afbe-c687-40f0-bea7-1db62002c3d2.jpg)

Initial state for dynamics at Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-62778905-4a3c-4bae-aab1-7ca017fb6e18.jpg)

Figure A.13: Initial pressure of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

Figure A.14: Drag coefficient from DNS ( ), from the diffusion model ( ); Lift coefficient from DNS ( ), from the diffusion model ( )

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-5a0a84d0-5d80-4712-9dec-ec70519af1e6.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-c034bf46-ae7b-4990-b77a-403096ae5a18.jpg)

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-9c78e0d1-e15b-4c14-aac1-8386bc4606f1.jpg)

Figure A.15: A sparse observation set consisting of only 256 points (fewer than those 1024 pivotal points shown in Figure 2 for the encoder) is utilized for the reconstruction of 11,644 cells depicted in Figure 1.

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-86869935-396e-4acc-8a82-fe44777e5d99.jpg)

## Reconstruction at t ' 50 for Re ' 106

Truth at t ' 50 for Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-8103c328-3058-47ca-9ecf-d93c7a217afd.jpg)

´

5

Reconstruction at t ' 50 for Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-cc929de5-269b-4125-88ee-d66a9e1025b9.jpg)

´

Truth at t ' 50 for Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-9be48664-3f83-4237-9be9-f4aeaae78385.jpg)

Reconstruction at t ' 50 for Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-519b788b-1a9a-4128-b98a-44293fa88f03.jpg)

Truth at t ' 50 for Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-3893da62-bbe2-4a8d-abc2-0d219a3cf923.jpg)

Figure A.16: Velocity magnitude of the viscous flow at three test Reynolds numbers, as reconstructed by the diffusion model and compared with results from DNS.

Reconstruction at t ' 100 for Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-963ea050-9b13-4eaf-988c-71ab10e74396.jpg)

´

5

Reconstruction at t ' 100 for Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-dde3eca0-9a4e-415d-853b-6ed1c7da5160.jpg)

´

Truth at t ' 100 for Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-a7decd00-67e6-4076-a054-189e0e3364fc.jpg)

Truth at t ' 100 for Re ' 238

Reconstruction at t ' 100 for Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-6e8e6aad-9731-49b8-957f-dadcdb15f019.jpg)

Truth at t ' 100 for Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-3f76448b-5bb3-449f-bc73-1133006d599b.jpg)

Figure A.17: Velocity magnitude of the viscous flow at three test Reynolds numbers, as reconstructed by the diffusion model and compared with results from DNS.

Reconstruction at t ' 150 for Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-060e9474-793d-435c-8da0-a0885d29b35d.jpg)

´

5

Reconstruction at t ' 150 for Re ' 238

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-95aa1687-49d4-495f-8d30-aaad8c4046da.jpg)

´

Truth at t ' 150 for Re ' 106

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-438253fe-f432-4a5c-99d2-606ef74a6808.jpg)

Truth at t ' 150 for Re ' 238

Reconstruction at t ' 150 for Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-70b86f09-c41d-46b9-aa31-13621596503a.jpg)

Truth at t ' 150 for Re ' 382

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-8d2030b8-d00b-4b16-b8f4-34e78d81f622.jpg)

Figure A.18: Velocity magnitude of the viscous flow at three test Reynolds numbers, as reconstructed by the diffusion model and compared with results from DNS.

## Appendix B. Additional results for channel flow

Diffusion model at

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-d81c7a18-06c7-46f1-ab1f-a30a780d28d2.jpg)

Re

226

Figure B.19: Wallnormal velocity of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

## Diffusion model at Re τ ' 226

DNS at Re τ ' 226

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-b7440950-7f92-41c0-87e7-90e79cc35404.jpg)

Figure B.20: Spanwise velocity of the viscous flow at three test Reynolds numbers, as generated by the diffusion model and compared with results from DNS.

Figure B.21: Wallnormal velocity spatial and temporal correlation at y ` ' 20 from DNS ( ) and diffusion model ( ); at y ` ' 50 from DNS ( ) and diffusion model ( ); at y ` ' 100 from DNS ( ) and diffusion model ( ).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-82f73444-dd19-4ea6-b0e5-37f1c73a98b8.jpg)

Figure B.22: Spanwise velocity spatial and temporal correlation at y ` ' 20 from DNS ( ) and diffusion model ( ); at y ` ' 50 from DNS ( ) and diffusion model ( ); at y ` ' 100 from DNS ( ) and diffusion model ( ).

![Image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00157v1-figure-3775bfcf-e3c3-4050-a62e-528ba6b320be.jpg)

## Appendix C. Model settings

Table C.1: Important hyperparameters of for the diffusion model.

| hyperparameters                          | Cylinder flow   | Channel flow    |
|------------------------------------------|-----------------|-----------------|
| number of channels in convolution layers | (32,64,128,256) | (32,64,128,256) |
| number of noise step ( N noise )         | 20              | 20              |
| noise range in diffusion model           | (0.002,80)      | (0.002,80)      |

Table C.2: Important hyperparameters of for the encoders and decoders.

| hyperparameters                      | Cylinder flow   | Channel flow         |
|--------------------------------------|-----------------|----------------------|
| number of channels in encoder layers | (128,128,128)   | None                 |
| number of channels in decoder layers | (128,128,128)   | (64,64)              |
| virtual observation model            | None            | MLP (1,64,64,64,128) |

## Acknowledgments

S.K. and P.K. acknowledge support by The European High Performance Computing Joint Undertaking (EuroHPC) Grant DCoMEX (956201-H2020-JTI-EuroHPC-2019-1).

## References

- [1] Steven L Brunton, Bernd R Noack, and Petros Koumoutsakos. Machine learning for fluid mechanics. Annual Review of Fluid Mechanics, 52, 2019.
- [2] National Research Council. A National Strategy for Advancing Climate Modeling. The National Academies Press, 2012.
- [3] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
- [4] Pan Du, Meet Hemant Parikh, Xiantao Fan, Xin-Yang Liu, and Jian-Xun Wang. Confild: Conditional neural field latent diffusion model generating spatiotemporal turbulence. arXiv preprint arXiv:2403.05940, 2024.
- [5] Han Gao, Xu Han, Xiantao Fan, Luning Sun, Li-Ping Liu, Lian Duan, and Jian-Xun Wang. Bayesian conditional diffusion models for versatile spatiotemporal turbulence generation. Computer Methods in Applied Mechanics and Engineering, 427:117023, 2024.
- [6] Nicholas Geneva and Nicholas Zabaras. Transformers for modeling physical systems. Neural Networks, 146:272-289, 2022.
- [7] XU HAN, Han Gao, Tobias Pfaff, Jian-Xun Wang, and Liping Liu. Predicting physics in mesh-reduced space with temporal attention. In International Conference on Learning Representations, 2022.
- [8] Gerhard A Holzapfel. Nonlinear solid mechanics: a continuum approach for engineering science, 2002.
- [9] Christian Jacobsen, Yilin Zhuang, and Karthik Duraisamy. Cocogen: Physically-consistent and conditioned score-based generative models for forward and inverse problems. arXiv preprint arXiv:2312.10527, 2023.
- [10] Yayati Jadhav, Joseph Berthel, Chunshan Hu, Rahul Panat, Jack Beuth, and Amir Barati Farimani. Stressd: 2d stress estimation using denoising diffusion model. Computer Methods in Applied Mechanics and Engineering, 416:116343, 2023.

| [11]   | Sebastian Kaltenbach and Phaedon-Stelios Koutsourelakis. Incorporating physical constraints in a deep probabilistic machine learning framework for coarse-graining dynamical systems. Journal of Computational Physics, 419:109673, 2020.                                     |
|--------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [12]   | Sebastian Kaltenbach and Phaedon-Stelios Koutsourelakis. Physics-aware, probabilistic model order reduction with guaranteed stability. ICLR, 2021.                                                                                                                            |
| [13]   | Petr Karnakov, Sergey Litvinov, and Petros Koumoutsakos. Optimizing a discrete loss (odil) to solve forward and inverse problems for partial differential equations using machine learning tools. arXiv preprint arXiv:2205.04611, 2022.                                      |
| [14]   | Petr Karnakov, Sergey Litvinov, and Petros Koumoutsakos. Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks. PNAS nexus, page pgae005, 2024.                                                               |
| [15]   | George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422-440, 2021.                                                                                                |
| [16]   | Matt J Keeling and Ken TD Eames. Networks and epidemic models. Journal of the royal society interface, 2(4):295-307, 2005.                                                                                                                                                    |
| [17]   | Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013.                                                                                                                                                                     |
| [18]   | Phaedon-Stelios Koutsourelakis. Accurate uncertainty quantification using inaccurate computational models. SIAM Journal on Scientific Computing, 31(5):3274-3300, 2009.                                                                                                       |
| [19]   | Tianyi Li, Alessandra S Lanotte, Michele Buzzicotti, Fabio Bonaccorso, and Luca Biferale. Multi-scale reconstruction of turbulent rotating flows with generative diffusion models. Atmosphere, 15(1):60, 2023.                                                                |
| [20]   | Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020.                                    |
| [21]   | Marten Lienen, Jan Hansen-Palmus, David L¨dke, u and Stephan G¨nnemann. u Generative diffusion for 3d turbulent flows. arXiv preprint arXiv:2306.01776, 2023.                                                                                                                 |
| [22]   | Qiang Liu and Nils Thuerey. Uncertainty-aware surrogate models for airfoil flow simulations with denoising diffusion probabilistic models. AIAA Journal, pages 1-22, 2024.                                                                                                    |
| [23]   | Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3(3):218-229, 2021.                                          |
| [24]   | Emmanuel Menier, Sebastian Kaltenbach, Mouadh Yagoubi, Marc Schoenauer, and Petros Koumout- sakos. Interpretable learning of effective dynamics for multiscale systems. arXiv preprint arXiv:2309.05812, 2023.                                                                |
| [25]   | Robert D Moser. Numerical challenges in turbulence simulation. In Numerical Methods in Turbulence Simulation, pages 1-43. Elsevier, 2023.                                                                                                                                     |
| [26]   | Tim Palmer. Modelling: Build imprecise supercomputers. Nature, 526(7571):32-33, 2015.                                                                                                                                                                                         |
| [27]   | Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Timo Ewalds, Andrew El-Kadi, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Gencast: Diffusion-based ensemble forecasting for medium-range weather. arXiv preprint arXiv:2312.15796, 2023. |
| [28]   | Maziar Raissi, Paris Perdikaris, and George E Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378:686-707, 2019.      |

| [29]   | Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to- image diffusion models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-36494, 2022.   |
|--------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| [30]   | Dule Shu, Zijie Li, and Amir Barati Farimani. A physics-informed diffusion model for high-fidelity flow field reconstruction. Journal of Computational Physics, 478:111972, 2023.                                                                                                                                                  |
| [31]   | Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020.                                                                                                               |
| [32]   | Giovanni Stabile and Gianluigi Rozza. Finite volume pod-galerkin stabilised reduced order methods for the parametrised incompressible navier-stokes equations. Computers & Fluids, 173:273-284, 2018.                                                                                                                              |
| [33]   | Pantelis R Vlachas, Georgios Arampatzis, Caroline Uhler, and Petros Koumoutsakos. Multiscale simula- tions of complex systems by learning their effective dynamics. Nature Machine Intelligence, 4(4):359-366, 2022.                                                                                                               |
| [34]   | Pantelis R Vlachas, Wonmin Byeon, Zhong Y Wan, Themistoklis P Sapsis, and Petros Koumoutsakos. Data-driven forecasting of high-dimensional chaotic systems with long short-term memory networks. Proc. R. Soc. A, 474(2213):20170844, 2018.                                                                                        |
| [35]   | Zhong Yi Wan, Ricardo Baptista, Anudhyan Boral, Yi-Fan Chen, John Anderson, Fei Sha, and Leonardo Zepeda-N´˜ez. un Debias coarsely, sample conditionally: Statistical downscaling through optimal transport and probabilistic diffusion models. Advances in Neural Information Processing Systems, 36, 2024.                       |
| [36]   | Sifan Wang, Hanwen Wang, and Paris Perdikaris. Learning the solution operator of parametric partial differential equations with physics-informed deeponets. Science advances, 7(40):eabi8605, 2021.                                                                                                                                |
| [37]   | Henry GWeller, Gavin Tabor, Hrvoje Jasak, and Christer Fureby. Atensorial approach to computational continuum mechanics using object-oriented techniques. Computers in physics, 12(6):620-631, 1998.                                                                                                                               |
| [38]   | David C Wilcox. Multiscale model for turbulent flows. AIAA journal, 26(11):1311-1320, 1988.                                                                                                                                                                                                                                        |
Han Gao Sebastian Kaltenbach Petros Koumoutsakos
Generative Learning of the Solution of Parametric Partial Differential Equations Using Guided Diffusion Models and Virtual Observations
We introduce a generative learning framework to model high-dimensional
parametric systems using gradient guidance and virtual observations. We
consider systems described by Partial Differential Equations (PDEs) discretized
with structured or unstructured grids. The framework integrates multi-level
information to generate high fidelity time sequences of the system dynamics. We
demonstrate the effectiveness and versatility of our framework with two case
studies in incompressible, two dimensional, low Reynolds cylinder flow on an
unstructured mesh and incompressible turbulent channel flow on a structured
mesh, both parameterized by the Reynolds number. Our results illustrate the
framework's robustness and ability to generate accurate flow sequences across
various parameter settings, significantly reducing computational costs allowing
for efficient forecasting and reconstruction of flow dynamics.